Abstract : We are interested in the large time behavior of the solutions to the growth-fragmentation equation. We work in the space of integrable functions weighted with the principal dual eigenfunction of the growth-fragmentation operator. This space is the largest one in which we can expect convergence to the steady size distribution. Although this convergence is known to occur under fairly general conditions on the coefficients of the equation, we prove that it does not happen uniformly with respect to the initial data when the fragmentation rate in bounded. First we get the result for fragmentation kernels which do not form arbitrarily small fragments by taking advantage of the Dyson-Phillips series. Then we extend it to general kernels by using the notion of quasi-compactness and the fact that it is a topological invariant.