The fragmentation dynamics of aggregate of non-Brownian particles in shear flow is investigated numerically. The breakup behaviors of aggregates having the same connectivity but the different space-filling properties are examined. The Lagrangian particle simulation in a linear flow field is performed. The effect of surrounding fluid on the motion of multiple particles is estimated by Stokesian dynamics approach. The inter-particle force is calculated from the retarded van der Waals potential based on the Lifshitz theory. The results obtained in this work indicate that the fragmentation behavior of colloidal aggregates depends on their fractal structure. However, if the resultant aggregate size is smaller than the critical one, the fragmentation behavior shows the universality regardless of their original structure. Furthermore, the restructuring of aggregate in shear flow and its effect on the fragmentation process are also discussed.