This study presents a novel statistical volume element (SVE) for micromechanical modeling of the white matter structures, with histology-informed randomized distribution of axonal tracts within the extracellular matrix. The model was constructed based on the probability distribution functions obtained from the results of diffusion tensor imaging as well as the histological observations of scanning electron micrograph, at two structures of white matter susceptible to traumatic brain injury, i.e. corpus callosum and corona radiata. A simplistic representative volume element (RVE) with symmetrical arrangement of fully alligned axonal fibers was also created as a reference for comparison. A parametric study was conducted to find the optimum grid and edge size which ensured the periodicity and ergodicity of the SVE and RVE models. A multi-objective evolutionary optimization procedure was used to find the hyperelastic and viscoelastic material constants of the constituents, based on the experimentally reported responses of corpus callosum to axonal and transverse loadings. The optimal material properties were then used to predict the homogenized and localized responses of corpus callosum and corona radiata. The results indicated similar homogenized responses of the SVE and RVE under quasi-static extension, which were in good agreement with the experimental data. Under shear strain, however, the models exhibited different behaviors, with the SVE model showing much closer response to the experimental observations. Moreover, the SVE model displayed a significantly better agreement with the reports of the experiments at high strain rates. The results suggest that the randomized fiber architecture has a great influence on the validity of the micromechanical models of white matter, with a distinguished impact on the model’s response to shear deformation and high strain rates. Moreover, it can provide a more detailed presentation of the localized responses of the tissue substructures, including the stress concentrations around the low caliber axonal tracts, which is critical for studying the axonal injury mechanisms.