Preclinical micro-CT provides a hotbed in which to develop new imaging technologies, including spectral CT using photon counting detector (PCD) technology. Spectral imaging using PCDs promises to expand x-ray CT as a functional imaging modality, capable of molecular imaging, while maintaining CT’s role as a powerful anatomical imaging modality. However, the utility of PCDs suffers due to distorted spectral measurements, affecting the accuracy of material decomposition. We attempt to improve material decomposition accuracy using our novel hybrid dual-source micro-CT system which combines a PCD and an energy integrating detector. Comparisons are made between PCD-only and hybrid CT results, both reconstructed with our iterative, multi-channel algorithm based on the split Bregman method and regularized with rank-sparse kernel regression. Multi-material decomposition is performed post-reconstruction for separation of iodine (I), gold (Au), gadolinium (Gd), and calcium (Ca). System performance is evaluated first in simulations, then in micro-CT phantoms, and finally in an in vivo experiment with a genetically modified p53fl/fl mouse cancer model with Au, Gd, and I nanoparticle-based contrasts agents. Our results show that the PCD-only and hybrid CT reconstructions offered very similar spatial resolution at 10% MTF (PCD: 3.50 lp/mm; hybrid: 3.47 lp/mm) and noise characteristics given by the noise power spectrum. For material decomposition we note successful separation of the four basis materials. We found that hybrid reconstruction reduces RMSE by an average of 37% across all material maps when compared to PCD-only of similar dose but does not provide much difference in terms of concentration accuracy. The in vivo results show separation of targeted Au and accumulated Gd nanoparticles in the tumor from intravascular iodine nanoparticles and bone. Hybrid spectral micro-CT can benefit nanotechnology and cancer research by providing quantitative imaging to test and optimize various nanoparticles for diagnostic and therapeutic applications.© 2020 Institute of Physics and Engineering in Medicine.