We have developed phase interferometric particle imaging (PHIPI) of Mie scattering to simultaneously measure the spherical and transparent droplet size at the micron scale and transient size changes at the nanoscale. The theoretical derivation of PHIPI reveals that the phase of interferometric fringes between direct transmission and reflection linearly shifts with the change in droplet size. After a proof-of-concept validation with simulation, a PHIPI system is then implemented with an elaborately designed Fourier imaging system and applied to measure single evaporating droplets of ethanol and n-Nonane. The results show that the PHIPI technique can determine the size change down to several nanometers, providing a powerful tool for accurate in-situ characterization of droplet dynamics, e.g., droplet evaporation and oscillation.