The rapid development of intelligent transportation and autonomous driving technologies, driven by the Internet of Vehicles (IoV), faces significant challenges owing to data and system heterogeneity. These challenges stem from the multidomain nature of the IoV and threats such as data leaks and model-finding attacks, which complicate data processing and model training. To address these issues, in this study, we proposed federated multi-domain learning for IoV (FMD-IoV). FMD-IoV addresses data heterogeneity by employing clustered techniques to group similar viewpoints and multidomain machine learning to map diverse data types into a unified feature space. To address the system heterogeneity caused by diverse vehicle types, the framework introduces a similarity-based aggregation method and model weight de-regularization to enhance robustness and generalizability. Experimental results demonstrated that FMD-IoV reduced the mean square error (MSE) by 0.05 on the Synthia dataset and 0.13 on the CityScape dataset compared with the state-of-the-art methods. Moreover, it maintained or improved the MSE as the number of nodes increased, demonstrating its adaptability to complex scenarios and large-scale data. These results highlight the flexibility, resilience, and efficacy of FMD-IoV in multi-view data fusion within large-scale IoV environments.