1. Introduction

Accurate reproduction (visualization) of modern automobile paints enables the modeling and evaluation of a car’s desired color. The appearance of car paint is highly complex. It depends on the direction of lighting, the viewing angle, as well as the distribution of sparkle texture on the car body [1]. The problem of correct visualization of paints is the subject of numerous studies. For example, the authors of [2–4] propose visualization models based on the analysis of photographic images or the results of measurements of real paint samples. Fig. 1 demonstrates the visualization of the designed paints on a virtual car model. The paint modeling program and realistic visualization system used to generate the images in Fig. 1 were developed at the Keldysh Institute of Applied Mathematics RAS.

 

Fig. 1. Visualization of the designed car paints on a virtual car model.

 

To reproduce the car paint appearance, it is possible to model the propagation of light in paint layers with a complex microstructure. Such modeling is a challenging task and does not always provide the required accuracy. A paint model consisting of plane-parallel homogeneous layers was proposed in [5]. Based on a statistical approach it accurately describes the interaction of light inside the paint, including reflections on aluminum flakes and pearlescent effects on interference plates. This model turned out to be quite realistic and was subsequently used and developed in [6–8].

Modern car paints have a complex structure containing various pigments, such as interference platelets, mirror-like flakes, and conventional sub-wavelength pigments, suspended in a transparent binder. The appearance of the paint is characterized by its color, brightness, surface gloss, spatial heterogeneity (texture), etc. While the bidirectional reflectance distribution function (BRDF) describes the reflective properties of painted surfaces, it cannot capture spatial texture, which is particularly important for metallic paints.

Modeling of metallic paints containing large reflective flakes is considerably more challenging than conventional diffuse paints composed solely of light-scattering pigments. This complexity stems from both use of various computational methods and difficulties in obtaining accurate input parameters. When modeling diffuse paints, continuous medium calculation methods have proven themselves well, while for metallic paints the accuracy of these methods may be insufficient. Real-world aluminum flakes used in metallic paints may not be flat, but have a complex shape of multilayer flakes. Their surface may be polished and then reflect the light ray in a specular manner, but it may also be rough with a complex reflection function. Since these properties are frequently unknown, and therefore not all of them can be taken into account in modeling.

One of the most complex while practically necessary tasks is determining paint composition (i.e. a set of flakes, pigments and their concentrations) that provides a given appearance, namely color and texture. This inverse problem is particularly relevant for automotive repairs when the original paint formulation is unavailable. Moreover, even if the original paint composition is known, the appearance of the painted surface can change over time under the influence of atmospheric phenomena, reagents used on the roads, etc. Additional factors include differences in pigment batches, a variety of painting technologies and working conditions. As a result, the appearance of the surface may differ from the original. Several research teams tried to find an automatic solution to this very complex problem [9–12]. We also proposed general solution based on simulation of light propagation in paint layers [13, 14] and worked out its various aspects [15–17]. We achieved reasonable color matching when visualizing measured real paint samples and paints calculated using our model. However, the colors were visually distinguishable for some paints indicating the problem is not yet fully solved.

In this paper, we propose a hybrid approach to address the challenge of determining the paint composition that achieves target color specifications. The approach integrates artificial intelligence (AI) techniques that provide reasonable results for paints and pigments with substantial available training data, and physics-based light propagation simulations in the paint layer for novel or insufficiently characterized pigments entering the market.

2. AI based technique

Currently the company "Dva stakhanovtsa” (“Two Stakhanovites") [18] has achieved notable success in practical applications. They have developed an AI-based solution for the problem of paint color matching. The program they developed, StartColor [19], automatically determines the composition of a paint based on visual appearance and spectral measurements obtained using a mobile spectrophotometer XRite T12 [20] or its equivalent BYK-mac i [21]. The system demonstrates exceptional matching accuracy for numerous paint formulations (Fig. 2).

 

Fig. 2. Each photo shows a sample with the target paint (left) and a sample painted with the composition found by the SartColor program (right).

 

While the concept of using a neural network to solve a color matching problem is well-established [22], implementing this technology for automotive paints presents unique technical challenges. The core of the SartColor program is a fully connected multilayer perceptron (MLP), as shown in Figure 3. The input layer processes standardized paint formulation data. This layer can be customized for the specific model of weight scales used in the auto body shop. The model incorporates three hidden layers, each with one hundred neurons. These layers remain unchanged. The output layer generates comprehensive data including spectral color values and paint texture characteristics of the paint. The output layer can be customized with adaptable parameters to accommodate various spectrophotometer models.

 

Fig. 3. Multilayer perceptron in the StartColor program.

 

The key competitive advantage lies in the proprietary database meticulously compiled by “Two Stakhanovites" company over several years of operation. This database documents successful color matching cases from actual automotive refinishing scenarios. Unlike the manufacturer-provided databases that exclusively feature idealized laboratory conditions and do not take into account practical variables including differences in pigment batches, variety of painting technologies and working conditions the “Two Stakhanovites" (StartColor) database inherently takes these factors into account. The database scale varies according to multiple parameters including paint formulations, application methods, and measurement equipment specifications. For example, the dataset comprises approximately 61,000 samples for spectrophotometers featuring five measurement geometries.

StartColor neural network undergoes a multi-stage training process. In the first stage, the database is partitioned into five distinct subsets. Four of these subsets serve sequentially for model training, with each training session comprising 300 epochs utilizing standard parameters followed by an additional 300 fine-tuning epochs with reduced learning rates. The network employs backpropagation throughout all training epochs. The fifth part functions as a validation set, enabling identification and removal of the input data of statistical outliers. The second stage involves final model training on the cleaned data. A concluding adaptation phase of 150 epochs optimizes the network for specific production environments and hardware configurations.

The StartColor database undergoes continuous expansion through its operational architecture. The SartColor software is hosted on a centralized server infrastructure processing requests from networked Colorist’s Workplaces (CW) via Internet connections (Fig. 4). Currently, approximately 150 workstations are deployed across multiple urban locations, handling over 1000 daily queries. For each request the StartColor program performs comprehensive data validation, prepares neural network-compatible input parameters, and generates preliminary paint formulations. Successful matching cases are systematically incorporated into the growing knowledge base. To maintain optimal performance with this expanding dataset, the StartColor neural network undergoes complete monthly retraining.

 

Fig. 4. Scheme of the StartColor software usage.

 

The effectiveness of the AI methodology depends primarily on the training database. As a case in point, the “Two Stakhanovites" company initially developed their AI solution using the paints from the PPG company [23] which employs standardized pigments and painting technology. In this controlled environment the calculation of paint composition achieved an 85% success rate. However, training databases created on the basis of pigments from one company frequently demonstrate limited effectiveness for paints produced from pigments from another company. Given the current market landscape with numerous pigment producers, developing comprehensive training datasets for each manufacturer would require impractical time investments. This limitation has motivated our proposed modeling approach for novel pigments, utilizing light propagation simulation within paint structures to address color matching challenges.

3. Paint modeling method: coloring the metallic paint

In metallic paints, aluminum particles are mainly responsible for brightness and texture and coloring diffuse pigments for color. As already mentioned, texture cannot be conveyed by a single BRDF function. From practice it is known that there are a limited number (about dozen) of different types of aluminum flakes used in car paints. This allows us to divide the color matching task for the metallic car paint into two stages: separately select the type of aluminum particles (for example, recognize using AI techniques) and then simulate the color change using coloring pigments. In this formulation, for the second stage, the final BRDF is the criterion for achieving the result, since the issue of the correctness of the texture is resolved at the first stage. It should be noted that the diffuse pigment is added in a small concentration (usually a few percent); metallic particles predominate in the paint.

In our research we consider the BRDF of the surface for the angles of incidence and observation that can be obtained using the spectrophotometer T12 [20]. This also allows us to compare the results of our modeling with the measured data of real paint samples specially prepared according to known compositions. Two components of metallic paint were selected for the experiments: aluminum particles TYM24 and diffuse coloring pigment TJM51 from the MARIPOSA pigment series [24]. Several samples with different ratios of aluminum flakes and coloring pigment concentrations were prepared and measured.

As the analysis of the measurements showed, the spectrum of pure metallic without coloring pigment is almost flat (i.e. the brightness coefficient is almost independent of the wavelength), and the addition of dye leads to noticeable non-uniformity of this spectrum (Fig. 5a). To see how the dye changes color, we take the ratio

,

where is the spectrum of colored metallic and is the spectrum of pure metallic. This value is close to 0 for low dye concentrations and a strong variation is obtained for higher concentrations.

Let us normalize this expression:

.

If we now superimpose the normalized curves for different pigment concentrations, they almost coincide. They are also close to the extinction spectrum of the dye normalized similarly (Fig. 5b). The coincidence is not ideally precise but quite good; too good to be accidental.

 

Fig. 5. (a) The luminance spectrum of pure TYM24 and its mixtures with TJM51 at different concentrations. The vertical axis is the luminance, the horizontal axis is the wavelength (nm). Different colors of the graphs correspond to different pigment concentrations. The black solid, almost flat line, corresponds to the spectrum of pure metallic. (b) Transformation of the normalized spectrum for different concentrations of TJM51 dye. The dashed line is the similarly normalized extinction spectrum of the TJM51 pigment.

 

Since the shape of the spectrum is almost fixed, it can be uniquely specified by the value . Let's call it the "degree of non-uniformity". It increases with the dye concentration. As can be seen from Fig. 6, the "span" of the spectrum change increases linearly with the dye concentration with high accuracy. The shape of the spectrum approximately coincides with the normalized extinction spectrum.

 

Fig. 6. “Degree of non-uniformity” as a function of TJM51 concentration.

 

The observed behavior (constancy of the normalized spectrum and linear dependence of its “span” on concentration) is characteristic of the single-collision model. In this model the light ray changes its spectrum as it travels to the aluminum particle due to absorption by the dye. Then it changes its spectrum when it reflects from not perfectly white aluminum (in Fig. 5a, you can see that the reflectance spectrum of pure metallic slightly drops off toward the edges of the range). And finally, it changes its spectrum as it travels from the aluminum particle to the surface of the dye layer. We derived the formula for the single-collision model BRDF in [25] on the basis of a complete model of the interaction of light rays with a layer of flakes distributed in a color binder. Thus, it is formally possible to predict the BRDF of this paint based on its composition.

The formula represents the colored paint BRDF in the form of scaled uncolored BRDF . This representation allows simplifying the color matching for metallic paints. In practice, predicting the change of the color is more accurate than calculating the full BRDF. The main reason is that the most important parameter that affected the BRDF of a paint layer is the flake orientation distribution. It is almost always unknown. But we can make some plausible assumptions about this distribution, change its parameters in a reasonable range and see which properties of the resulting BRDF remain stable with respect to this change. We can then use these characteristics in a situation of uncertainty of the flake orientation distribution because the scaling factor is completely independent of it. We can use this result as an approximation for real flakes. As tests show, this is a good approximation. As a result, we predict the paint color not from scratch, but as a scaling of the BRDF for a pure metallic.

So it is a hybrid model where some terms are calculated and some are measured or obtained by AI techniques. Moreover, the scaling is a simple algebraic function and can be easily calculated while the full BRDF calculation requires much more time-consuming ray tracing.

4. Validation of the metallic paint coloring method

To perform the tests we need to determine the paint pigment characteristics as plausibly as possible. There are characteristics that are known explicitly, such as pigment concentrations. Some, such as dye extinction (i.e. attenuation of light by a dye) can be measured using the approach described in [15]. But many parameters, especially the flake orientation distribution, are hardly possible to measure directly. For this distribution we tried to find the most suitable values that would ensure agreement between the measured and calculated BRDF of a paint consisting only of metal flakes. For other parameters, such as flake thickness, we made some reasonable assumptions consistent with micrographs, manufacturer data, known measurements, etc. To obtain the parameters of aluminum particles, we used several samples with high concentrations of TYM24 flakes. They were measured on goniospectrophotometer [26] developed at the Keldysh Institute of Applied Mathematics and on a T12 device [20]. We assumed that the flakes are perfectly flat thin aluminum disks uniformly distributed in the paint layer, and their orientation is random; its distribution is rotationally symmetric. We have the following parameters of the aluminum flakes:

∙ concentration;

∙ radius;

∙ thickness;

∙ flake surface reflectivity;

∙ flake orientation distribution.

They are not exactly known, but we know some realistic boundaries from the manufacturers' data, from microphotographs, from measurements of the reflectivity of pure polished aluminum. Then we varied the above parameters within realistic limits and observed how the calculated BRDF changes, whether it approaches the measured values or deviates from them. We did not perform a real optimization by the descent method, we just found some option that looks better than the others.

The TYM24 is a pigment paste, not a powder of pure aluminum flakes. In the paste, the flakes are dispersed in a binder with the addition of solvents and other volatile components. The flakes themselves are not the main component there. We conducted several experiments to remove the binder and obtained a PVC (Pigment Volume Concentration) value of about 13% relative to dry TYM24. We used this PVC in calculations. A more or less realistic value of 0.5 microns was taken for the particle thickness. The radius of the flakes we estimated from a microphotograph of a layer of highly dissolved TYM24. In the computational experiments we used three realistic radiuses: 5, 10, and 20 microns. The reflectivity of the flake was taken as reflectivity of polished aluminum. The Gaussian distribution we  chose for the flake orientation.

To check the selected parameters, we calculated the BRDF of the paint with the only TYM24 flakes. The calculated result is close to the measured paint sample containing only TYM24 particles.

We assumed that the diffuse pigment obeys the continuous medium model and its scattering can be approximated by the Henyey-Greenstein phase function [27]. Thus, the pigment is characterized by extinction, scattering and the asymmetry parameter g. To measure them we developed and tested a method [17], which gave very good results. For the experiments we used the TJM51 dye and a sample of a colored metallic consisting of TYM24 flakes and TJM51 pigment.

Six samples with different ratios of aluminum flake to diffuse pigment concentrations were prepared and measured (Table 1). The concentration (PVC) of pigments in the dry paint was calculated based on the initial composition values in the paste (PWC, Pigment Weight Concentration, first two columns of Table 1) as described in [17]. The flake concentration (right column) is simply the TYM24 concentration multiplied by 0.13 (the expected flake volume fraction in dry TYM24).

Table 1. Concentrations of TYM24 aluminum flakes and TJM51 dye pigment in test samples.

A comparison of the results of calculations of the BRDF of the colored metallic by our methods with the measured data for all samples is shown in Fig. 7. The solid line shows the measured spectra, the dotted line shows the result calculated according to our model. Different colors of the graphs correspond to two different observation angles. Thus, for the considered samples we see a good agreement between the calculated and measured spectrograms.

 

Fig. 7. The measured spectra (solid lines) with spectra calculated according to our model (dashed lines). Vertical axis shows luminance factor, horizontal axis shows wavelength (nm).

5. Conclusion

We propose a hybrid approach to solving the problem of color matching for automotive metallic paints. It integrates AI technologies with modeling of metallic paint coloring with diffuse particles. For modeling, an original method for calculating the color of metallic paint based on its composition was developed and tested. Using the proposed method, we predict the paint color not from scratch but as a scaling of the BRDF for pure metallic. Thus, this is a hybrid model in which some terms are calculated and some are obtained using AI technologies or measured. At the same time, if we apply our method of the calculated scaling to the measured base BRDF, the resulting spectra are predicted with quite acceptable accuracy (Fig. 7). Our method requires significantly fewer computing resources, or in other words, much faster (fractions of a second instead of several minutes for an accurate calculation) than a traditional ray tracing. This creates good opportunities for using this method to solve the optimization problem of color matching, where the color calculation is performed multiple times.

References

1. Takagi, A., Watanabe, A., Baba, G., Prediction of Spectral Re?ectance Factor Distribution of Automotive Paint Finishes, Color Research & Application, 30(4), 2005, pp. 247-318. doi:10.1002/col.20125

2. Rump, M., Muller, G., Sarlette, R., Koch, D., Klein, R., Photo-realistic Rendering of Metallic Car Paint from Image-Based Measurements, Computer Graphics Forum, 27(2), 2008, pp. 527–536. doi:10.1111/j.1467-8659.2008.01150.x

3. Kneiphof, T., Klein, R., Real-time image-based lighting of metallic and pearlescent car paints, Computers & Graphics, 105, 2022, pp. 36–45. doi:10.1016/j.cag.2022.04.009

4. Kim, D.B., Seo, M.K., Kim, K.Y., Lee, K.H., Acquisition and representation of pearlescent paints using an image-based goniospectrophotometer, Optical Engineering, 49(4), 2010, 043604. doi:10.1117/1.3407431

5. Ershov, S., Kolchin, K., Myszkowski, K., Rendering pearlescent appearance based on paint-composition modeling, Computer Graphics Forum, 20(3), 2001, pp. 227–238. doi:10.1111/1467-8659.00515

6. Jakob, W., d'Eon, E., Jakob, O., Marschner, S., A comprehensive framework for rendering layered materials, ACM Transactions on Graphics, 33(4), 2014, Article No. 118, pp. 1–14. doi:10.1145/2601097.2601139

7. Belcour, L., Efficient Rendering of Layered Materials using an Atomic Decomposition with Statistical Operators, ACM Transactions on Graphics, 37(4), 2018, pp. 1. doi:10.1145/3197517.3201289

8. Guo, J., Chen, Y.J., Guo, Y.W., Pan, J.G., A physically-based appearance model for special effect pigments, Computer Graphics Forum, 37(4), 2018, pp. 67–76. doi: 10.1111/cgf.13476

9. Ergun, S., Onel, S., Ozturk, A., A general micro-flake model for predicting the appearance of car paint, EGSR'16: Proceedings of the Eurographics Symposium on Rendering: Experimental Ideas & Implementations, 2016, pp. 65–71.

10. Golla, T., Klein, R., Interactive interpolation of metallic effect car paints, EG VMV'18: Proceedings of the Conference on Vision, Modeling, and Visualization, 2018, pp. 11–20. doi:10.2312/vmv.20181248

11. Wu, H., Dorsey, J., Rushmeier, H., Physically-based interactive bi-scale material design, ACM Transactions on Graphics, 30(6), 2011, pp. 1–10. doi:10.1145/2070781.2024179

12. Bati, M., Barla, P., Pacanowski, R., An inverse method for the exploration of layered material appearance, ACM Transactions on Graphics, 40(4), 2021, Article No. 176, pp. 1–15. doi:10.1145/3450626.3459857

13. Ershov, S., Kolchin, K., Myszkowski, K., Reverse engineering approach to appearance-based design of metallic and pearlescent paints, The Visual Computer, 20, 2004, pp. 586–600.

14. Voloboy, A., Ershov, S., Pozdnyakov S., Interactive modeling automotive paints, Proceedings of the 22 International Conference on Computer Graphics and Machine Vision, 2012, pp. 242–247.

15. Ershov, S., Voloboy, A.,Barladian, B., Sokolov, V., Potemin, I., Zhdanov, D., Light scattering in automotive paints: continuous medium approach vs correlations between particles, Proc. SPIE 11874, Illumination Optics VI, 1187408, 2021. doi:10.1117/12.2597166

16. Ershov S., Voloboy A., Galaktionov V., Simulation of Light Scattering in Automotive Paints: Role of Particle Size, Mathematics, 11(11), 2023, 2429. doi:10.3390/math11112429

17. Ershov, S., Voloboy, A., Pozdnyakov, S., Galaktionov, V., Extraction of optical characteristics of diffuse particles for modelling a dispersed medium, Light & Engineering, 32(2), 2024, pp. 86–98. doi:10.33383/2023-082

18. Two Stakhanovites, production and distribution of paints and polishing pastes, URL: https://2stn.ru/ (accessed on 15.07.2025).

19. StartColor, Excellent in Color, URL: https://startcolor.ru/ (accessed on 15.07.2025).

20. MA T12 Multi-angle Spectrophotometer. URL: https://www.xrite.com/-/media/xrite/files/literature/l10/l10-500_l10-599/l10-565_mat12_sellsheet/l10-565-ma-t12-sell-sheet-en.pdf (accessed on 11.03.2025).

21. Spectrophotometer BYK-mac i. URL: https://qualicont.ru/products/byk/bykmac/ (accessed on 15.07.2025).

22. Moon, J., Yang, G., Tae, H., A study on DNN-based practical model for predicting spot color, Applied sciences, 13(24), 2023, 13100. doi: 10.3390/app132413100

23. PPG. Automotive OEM Coatings. URL: https://www.ppg.com/en-US/autocoatings (accessed on 15.07.2025).

24. MARIPOSA pigments, TYM and TJM series. URL: https://www.paintbc.ru/TDS MARIPOSA/TJM TYM_Ñåðèÿ.pdf (accessed on 15.07.2025).

25. Ershov, S., Voloboy, A., A Simplified Method for Calculating Metallic Paint Color, Proceedings of the 34 International Conference on Computer Graphics and Machine Vision, 2024, pp. 225-237. doi:10.25206/978-5-8149-3873-2-2024-225-237

26. Letunov, A., Barladian, B., Galaktionov, V., Ershov, S., Voloboy, A., Zueva, E., Device for Measuring BDF – Spatial - Spectral Light - Scattering Properties of Surfaces, DAAAM International Scientific Book, Chapter 40, 2012, pp. 479-490. doi:10.2507/daaam.scibook.2012.40

27. The Henyey-Greenstein phase function. URL: https://www.astro.umd.edu/~jph/HG_note.pdf (accessed on 15.07.2025).