Word embeddings as autonomous predictors in materials design-the effect of inherent variability on information...

We propose that word embeddings of atoms derived from scientific literature are revisited as autonomous machine learning predictors in materials design. If static word embeddings encode comprehensive physicochemical information, joined embeddings of chemical elements constituting a chemical compound represent a viable source of physicochemical knowledge. Nevertheless, static word embeddings are susceptible to variability due to information heterogeneity within training material. We analysed whether variability occurs in embeddings affiliated with physicochemical entities, including explicit atoms, and whether it affects therein-encoded domain-specialized information or inhibits the information transfer. Results demonstrate the substantial variability in individual atomic embeddings, which is highly dependent on vocabulary terms selected for language modelling. Regardless, variability does not obstruct the mapping of materials' composite predictors into physicochemical properties when joined atomic embeddings are implemented within a regression model estimating the compound stability by predicting its formation energy. Moreover, the encoded information and the model's predictive performance maintained stability following compound vector calibration via dimensional reduction.Scientific contributionThe magnitude of variability in word embeddings of physicochemical entities, including chemical elements, occurring due to information heterogeneity in complementary training material of materials science, chemistry, and physics scientific literature was observed and quantified. The research shows that notable variability of vectorial representations of chemical elements does not obstruct the underlying statistical properties, nor does it inhibit the information transfer. Accordingly, regardless of their origin, conjoined atomic embeddings representing chemical compounds facilitate stable predictive performance when implemented within a regression model.