Apple, a technology company known for its innovative products, has recently made strides in the field of artificial intelligence (AI) by releasing several open source large language models (LLMs). These models, named OpenELM (Open-source Efficient Language Models), are designed to run on-device rather than through cloud servers and are available on the Hugging Face Hub. The release of these LLMs comes ahead of the anticipated debut of iOS 18 at WWDC in June.
According to Apple, OpenELM offers similar performance to other open language models but with less training data. The company has provided the complete framework for training and evaluation of the language model on publicly available datasets, including training logs, multiple checkpoints, and pre-training configurations. This comprehensive release aims to empower and strengthen the open research community.
Apple's OpenELM database is available on Hugging Face. The models use a layer-wise scaling strategy that aims to improve accuracy and efficiency in the language models. Apple has released eight total OpenELM models, four of which were pre-trained using the CoreNet library, and four instruction tuned models.
Apple's move towards open source AI follows its release of an open-source reference PyTorch implementation of the Transformer architecture at WWDC 2022. The company has a long history of developing AI technologies in-house, including its neural processing unit (NPU) in iPhones since 2017, which is a dedicated AI accelerator hardware.
The release of OpenELM is significant as it marks Apple's entry into the generative AI space, where companies like Microsoft and Google have already established a presence with their offerings such as Copilot and ChatGPT respectively. The on-device nature of OpenELM sets it apart from these competitors, offering potential privacy and security benefits to users.
Apple's focus on open source AI is part of a larger trend in the industry towards greater collaboration and sharing of research findings. This approach allows for faster progress and more trustworthy results as multiple researchers can build upon each other's work.
In conclusion, Apple's release of OpenELM represents an important step forward for the company in the field of AI. The on-device language models offer similar performance to other open language models while providing greater privacy and security benefits to users. The comprehensive release of training materials and frameworks aims to empower and strengthen the open research community, paving the way for future advancements in AI technology.
