Microsoft, in collaboration with OpenAI, continues to integrate AI capabilities into its products and services while also developing specialized case-specific models. The latest addition to its AI lineup is Orca, an AI model unveiled by Microsoft Research. Orca stands out by learning through imitation of large language models, aiming to overcome the limitations of smaller models and mimic the reasoning processes of foundational models like GPT-4.
Orca and similar language models can be fine-tuned for specific tasks using large language models such as GPT-4. One advantage of Orca’s smaller size is its reduced computing resource requirements, enabling researchers to optimize and independently run their models without relying on a large data center.
Orca, with 13 billion parameters, is built upon the Vicuna model and is capable of learning explanations, step-by-step thought processes, and complex instructions with the assistance of GPT-4, which is rumored to have over one trillion parameters.
To promote progressive learning, Microsoft utilizes large-scale and diverse imitation data with Orca. Already, Orca has achieved a 100% improvement over Vicuna in complex zero-shot reasoning benchmarks like Big-Bench Hard (BBH). Moreover, the model boasts a 42% speed advantage over conventional AI models on AGIEval.
Despite its smaller size, Orca exhibits similar reasoning capabilities to ChatGPT on benchmarks like BBH. Additionally, it demonstrates competitive performance on academic examinations such as SAT, LSAT, GRE, and GMAT, although it falls short of GPT-4.
According to the Microsoft research team, Orca can learn from human-generated step-by-step explanations and more advanced language models. As the model continues to develop, it is expected to acquire improved skills and capabilities.
Microsoft’s commitment to integrating AI into its products and services, along with its partnership with OpenAI, showcases the company’s dedication to advancing AI research and delivering innovative solutions to its users. With Orca and other AI models, Microsoft aims to empower researchers and developers to create more efficient and specialized AI applications.
