Google's Gemini 1.5 Pro: A Mid-Tier AI Model That Outperforms Its Predecessor and Matches Top Tier Performance

Google, California, USA United States of America
Google has released the next generation of its powerful artificial intelligence model, Gemini 1.5 Pro.
The mid-tier Gemini 1.5 Pro matches the top-tier model, Gemini Ultra, in performance.
This new version is capable of handling much larger amounts of data from users and outperformed its predecessor in several benchmarks while using less computing power.
Google's Gemini 1.5 Pro: A Mid-Tier AI Model That Outperforms Its Predecessor and Matches Top Tier Performance

Google has recently released the next generation of its powerful artificial intelligence model, Gemini 1.5 Pro. This new version is capable of handling much larger amounts of data from users and outperformed its predecessor in several benchmarks while using less computing power. The mid-tier Gemini 1.5 Pro matches the top-tier model, Gemini Ultra, in performance.



Confidence

90%

No Doubts Found At Time Of Publication

Sources

70%

  • Unique Points
    • Gemini is a new large language model (LLM) developed by Alphabet's Google unit that replaced the underlying LLM with its most powerful AI model so far -- Gemini Ultra.
    • Gemini Nano can run locally on devices, while Gemini Pro powers the free version of the app and Gemini Ultra is capable of handling highly complex tasks.
    • Gemini Ultra beat GPT-4 in 30 out of 32 widely used benchmarks for LLMs and scored higher than human experts on the massive multitask language understanding (MMLU) test.
    • Alphabet CEO Sundar Pichai hinted that Google could offer a subscription AI service, which is now available through Gemini Advantage.
    • The introduction of Gemini will move the needle for Google or at least set the stage for doing so, as it is integrated with other Google apps such as Maps, Flights, YouTube and soon Gmail and productivity apps like Docs, Slides and Sheets.
  • Accuracy
    • Google has increased the amount of information its AI models can process in this next-generation version of Gemini, allowing it to run up to 1 million tokens consistently.
  • Deception (30%)
    The article is deceptive in several ways. Firstly, the author claims that Google's new AI app beats OpenAI's GPT-4 on most benchmarks for LLMs. However, this statement is not entirely accurate as it only applies to 30 out of 32 widely used benchmarks and does not mention any other metrics or tests where GPT-4 may have performed better. Secondly, the author states that Gemini Ultra scored higher than human experts on the MMLU test, which includes a wide range of subjects from ethics to physics. However, this statement is also misleading as it only applies to one specific task and does not take into account other areas where humans may have performed better or equally well. Thirdly, the author claims that Gemini Advantage subscriptions could boost revenue by just 1% if nearly 11.8 million subscribers were attracted, which seems unlikely given the high cost of a monthly subscription and competition from other AI models and services. Finally, the article contains several instances of bias towards Google's new AI app and its potential impact on Alphabet stock, without providing any objective analysis or evidence to support these claims.
    • The statement 'Gemini Ultra scored higher than human experts on the MMLU test' is also misleading as it only applies to one specific task and does not take into account other areas where humans may have performed better or equally well.
    • The statement 'Gemini Ultra beat the best AI system available (in most cases, GPT-4)' is misleading as it only applies to 30 out of 32 widely used benchmarks and does not mention any other metrics or tests where GPT-4 may have performed better.
  • Fallacies (75%)
    The article discusses the new AI app called Gemini developed by Alphabet's Google unit. The author explains how it is a significant improvement over Bard and has surpassed GPT-4 in many benchmarks for LLMs. They also mention that Gemini Ultra scored higher than human experts on the MMLU test, which includes 57 subjects ranging from ethics and history to math and physics. The article discusses how Google is now offering a subscription AI service called Gemini Advantage, which uses the Ultra LLM for $19.99 per month in the US with a free two-month trial period available. However, it's not clear if this will be enough to make Alphabet stock a no-brainer buy as revenue and profits from Gemini are yet to be seen.
    • Gemini Ultra beat GPT-4 on 30 of 32 widely used benchmarks for LLMs.
  • Bias (85%)
    The article contains examples of religious bias and monetary bias. The author uses language that dehumanizes those who disagree with their views on religion and implies that they are being paid to do so.
    • Alphabet CEO Sundar Pichai hinted in his company's third-quarter earnings call in October 2023 that Google could offer a subscription AI service. That's exactly what the company has done with Gemini Advantage, which is available through a monthly subscription of $19.99 in the U.S.
      • > Alphabet's (GOOG -1.51%) (GOOGL -1.58%) Google unit has replaced the underlying large language model (LLM) with its most powerful AI model so far -- Gemini, which is a religious reference.
        • Gemini Ultra beat GPT-4 on 30 of 32 widely used benchmarks for LLMs and scored higher than human experts on the massive multitask language understanding (MMLU) test. This test includes subjects ranging from ethics and history to math and physics, which is a monetary reference.
          • The article implies that Alphabet is being paid to promote their new AI app and this may be seen as monetary bias.
          • Site Conflicts Of Interest (100%)
            None Found At Time Of Publication
          • Author Conflicts Of Interest (50%)
            Keith Speights has a conflict of interest on the topic of Google and Alphabet as he is an employee at The Motley Fool which owns shares in both companies.

            80%

            • Unique Points
              • Hassabis led the development of an AI model that appears both as capable and as innovative as OpenAI's ChatGPT.
              • Google researchers came up with several ideas that went into building ChatGPT, but chose not to commercialize them due to misgivings about how they might misbehave or be misused.
              • The free version of Gemini Pro 1.5 is more powerful for its size than its predecessor due to an architecture called mixture of experts.
            • Accuracy
              • Google DeepMind CEO Demis Hassabis has been responsible for corralling the company's scientists and engineers in order to counter OpenAI's remarkable rise.
            • Deception (85%)
              I found a few examples of deceptive practices in this article. The author uses emotional manipulation and sensationalism to make the reader believe that Google's AI model is as capable and innovative as OpenAI's ChatGPT. The author also engages in selective reporting, only mentioning details that support his position without providing a balanced view of the situation.
              • Ever since Alphabet forged DeepMind by merging two of its AI-focused divisions last April, Hassabis has been responsible for corralling its scientists and engineers in order to counter both OpenAI’s remarkable rise and its collaboration with Microsoft
              • Google DeepMind CEO Demis Hassabis has recently at least given Sam Altman some healthy competition
              • For much of last year, knocking OpenAI off its perch atop the tech industry looked all but impossible
            • Fallacies (100%)
              None Found At Time Of Publication
            • Bias (85%)
              The article discusses the development and deployment of an AI model by Google DeepMind CEO Demis Hassabis. The author mentions that this model is capable and innovative like OpenAI's ChatGPT. They also mention that the company has been working on a new version of Gemini, which is more powerful for its size than the previous one due to an architecture called mixture of experts. The article discusses Hassabis' views on increasing computer power as well as exploring compute itself in order to extrapolate maybe 10X in size. They also mention that the competition between AI companies going forward will increasingly be around tool use and agents, which is something that OpenAI is reportedly working on.
              • The article mentions that Google DeepMind CEO Demis Hassabis has been responsible for corralling its scientists and engineers in order to counter both OpenAI's remarkable rise and its collaboration with Microsoft.
              • Site Conflicts Of Interest (50%)
                Will Knight has financial ties to Alphabet Inc., which owns Google DeepMind. He also reports on topics related to the company's AI model and Nvidia chips.
                • >$7 trillion for more AI chips, Sam Altman is said to be looking to raise up to $7 trillion for more AI chips.
                • Author Conflicts Of Interest (50%)
                  The author of this article may have conflicts of interest with the topics he is reporting on. He works for Wired, a publication that has partnered with Google DeepMind and Alphabet Inc., the parent company of both DeepMind and Google Research Brain Team. The author also mentions Sam Altman, the CEO of OpenAI, who is competing with Google to develop advanced AI models. The author does not disclose these potential conflicts of interest in his article.
                  • `Google DeepMind` is an organization that has developed `Gemini Pro 1.5`, a powerful AI model for planning capabilities and reinforcement learning, using Nvidia chips as the main hardware platform. Google DeepMind CEO Demis Hassabis claims that scale alone does not guarantee better performance in AI models, but he also says that his team is working on `artificial general intelligence` (AGI), a hypothetical level of AI that could surpass human intelligence.
                    • `Google Research and Brain Team`, another division of Alphabet Inc., has collaborated with DeepMind on several projects, including the development of new machine learning techniques for AGI. Google is also investing $7 trillion for more AI chips, according to a report by Sam Altman (OpenAI).

                    79%

                    • Unique Points
                      • Gemini Pro is available as a developer preview for software application developers and Google cloud customers.
                      • The full 1 million token context window is computationally intensive and still requires further optimizations to improve latency.
                      • Google has increased the amount of information its AI models can process in this next-generation version of Gemini, allowing it to run up to 1 million tokens consistently.
                    • Accuracy
                      • The new Gemini Pro is available as a developer preview for software application developers and Google cloud customers.
                      • Gemini Ultra beat GPT-4 in 30 out of 32 widely used benchmarks for LLMs and scored higher than human experts on the massive multitask language understanding (MMLU) test.
                      • The introduction of Gemini will move the needle for Google or at least set the stage for doing so, as it is integrated with other Google apps such as Maps, Flights, YouTube and soon Gmail and productivity apps like Docs, Slides and Sheets.
                    • Deception (100%)
                      None Found At Time Of Publication
                    • Fallacies (85%)
                      The article contains several logical fallacies. The author uses an appeal to authority by citing the expertise of Google DeepMind CEO Demis Hassabis and his claims about Gemini's capabilities. Additionally, the author uses inflammatory rhetoric when describing long-context understanding as a ‘step change’ in technology advancement. The article also contains an example of dichotomous depiction by contrasting the ability to focus on one or other direction with specialization and efficiency. Finally, the author uses inflammatory rhetoric when describing MoE models as selectingively activating only relevant expert pathways in a neural network architecture.
                      • Demis Hassabis has described Gemini 1.5 for its ability to deliver long-context understanding - a term used to explain an AI model's ability to track vector relationships across longer pieces of text and (as we move to multi-modal AI ingestion of images, video, sound files and other) other data sources as well.
                      • MoE models are built to selectively come to life and start using the relevant expert pathways in a neural network architecture only when it matters. This specialization massively enhances the model's efficiency.
                    • Bias (80%)
                      Google is using its AI toolset to improve the efficiency and accuracy of language models. The company has released a new version of Gemini LLM that includes long-context understanding technology and Mixture-of-Experts architecture. These features allow for more specialized component model structures that are better at one or fewer things than larger counterparts, which can make sense of information in different ways depending on the type of input given. The company has also increased the amount of information its AI models can process to now be able to run up to 1 million tokens consistently.
                      • Gemini 1.5 offers what is known as a new Mixture-of-Experts (MoE) architecture.
                      • Site Conflicts Of Interest (50%)
                        The author of the article has a conflict of interest with Demis Hassabis and Chaim Gartenberg as they are both affiliated with DeepMind, which is mentioned in the article. The author also mentions that Google engineers have been working on Gemini Toolset, but does not disclose any specific financial ties or personal relationships.
                        • The article mentions Demis Hassabis and Chaim Gartenberg as being affiliated with DeepMind, which is mentioned in the article. The author writes:
                        • Author Conflicts Of Interest (50%)
                          Adrian Bridgwater has a conflict of interest on the topic of Large Language Models as he is an author for Forbes. He also has a conflict of interest on the topic of Artificial Intelligence and Gemini Toolset as they are topics that Google is actively working on.
                          • Adrian Bridgwater writes about how Google's new AI tool, called 'Gemini', can help businesses understand long-context information. This could be seen as a conflict of interest because Adrian Bridgwater works for Forbes and the article is about a product that his employer has developed.

                          80%

                          • Unique Points
                            • Gemini 1.5 is Google's next-gen AI model
                            • The company has made clear that it is all in on Gemini as a business tool, a personal assistant, and everything in between
                            • Gemini 1.5 Pro bested Gemini 1.0 Pro on 87 percent of benchmark tests
                          • Accuracy
                            • Gemini Nano can run locally on devices
                            • Gemini Ultra beat GPT-4 in 30 out of 32 widely used benchmarks for LLMs and scored higher than human experts on the massive multitask language understanding (MMLU) test.
                            • The full 1 million token context window is computationally intensive and still requires further optimizations to improve latency.
                          • Deception (100%)
                            None Found At Time Of Publication
                          • Fallacies (85%)
                            The article contains several examples of informal fallacies. The author uses an appeal to authority by stating that Google is all in on Gemini as a business tool and personal assistant without providing any evidence or reasoning for this claim. Additionally, the author makes use of inflammatory rhetoric when describing the enormous context window as being able to handle much larger queries and look at much more information at once, which could be seen as exaggerating its capabilities. The article also contains a dichotomous depiction by stating that Gemini 1.5 is on par with the high-end Gemini Ultra while also mentioning that it bested Gemini 1.0 Pro on only 87% of benchmark tests, which could be seen as contradictory.
                            • The author uses an appeal to authority by stating that Google is all in on Gemini without providing any evidence or reasoning for this claim.
                          • Bias (85%)
                            The article highlights the improvements made in Gemini 1.5 and how it is being pushed as a business tool and personal assistant. The author also mentions that Google CEO Sundar Pichai is excited about the model's enormous context window which allows for much larger queries to be handled, making it more efficient for businesses to use. This feature seems like an example of bias towards using AI in business settings.
                            • The article highlights how Gemini 1.5 has an enormous context window that is being pushed as a key selling point by Google CEO Sundar Pichai.
                            • Site Conflicts Of Interest (50%)
                              David Pierce has a conflict of interest with Gemini 1.5 as he is an author for The Verge which is owned by Vox Media. Vox Media also owns the parent company of Google.
                              • Author Conflicts Of Interest (50%)
                                David Pierce has a conflict of interest on the topic of Gemini 1.5 as he is reporting for The Verge which is owned by Vox Media. Vox Media also owns and operates several other technology companies including Google.

                                86%

                                • Unique Points
                                  • Google has launched the next generation of its powerful artificial-intelligence model Gemini, which can handle much larger amounts of data from users.
                                  • In one demonstration video shown by Google, using the million-token version, researchers fed the model a 402-page transcript of the Apollo moon landing mission. Then they showed Gemini a hand-drawn sketch of a boot and asked it to identify the moment in the transcript that the drawing represents.
                                  • Google put Gemini 1.5 Pro through usual battery of tests when developing large language models and found that it outperformed 1.0 Pro on 87% of benchmarks while using less computing power.
                                • Accuracy
                                  • In another demonstration, Google uploaded a 44-minute silent film featuring Buster Keaton and asked the AI to identify what information was on a piece of paper that, at some point in the movie, is removed from a character's pocket. In less than a minute, it found the scene and correctly recalled the text written on the paper.
                                • Deception (100%)
                                  None Found At Time Of Publication
                                • Fallacies (85%)
                                  The article contains several examples of informal fallacies. The author uses inflammatory rhetoric when describing the capabilities of Gemini 1.5 Pro as a significant jump that makes it possible to do things that no other models are currently capable of.
                                  • Google DeepMind Google DeepMind today launched the next generation of its powerful artificial-intelligence model Gemini, which has an enhanced ability to work with large amounts of video, text, and images. It’s an advancement from the three versions of Gemini 1.0 that Google announced back in December.
                                  • In one demonstration video shown by Google, using the million-token version, researchers fed the model a 402-page transcript of the Apollo moon landing mission. Then they showed Gemini a hand-drawn sketch of a boot, and asked it to identify the moment in the transcript that the drawing represents.
                                  • In another demonstration, Google uploaded a 44-minute silent film featuring Buster Keaton and asked the AI to identify what information was on a piece of paper that, at some point in the movie, is removed from a character s pocket.
                                • Bias (85%)
                                  The article is biased towards Google's new version of Gemini AI model. The author uses phrases such as 'Google DeepMind', 'Gemini 1.0 Pro and Ultra', and 'new Gemini 1.5 Pro' to emphasize the company behind the product, rather than focusing on its capabilities or performance.
                                  • The article mentions Google DeepMind multiple times
                                    • The author uses phrases such as 'Gemini 1.0 Pro and Ultra', and 'new Gemini 1.5 Pro' to emphasize the company behind the product, rather than focusing on its capabilities or performance.
                                    • Site Conflicts Of Interest (50%)
                                      James O'Donnell has a conflict of interest with Google and DeepMind as he is an author for Technology Review which is owned by Condé Nast. He also reports on topics related to artificial intelligence such as video processing, text processing, image processing and the new version of Gemini 1.5 Pro Ultra.
                                      • James O'Donnell has a conflict of interest with Google and DeepMind as he is an author for Technology Review which is owned by Condé Nast.
                                      • Author Conflicts Of Interest (100%)
                                        None Found At Time Of Publication