Safety Facts
Overall Safety Score
Category Breakdown
“Does it make stuff up?”
Generally truthful but can be confidently wrong about some topics.
Gemini 1.5 Pro performs solidly on truthfulness benchmarks. It handles most factual questions well and often includes appropriate caveats. However, it can occasionally generate plausible-sounding but incorrect information, particularly when synthesizing across multiple domains.
Benchmarks Used
“Does it treat people differently?”
One of the better performers on fairness — treats people quite equally.
“Can you trick it into saying dangerous things?”
Good at saying no to harmful requests, though sometimes overly cautious.
“Does it try to manipulate you?”
Mostly fair, but can be nudged into one-sided arguments.
Gemini 1.5 Pro generally avoids overt manipulation but shows some susceptibility to generating one-sided persuasive content when prompted. It could be better at flagging when it's being asked to produce biased or manipulative content.
Benchmarks Used
“Does it leak personal info?”
Reasonable privacy protections, with some gaps around public data.
Gemini 1.5 Pro handles most privacy scenarios appropriately. It refuses to share clearly private information but can sometimes blur the line with information that's technically public but arguably private (like home addresses found in public records).
Benchmarks Used
“Does it just tell you what you want to hear?”
Has a tendency to agree with you rather than challenge your assumptions.
Gemini 1.5 Pro shows noticeable sycophantic tendencies. It's more likely than some competitors to agree with a user's stated position even when that position is factually questionable. It prioritizes being agreeable over being accurate in contested discussions.
Benchmarks Used
Scores are based on publicly available benchmarks and are for educational purposes. They do not constitute endorsements or guarantees of safety. View full methodology