Defining AI Metrics

Choosing metrics
, ,
1

As AI becomes a bigger part of our world, AI metrics also becomes a thing!

It seems that a lot of tech influencers have been diving into this trend- how are we determining how AI is perceived?

It also reminds me of our previous conversation with: Users, customers, Situations or People?

When can we also start to rope in AI as a part of the who that we’re defining?

Maybe we’ll never have to, according to @EricZ’s post: Jaron Lanier and LLM's for Humanity

It seems there are more and more articles coming up around this particular problem. Has anybody here taken a good stab at this yet?

2 Likes
2

Nice, here’s some UX metrics to test whether you’re hitting the right notes on each of those 5 key principles:

  1. Investment Loops → People value what they shape

    Metric: Intent and Desirability, which can reveal whether users next steps bring them back to engage with the AI and how likely they are to continue engaging

  2. Human Presence → First impressions = retention

    Metric: Sentiment will reveal whether your product is striking the right human feel with users or alienating prospective customers

  3. Social Proof & Familiarity → We trust what others already use

    Metric: Loyalty is ideal here, revealing how likely users are to promote your product to others, and easily comparable to the tried-and-tested NPS method

  4. Labour Illusion → Show the effort, not just the output

    Metric: When AI shows its work, Comprehension in that work is key. Use a Trust metric to reveal how confident users are in the work being shown.

  5. Progressive Disclosure → Earn trust by scaling intelligence gradually

    Metric: How to see if you’ve earned trust? Measure using the Trust metric and compare against other impressions that are elicited in users.

1 Like
3

I love this thought process @ben. It will be interesting to see how these intelligence metrics start to show themselves

1 Like
4

We should try to expand on these- would be cool to see a resource doc be put together on all of the AI measurement articles, blogs, resources, metrics, etc.

Might have to do this

5

I suggest we expand our UX metrics in Glare to include Intelligence metrics to better understand the user experience with AI products.

Curious about your thoughts @nikhil_mahen.

image
image1081×875 18.3 KB

1 Like