How Do Designers Design AI-centric UIs? (And should they?)

[ben]

@EricZ has done some interesting work putting together UI interfaces for tools that produce AI outputs. But now the question is, how do designers also put in their piece when it comes to the design of the agents themselves (and should they?).

There are three main areas that designers can have a significant amount of impact:

1) Structured Outputs

Defining structured outputs is important to condensing information that is too complex to grasp at a text-only level. It not only provides easy to summarize visuals, but also removes a layer of complexity from the agent’s output, increasing the overall quality by reducing required context (if done right).

2) Creating Transparency

Designers are well seasoned for understanding what should and shouldn’t be visible to the user. Tool calls in an AI system are very significant for the AI output. While these tool calls are running, how should they be displayed to the user? Should the user be able to see the results? Should they be shown up-front or hidden behind a dropdown?

3) The Interface

A designer’s bread and butter. A solid interface is a game-changer for working with AI. If the application is built around project level work, it should be primarily be oriented around that idea, making it as intuitive as possible to users.

A note that UIs within the chats themselves are very dynamic, which I’m sure a few designers would find super fun and complex to work with. Gaining understanding of the lower-level technical systems will only help designers’ capabilities as they know what sort of snags developers will run into when designing out these systems.

[EricZ]

Yes! There’s a real balance to be struck here. On the one hand, consistent styling can make it clear what is being generated by the LLM or brand and what is an asset injected by the user.

On the other hand, providing flexibility can allow the LLM to excel by shaping requests into novel outputs.

OR, we could just let it do what Gemini does today and feed the user tables in a different format or markup each time.

[ben]

Yeah, it’s all a really interesting game that we’re having to play. One thing that I know is that we’ll keep learning how to do this best.

Our experimentation has already given us a significant amount of knowledge that we can keep shaping into higher quality outputs, which I think is the most important part. Reducing hallucinations and providing structured patterns that users can recognize.

Those both are going to play into something extremely solid that I don’t think other tools are going to even try do.