Learning Endlessly, asking for a friend (AI edition)

[nathaliesmith]

Well, I’m going to spew, slightly. Only because I think the benefit of the forum is being able to be honest and ask for help.

From my lens, I’m starting to feel like the hardest problems with AI are operational.

The growth is insane, the outputs are generally helpful… But what keeps feeling messy (at least where I’m sitting) is everything around it:

When should I challenge or override it?
Who actually owns its behavior? (Me, right?)
How do we even define quality anymore?
How do I add AI to every layer to an org, in hopes its going to 10x outcomes?

I’ve caught myself realizing that I’m not struggling with the tools as much as I’m struggling with the habits around them. The team norms. The judgment calls. The evolution of the work that seemingly has to happen?

Still learning in public over here. Curious how others are navigating the human side of this shift?

[Bryan]

This is where @menno was highlighting the opportunity to figure out a framework:

What does human-in-the-loop really mean in practice? (Q&A)

No, I was more thinking related with the “in the loop”, “on the loop”, “above the loop” etc… Like if trust is high, focus on “on or above” or even “behind” the loop. Or the greater the task size, the more likely it will be “in” the loop.. So a decision model for “where to put the human”. This may very well vary per line of business and or use case though. But there must be generic principles, like when the tasks go into the 10s of thousands, you want to see a report. When there are few but “e…

It gets really interesting once you attach this to autonomous agents!

[nathaliesmith]

Bringing this back up to the top with a new heavy hitting question. We’re actively trying to improve and automate some of our workflows.

Where is everyone finding the most success in automating their workflows with AI right now?

[ben]

@nathaliesmith

Where is everyone finding the most success in automating their workflows with AI right now?

The hard part is that automating workflows sort of also relies on the capabilities of the models at this stage.

I think they’ll get better, making automating with them a bit easier (check out Claude Code).

What’s interesting too is that a lot of automation doesn’t even have to be done with AI. Really what AI is for most tasks, is a decision maker for more ambiguous asks (where a simple if/else will not suffice).