Laura Johnson, Boban Stanojevic
Myplanet, Toronto, Canada

As developers and content editors, we often feel the urge to take shortcuts when entering image captions. It’s time consuming! (And weirdly kind of hard sometimes to come up with the right words.) But what if you had an option to have your caption automatically generated with AI? Well that would be very nice, indeed.

In this session, we’ll showcase a proof of concept using an AI-driven image captioning API from IBM to suggest image captions when uploading images to the media module. AI image captioning could be extremely helpful, but it’s never going to be as good as human image captioning (and often gives hilariously bad suggestions), which is why our POC includes a user prompt to review the caption before saving.

We’ll demo the proof of concept and show the code behind it, talk about the possible benefits to accessibility using this approach, as well as possible downsides, and we’ll give an overview of the elements behind the AI model and talk about considerations when choosing an AI model for integrations. We’ll also talk about how the same method can be adapted as a CKEditor plugin for inline images.

What you’ll learn:
How we integrated the AI Captioning API with Media
Accessibility benefits and possible pitfalls
Elements behind the AI Captioning model
Considerations when choosing an AI model
Possible variations and improvements to make in the future