[Vcon] Ideas for Representing Conversations and Meetings

[Adam Sobieski <[e-mail]>]

IETF vCon Working Group,


Hello. My name is Adam Sobieski. I am new to the IETF vCon Working Group. I would like to introduce myself and to share a few ideas for discussion.

I am interested in use cases and scenarios including AI systems processing and analyzing (public-sector) meetings’ transcripts and minutes. Pertinent scenarios include man-machine Q&A and dialogue about (public-sector) meetings.

A recent and hopefully interesting sketch of mine includes a WebVTT-metadata-inspired JSON schema: https://github.com/w3c/webrtc-charter/issues/84 . As indicated in this rough-draft sketch, in addition to transcribed speech and attachments, transcripts and minutes might also include metadata, timed hyperlinks, and slideshow presentation events.

Metadata objects could be useful for attaching metadata to transcripts and minutes. Topics for such metadata would include, but not be limited to, meetings' transcribing parties, attendees, venues, enabling software tools, and hints for these software tools such as terminological domains, pronunciation lexicons, or participants' planned discussion topics. Metadata objects could also be reused to attach metadata to attachment objects.

Timed hyperlink objects could resemble attachment objects which refer to their contents by URLs. Timed hyperlinks could be intended for display and potential navigation by virtual attendees and audiences. Timed hyperlinks might be intended to accompany meetings’ recordings in transcripts and minutes and to be displayed starting at times and for durations.

Slideshow presentation events could involve inserting into transcripts and minutes when slideshow presentations started, changed slides, and concluded. Individual slides could be inserted into meetings' transcripts and minutes as images and/or by URLs (e.g., “files/slideshow.pptx#3”). With presentations' events and slides available in transcripts and minutes, multimodal AI systems could answer questions and engage in dialogue involving presentations and their slides’ contents.

Also, for discussion, could AI agents be parties in conversations' or meetings' transcripts and minutes? Could the vCon format be of use for representing man-machine conversations and meetings (e.g., AutoGen: https://github.com/microsoft/autogen)? Thank you.


Best regards,
Adam Sobieski
http://www.phoster.com


Bibliography

Golany, Lotem, Filippo Galgani, Maya Mamo, Nimrod Parasol, Omer Vandsburger, Nadav Bar, and Ido Dagan. "Efficient data generation for source-grounded information-seeking dialogs: A use case for meeting transcripts." (2024). [arXiv<https://arxiv.org/abs/2405.01121>] [GitHub<https://github.com/google-research-datasets/MISeD>]