Third Workshop on Safety for Conversational AI

Location, format, and schedule

The third Safety for Conversational AI workshop will consist of up to three keynotes, a panel, and oral and poster paper presentations. The event will be hybrid (both in-person at LREC/COLING 2024 in Turin and remotely).

For the content of the poster sessions, we seek archival and non-archival submissions in the form of long, short, and positional papers (see Call for papers below). The topics for the panel will be informed by these submissions.

Schedule

All times in CET.

9:00-9:30: Welcome message from the organisers

9:40-10:30: Keynote 1

10:30-11:00: Coffee break 

11:00-13:00: Oral presentations

13:00-14:00: Lunch break

14:00-15:00: Keynote 2 

15:00-16:00: Keynote 3 

16:00-16:30: Coffee break

16:00-17:30: Round tables (TBC)

Call for papers

We look for regular or work-in-progress papers that report:

• Current research on identifying and/or preventing dialogue models from generating messages containing: hate speech, pornographic or sexual content, offensive or profane language, medical or self-harm advice, privacy leaks, or otherwise harmful behaviour.

• Work on distinguishing how dialogue may be different from written text documents, and its safety implications.

• Work on understanding how social biases are embedded in conversational AI and conversational agents, and approaches to mitigate the harms of such biases,

• Work on understanding safety implications in multimodal (e.g., vision and language or speech and language) setups.

• Qualitative or quantitative investigations on strategies for how models should best respond to various types of unsafe content,

• Relevant work in related fields, including HRI, virtual agents, etc.

• New methods and metrics for evaluating the relative safety of conversational AI models,

• New (dynamic) benchmarks and datasets to investigate these issues,

• Dataset releases that address current biases or imbalances in data used to train/evaluate dialogue systems,

• Conversational abuse detection and mitigation across multiple languages, dialects, and sociolects; spoken and typed interactions etc., this includes abusive, derogatory, or otherwise harmful speech, including but not limited to bullying, microaggressive behavior, toxic language, hate speech and offensive language.

• Reliably and robustly detecting safety critical situations,

• Related issues including privacy and privacy leaks, persona design, fairness, accessibility and inclusion for  Conversational AI systems,

• Best practices for researching and releasing E2E conversational AI,

• Position papers to reflect on the current state of the art in this topic, take stock of where we have been, where we are, where we are going and where we should go. 

Submission Formats

• Long papers (8 pages excluding references) must describe substantial, original, completed, and unpublished work. Wherever appropriate, concrete evaluation and analysis should be included. These papers will undergo the same peer-review process by the LREC/COLING program committee as papers submitted to the main LREC/COLING tracks. An unlimited number of pages is allowed for references. The submission can be accompanied by an appendix, which will appear in the main paper’s PDF, after the references, and can be of unlimited pages. The final version is allotted an extra page to address reviewers’ comments.

• Short papers (4 pages excluding references) must describe original and unpublished work. All rules for long papers apply. Please note that a short paper is not a shortened long paper. Instead, short papers should have a point that can be made in a few pages, such as a small, focused contribution, a negative result, or an interesting application nugget. An unlimited number of pages is allowed for references. The submission can be accompanied by an appendix, which will appear in the main paper’s PDF, after the references, and can be of unlimited pages. One additional page is allotted in the final version of the paper to address reviewers’ comments.

• Late-breaking and Work-in-progress abstracts (2 pages excluding references, non-archival) will showcase ongoing work and focused relevant contributions. Submissions need not present original work. The submissions will be reviewed by the program committee and posted on the special session website. Abstracts will be presented as lightning talks and/or posters during the session. Late-breaking and Work-in-progress will not be part of the workshop proceedings. An unlimited number of pages is allowed for references. The submission cannot be accompanied by an appendix.

Instructions For Double-Blind Review

We will adopt a double-blind reviewing process. 

As reviewing will be double-blind, papers must not include authors’ names and affiliations. Furthermore, self-references or links (such as GitHub) that reveal the author’s identity, e.g., “We previously showed (Smith, 1991) …” must be avoided. Instead, use citations such as “Smith previously showed (Smith, 1991) …” Papers that do not conform to these requirements will be rejected without review. Papers should not refer, for further detail, to documents that are not available to the reviewers. For example, do not omit or redact important citation information to preserve anonymity. Instead, use third person or named reference to this work, as described above (“Smith showed” rather than “we showed”). If important citations are not available to reviewers (e.g., awaiting publication), these paper/s should be anonymised and included in the appendix. They can then be referenced from the submission without compromising anonymity. Papers may be accompanied by a resource (software and/or data) described in the paper, but these resources should also be anonymized.

Submission Instructions

• Papers: Submit papers via this link. 

• Late-breaking and Work-In-Progress abstracts: Submit abstracts via this link. Please specify the paper type submission as "Abstract".

The general LREC/COLING 2024 submission and formatting guidelines apply. For any questions, use Slack to reach out to the workshop organizers.