Artificial Intelligence (AI) can sometimes produce inaccuracies known as hallucinations and omissions. These are recognised challenges in both developing and using AI in social care, and they have the potential to affect trust if not carefully managed.

In social care, where supporting people is always the priority, understanding these behaviours is especially important. That is why we have carried out thorough testing within Liquidlogic AI (FormFlow), working to keep these occurrences to a minimum.

We aim to help social care professionals spend more time focusing on care while ensuring they remain involved at every step.

Understanding Hallucinations in Social Care AI Solutions

In social care settings, AI hallucinations refer to situations where the system produces content in a transcript or summary that was not part of the original conversation between the practitioner and the citizen.

What to Look For:

  • Misunderstanding of spoken words due to differences in background, language use, or regional accents
  • Difficulty recognising social cues, behaviour, or visible signs of stress
  • Added phrases that may unintentionally suggest something not clearly stated
  • Drawing conclusions that were not mentioned in the conversation
  • Making assumptions that are not supported by the interaction

Example:

Insensitive hallucination: A summary may state that a citizen is at a particular risk, even though this was not discussed during the assessment

Omissions in Social Care AI Solutions

Omissions occur when important details from an interaction are not captured in the AI-generated content.

What to Look For:

  • Key issues, findings, or decisions discussed but missing from the summary
  • Loss of context, such as body language, reactions, or tone of voice
  • Missing information about home life, environment, or social context

Examples:

  • Omitted context: Important background details are present in the transcript but are not reflected in the summary
  • Omitted decisions: The transcript may not fully capture decisions or may present situations that are not suitable due to missing context

Discover Liquidlogic FormFlow AI

Built in partnership with Microsoft and securely hosted on Azure, FormFlow is designed to work quietly in the background as a supportive tool. Using AI-powered functionality, the solution records interactions via a mobile app, turning them into structured text and helping to auto-fill forms for practitioners to review.

It supports practitioners’ decision-making, recognises patterns, and helps with daily tasks, so documentation becomes a more natural part of delivering care.

Processes

FormFlow has been carefully designed to avoid creating information that has not been explicitly provided.

The solution follows strict human-created guidelines, including:

  • Do not ever diagnose citizens
  • Do not ever generate feedback
  • Do not ever learn from previous conversations or interactions to shape the model

We believe that trained professionals should always retain responsibility for decisions around care and support. FormFlow is built to reflect this, giving practitioners greater control and helping to maintain high standards of care.

Minimising Hallucinations and Omissions in FormFlow

FormFlow has been designed with both safety and quality in mind. Protecting vulnerable people is central to its purpose, and the solution supports smoother conversations while helping citizens access the support they need.

Targeted prompts are built into the system, and any transcriptions can be reviewed and amended. This reflects our human-first approach, where AI acts as support rather than a replacement.

Why We Do What We Do

To help ensure accuracy and reliability, all AI-generated content produced during an assessment or consultation is reviewed by a practitioner before it is finalised.

This approach keeps oversight at the heart of the process, allowing practitioners to check details, correct any issues, and ensure the safety and well-being of citizens before any actions are taken.

As a trusted AI partner, we are committed to developing safe and responsible solutions. Supporting both practitioners and citizens remains our highest priority, and we continue to refine our approach to reduce hallucinations and omissions as much as possible.

Keeping people at the centre of the process ensures that practitioners stay in control, helping them deliver the best possible care.

Next Steps

Find out how Liquidlogic AI enhances our social care software, supporting both children’s social care and adults’ social care software with intelligent, embedded automation. It helps teams streamline processes, improve data quality and focus more time on delivering better outcomes.

Explore our extensive collection of blogs to stay informed, inspired and ahead of the curve. Whether you are seeking expert advice, industry trends or thought-provoking perspectives, we have something for everyone.

Please get in touch with our friendly team of experts today. We’re here to assist you with any questions or concerns you may have.