OpenResearchAlpha

JMIR AI · 20 Nov 2026

Balancing Human and AI Features for Digital Autism Diagnostics from Home Videos: Human-In-The-Loop Machine Learning Study

Abdur Rasoolh-index 20, Kaitlyn Dunlap, Arman Husic, Aaron Kline, Parnian Azizian, Dennis P Wall, Peter Washingtonh-index 29

0
Citations
no database records one yet
0
Citations in 2026
so far
29
Leading author's h-index
Peter Washington, in QAI's scholar index
38
References
works it cites, from open citation data

Citations by database

Each database indexes a different set of journals, preprints and books, so they are shown side by side rather than blended.

  • OpenCitations0
  • Crossref0
  • Europe PMC0

Citations per year

The current year is still counting.

  • 2026 so far0

Abstract

Background Autism spectrum disorder (ASD or autism) diagnosis faces significant delays due to specialist shortages and subjective, clinic-based assessments. Although AI offers a potentially scalable solution via automated video analysis, current state-of-the-art deep learning models often fail to capture subtle, clinically nuanced social behaviors essential for reliable detection. Objective This study aimed to develop and evaluate a human-in-the-loop (HITL) framework that combines automated AI feature extraction with crowdsourced human annotation for autism classification on naturalistic home videos. We targeted low-confidence AI facial feature extraction outputs to be replaced by human features. We also sought to quantify the relationship between the confidence threshold used for swapping AI features with human features, the volume of human replacement of AI features, and the predictive diagnostic performance. Methods We implemented a HITL pipeline integrating deep learning models for facial emotion (7-class) feature extraction. A confidence threshold applied to the emitted probability of the AI feature extractor model was used to identify low-confidence facial features or those that the AI model was unable to make a prediction for (henceforth referred to as “unresolved”), which were then replaced by human “feature extraction” (ie, annotation) conducted by rigorously vetted crowd workers. We evaluated this approach on 100 high-quality videos (50 autism and 50 neurotypical) from the GuessWhat? digital therapeutic app. Performance was assessed across decision thresholds (0.1-0.9 in increments of 0.1) using sensitivity, specificity, F1-score, and area under the receiver operating characteristic curve (AUROC), with 5-fold cross-validation. Results Across decision thresholds, performance improved overall but was nonmonotonic at intermediate thresholds. At a threshold of 0.9, 59.7% (597/1000) of frames were replaced by human annotations, and mean sensitivity increased from 0.51 (SD 0.24) at a threshold of 0.1 to 0.66 (SD 0.10) at a threshold of 0.9, an absolute increase of approximately 15 percentage points. The HITL-enhanced multimodal model achieved an aggregate out-of-fold AUROC of 0.76, compared with 0.54 for the AI-only model. Conclusions In this proof-of-concept study, replacing AI features with human features tended to be associated with higher observed autism classification performance in this particular dataset. We targeted low-confidence and unresolved frames for human annotation. We note that these findings are exploratory and thus do not isolate the independent causal effect of human replacement or establish clinical diagnostic validity.

What the data shows

  • No database records a citation yet, as is usual for a paper this new.
  • Peter Washington has the highest h-index among the authors (29).

Graph

This paper in the middle, its authors and their most-cited papers below, the papers it cites to the left and the papers citing it to the right. The timeline lays every paper out by year.

Citing paper shares an authorAuthor without a profileSize shows citations, or h-index for authors

Select a node for its details. Drag to pan, scroll to zoom, drag a node to move it.

Start with

Authors (7)

Each author's standing in QAI's scholar index: the works their ORCID iD is attached to, and for anyone whose ORCID record names a lab OpenResearch follows, their arXiv preprints matched by name and a shared co-author. Names are as printed on the paper.

ProfileMost-cited paper

Cited by 0 papers

Who cites this paper, from open citation data, with Crossref's record of each; the 100 newest when there are more.

0 shown
No citing papers for this filter.

References (38)

What it cites, from open citation data, with Crossref's record of each.

arXiv
The Charleston Advisor · 2019
292

Sign in to see 18 more references

A free account opens the rest of the citing papers and references on every paper’s page.

Balancing Human and AI Features for Digital Autism Diagnostics from Home Videos: Human-In-The-Loop Machine Learning Study - QAI OpenResearch