GroundLM 2026 Shared Tasks

Shared Tasks

GroundLM 2026 will host shared tasks that turn grounding challenges into concrete, reproducible evaluations. The public datasets, input-only test sets, and online evaluator are now available. Participants should register their team on the evaluator Space, submit held-out predictions there, and report the official results in their system paper.

GoldenViewVQA LitTraceQA How to Participate Paper Instructions Shared Tasks Submission Discord Q&A
Overview

Task Format

Shared tasks are intended to complement the workshop CFP by giving participants a common evaluation setup for grounded language-model behavior. Each task provides public development material, submission-format guidance, and held-out evaluation data for final ranking. Participants can ask questions in the GroundLM Discord.

Shared Task 1

GoldenViewVQA

Multi-view visual question answering with evidence-source identification in driving scenes.

Input Six synchronized camera views + question
Output Supporting view + answer choice
Public dev set 55 labeled examples
Primary metric Joint accuracy
View Task Dataset Evaluator
Shared Task 2

LitTraceQA

Literature-grounded question answering with paper retrieval, evidence grounding, and answer generation.

Input Research question + paper metadata pool
Output Paper IDs + evidence + answer
Public dev set 55 labeled examples
Evaluation Retrieval, grounding, and answer accuracy
View Task Dataset Evaluator
Participation

How to Participate

Use the Hugging Face datasets for the public data and input-only test splits, then submit prediction files to the official evaluator Space. The evaluator is the source of record for held-out scores and leaderboard rankings.

Step 1 Sign in to the evaluator Space with a Hugging Face account
Step 2 Register one immutable team name and contact email
Step 3 Upload JSONL predictions for the relevant held-out split
Step 4 Report official evaluator results in the system paper

Official Evaluator Tasks

  • GoldenViewVQA: submit the dataset test split to the evaluator task named goldenviewvqa.
  • LitTraceQA required track: submit the dataset test split to littraceqa-test.
  • LitTraceQA optional diagnostic track: submit test-extra to littraceqa-test-extra. This is recommended but not required for shared-task participation.

Submission Rules

  • Each Hugging Face account, email address, and team name can be registered only once; registration is not editable after submission.
  • Each registered account may submit up to five times per evaluator task per day. The three evaluator tasks have separate daily limits, counted by the evaluator server day in America/Toronto time.
  • Files that fail sanity checks, such as malformed JSONL, missing required ids, duplicate ids, or unexpected ids, do not count toward the daily limit.
  • The leaderboard is separated by task and displays the best successful submission for each team. Contact emails are collected for organizers and are not shown on the leaderboard.
  • Use exactly the same team name on the evaluator, in the system paper title/results, and in the OpenReview teamname field.
Open Online Evaluator GoldenViewVQA Dataset LitTraceQA Dataset Discord Q&A
Timeline

Shared Task Dates

The schedule below applies to the GroundLM 2026 shared tasks. The input-only test sets and online evaluator are available now. Participants should sign in to the evaluator Space, register a team, and may submit up to five times per task per day.

ActivityDate
Dataset (dev) release June 8, 2026 (Shared Task 1); June 15, 2026 (Shared Task 2)
Dataset (test) release Released August 5, 2026
Submission of test output, code, and system papers August 19, 2026, AoE
Acceptance and winner announcement September 1, 2026
Camera-ready papers September 10, 2026
Final Submission

Final System Paper Submission Instructions

Participants in the GroundLM 2026 shared tasks are invited to submit final system description papers together with their test outputs and code. These papers should document the submitted system, experimental setup, official results, and lessons learned during participation. Papers are welcome regardless of leaderboard position, including submissions that focus on negative results, limitations, or detailed error analysis.

Participants are encouraged to begin drafting the system description before the test-set release, then update the official results and analysis after evaluation.

Who may submit Teams participating in one or both shared tasks
Submission due August 19, 2026, AoE
Paper length 4-8 pages of main content, plus references
Format Official ACL two-column style

Recommended Content

  • Abstract and introduction: summarize the task, system, main methods, and key results.
  • Data and preprocessing: describe the released data, input processing, prompts, augmentation, and any external resources.
  • System architecture and training: report models, retrieval or grounding pipelines, prompting strategies, fine-tuning methods, checkpoints, hyperparameters, frameworks, and compute resources.
  • Experimental results: include official test-set results from the online evaluator, all reported leaderboard metrics for each submitted task, development-set results where relevant, submitted-run comparisons, model variants, and ablations when available.
  • Error analysis: discuss representative failures, difficult examples, grounding errors, retrieval mistakes, evidence-selection issues, and differences between development and test performance.
  • Conclusion, limitations, and ethics: summarize findings, future work, known limitations, and responsible-use considerations for grounded multimodal or literature-grounded systems.

Title and Submission Materials

The paper title should use the team name from the shared-task submission, for example: 

<Team Name> at GroundLM 2026 Shared Tasks: <Paper Title>

Use the same team name when registering on the evaluator Space, reporting leaderboard results in the system paper, and filling the teamname field in OpenReview. Teams participating in both GoldenViewVQA and LitTraceQA are encouraged to describe both systems in a single paper. Final shared-task submissions should include the paper PDF, the JSONL test-output files uploaded to the evaluator, code or reproducibility materials, all external data and models used, and the information needed by the organizers for the shared-task overview paper.

Before Submitting

  • Use the official ACL style files without modifying margins, spacing, fonts, or page dimensions.
  • Keep the system description paper within 4-8 pages of main content; references may use additional pages.
  • Include author names, affiliations, and contact information in the final system paper.
  • Clearly disclose external datasets, pretrained models, tools, APIs, and generated or synthetic data.
  • Confirm that the submitted PDF opens correctly and that tables, figures, and links render as intended.
Submit System Paper ACL Style Files ACL PubCheck
Shared Task 1

GoldenViewVQA: Multi-View Grounded VQA for Driving Scenes

GoldenViewVQA evaluates whether multimodal systems can identify the visual evidence needed to answer a question. Given six synchronized NuScenes camera views and a question, a model must select the supporting camera view and answer the multiple-choice question.

Input Six synchronized NuScenes camera views + one question
Output Supporting view and answer choice
Public dev set 55 labeled examples, 330 image references
Primary metric Joint accuracy

Evaluation

Submissions should contain one JSON object per question with a question identifier, predicted supporting view, and predicted answer choice. 

{"question_id": "sfall_0001_causality", "predicted_view": "CAM_FRONT", "predicted_answer_id": "A"}

The development evaluator reports micro and macro view accuracy, answer accuracy, and joint accuracy. The official online evaluator is now available for the held-out input-only test set with private gold labels. For GoldenViewVQA, submit predictions for the test split through the evaluator Space after registering your team.

Dataset and Test Release

The Hugging Face dataset contains the public development data and the input-only test split for official evaluation. Use the development split for task familiarization, data loading, prompt and model development, local evaluation, and submission-format checks; use the test split for leaderboard submissions.

Resources

Submit to Shared Tasks Dataset on Hugging Face Online Evaluator Schema Evaluator Validator

The dataset provides benchmark annotations and NuScenes-relative image paths. Participants must obtain NuScenes through the official access process and comply with its terms of use.

Shared Task 2

LitTraceQA: Literature-Grounded Question Answering

LitTraceQA evaluates whether systems can answer research questions with grounded evidence from scientific papers. Given a question and requested answer types, a system must retrieve the relevant paper or papers from the released metadata pool, identify coarse evidence locations, and return the final answer in the requested format.

Input Research question, answer types, and paper metadata pool
Output Paper IDs, evidence locations, and final answer
Public dev set 55 labeled validation examples
Primary components Retrieval, evidence grounding, and answer accuracy

Evaluation

Submissions should contain one JSON object per question with a stable query identifier, retrieved paper IDs, supporting evidence locations, and the answer object. 

{"query_id": "q_001", "gold_papers": [{"paper_id": "acl2025_00005"}], "evidence": [{"paper_id": "acl2025_00005", "source_type": "table", "locator": {"page": 6, "table_id": "Table 4"}}], "answer": {"freeform": {"text": "14.70"}}}

The public evaluator reports macro paper retrieval precision, recall, and F1; macro evidence precision, recall, and F1 using coarse evidence matches; and answer metrics for multiple-choice, freeform, and table outputs. The official online evaluator is now available for the hidden test labels. Register your team on the evaluator Space, submit the required test split there, and optionally submit test-extra for additional diagnostic results.

Dataset and Test Release

The Hugging Face dataset includes gold validation records, input-only validation records, input-only test records, input-only test-extra records, sample submissions, a searchable paper metadata pool, a machine-readable schema, and a local evaluator. The test split is required for LitTraceQA shared-task participation. The test-extra split is recommended for additional diagnostic evaluation, but it is not required. Official scores for both splits are produced by the online evaluator Space.

Resources

Submit to Shared Tasks Dataset on Hugging Face Online Evaluator Format Evaluation Evaluator

LitTraceQA annotations and benchmark files are released under CC BY-NC 4.0. Paper metadata remains subject to the original publishers' terms, and PDFs are not redistributed in the dataset.