Researcher guide

This guide is for researchers who want to run a study in the CCAIR Behavioral Lab. It covers what the lab is, how to get access, and what to do from a first draft to a downloaded dataset.

Both email links open your own email app with a prepared message. Messages go to the lab director's university inbox (bampel@gsu.edu). Nothing is sent from this page, and the lab does not store what you write.

What the lab is

The CCAIR Behavioral Lab runs browser-based studies for the Center for CyberAI Research at Georgia State University. Participants open a link, give consent, and complete one or more sessions in their browser. The lab records their answers and, for most study types, how they arrived at them: timing, hesitation, answer changes, and where the pointer went.

One engine runs every study. It handles sign-in, consent, sessions spread over several weeks, balanced assignment to conditions, surveys before and after each session, storage, hourly backups, exports, and a live self-test. A study is content that you edit on its admin page, plus a study type that decides what the item screen looks like. You do not write code to run a study of an existing type.

Run the lab on your own computer

You can build and try a study on your laptop before anything goes online. You need Node.js (version 20 or newer, the "LTS" download) and a copy of the lab's folder.

1. On a Mac, double-click Start CCAIR Lab.command in the scripts folder. On any computer you can instead open a terminal in the lab's folder and type npm run lab.
2. Your browser opens the lab console, already signed in. Keep the small terminal window open while you work; close it (or press Ctrl+C) to stop the lab.
3. Everything you create is saved in the .lab-data folder, so it is still there next time. An example five-arm study is waiting for you to explore.

Nothing needs installing beyond Node.js. To let the AI caller rephrase its lines naturally, create a file named .env.lab in the lab's folder containing one line, ANTHROPIC_API_KEY= followed by the lab's key. Without it the caller speaks your script word for word, which is also the right setting for a fully controlled study.

Getting access

1. Request researcher access. The message asks for your name, institution and email, a short description of the study, your IRB status, and what you need. It goes to the lab director.
2. If the lab owner agrees, you receive an invitation link by email. It works once and lasts 7 days.
3. Open the link, enter your name, and choose a password of at least 12 characters. An authenticator app (a 6-digit code at sign-in) is optional and recommended if you will download participant data.
4. Next time, sign in at /lab/signin/. You land on My studies, which lists the studies your account can open and your role in each.

Forgot your password? Email the lab (the sign-in page has a link) and the owner sends a reset link that lasts 24 hours. Lost your authenticator? Use one of the recovery codes you saved, or ask the owner to remove it. Account settings are at /lab/account/.

Roles

Your role is set per study. The owner can change it at any time.

The lab owner additionally creates studies, lists them on the lab page, and invites researchers.

Creating a study

1. In the lab console, under Start a new study, pick a template: a survey, an A/B or multi-arm experiment, audio clip judgments, a conversation with an AI voice caller, image and video ratings, a mouse-tracking task, or the five-arm vishing training study. Give it a name and click Create. (On the lab's website, the owner creates studies and gives you a role.) Studies start unlisted, so nobody finds them by accident.
2. Click Preview as a participant right away: the study opens in a new tab exactly as a participant sees it, as a test participant that is kept out of the real counts. Pick an arm to see that arm's version.
3. Open the study from My studies. On its admin page, Study content holds everything participants see: consent text, sign-up rules, sessions, items and questions, surveys, and the wording of every screen. Every save makes a numbered version, and an old version can be restored.
4. Write the consent text and tick "IRB-approved" only when it is. Until then participants see a banner that says the consent is a draft.
5. Run the live self-test (owner), then pilot (below).
6. When the protocol is approved and you are ready to recruit, ask the owner to list the study on the lab page, or give participants its link: https://bampel.com/lab/s/<study-id>.

Sliders, rankings, and matrices. Surveys can also use a slider over any range (there is no handle until the participant moves it, so nobody is nudged toward a default, and you can tell who skipped it), a ranking (options are put in order with Up and Down buttons, which work with the keyboard and screen readers), and a matrix (several statements with the same answers; each statement gets its own variable name and column).

Randomizing order. Open a question's Display logic and layout. Shuffle makes it trade places with the other shuffled questions on the same page; questions you do not tick stay where they are. Show the options in a random order shuffles its answers; tick the options that must stay at the end, such as "Other" or "None of the above". Each participant keeps one order for the whole session, and the order they saw is saved next to their answers.

Counterbalancing block order. In multimodal and vignette studies, Block order turns on a Latin square: each participant sees the session's blocks (or, in vignette studies, its items) in one of a set of orders in which every block appears equally often in every position and right after every other block. Orders are handed out evenly within each condition, the tab previews them, and the data say which order each participant had.

Copying and sharing a design. Clone (lab console, owners) makes a new study with the same design and media, new completion codes (enter them in the new study's panel), and no participants: use it for a second wave or a replication. To remove a study for good, archive it first, download its complete backup if you need one, then choose Delete permanently on the archived row and type the study's link name. Every participant record, recording, and backup copy is deleted, and it cannot be undone. A study design file (Study content tab, Download study design) is one JSON file with the whole design and no data, stamped with the platform version that made it. Give it to another lab, put it in a paper's materials, or start a new study from it on the lab console (Create a study from a design file). Completion codes are left out of the file, so the new study gets fresh ones to enter in its panel.

Participants sign in with an email address on an allowed domain (for example GSU addresses), or with a pilot access code. There is no password for participants and no verification email, so anyone who knows a classmate's address could sign in as that person. Keep that in mind when you write the protocol.

Study types

Alert triage

Participants review security alerts with click-to-reveal evidence and, depending on condition, advice from an AI assistant. It is used by the built-in verification study: four weekly sessions, with and without AI advice.

Vignettes and surveys

Scenarios shown as plain text or emails, each followed by questions you choose: rating scales, multiple choice, dropdowns, text, a confidence line, numbers, and select-all. Conditions are defined in the content, and each item can have different wording per condition, so a manipulation needs no code. A study with no scenarios is a survey, including a longitudinal one. Questions can be shown or hidden depending on earlier answers, the participant's condition, or embedded data such as a Prolific ID read from the link.

Economic games

Prisoner's dilemma, public goods, trust, ultimatum, dictator, or a game you define with payoff formulas. Games can be played live (participants are matched in real time) or asynchronously (each participant decides alone and is matched with stored decisions). See Running economic games.

Reaction times (Stroop or flanker)

A keyboard reaction-time task for a classic interference effect: the Stroop task (press the key for the ink colour of a colour word) or the flanker task (the direction of the middle arrow). Start from the Stroop or flanker task template; its Task tab sets the trials, practice, timing, and keys. Trials are balanced between congruent and incongruent, response times run from the first display frame of each stimulus, and the template asks for a computer, full screen, and a quick measurement of each participant's screen. The data have one row per trial, ready for the classic analysis (slower, less accurate answers on incongruent trials).

Media, voice, and conversations

Items of five kinds, mixed freely: text (for example training material), an image, an audio clip, a video clip, or a phone call from an AI voice caller. After each item you can ask for a decision (for example accept, verify, or refuse) with mouse tracking, any of your questions, and a spoken answer. Items are grouped into blocks that run in order, each shown in a fixed or shuffled order. See Audio, video, and phone calls.

Audio, video, and phone calls

1. Add your clips. On Study content, Items, choose the kind of item, then Upload a file (up to about 2.5 MB) or paste an https:// address for larger video. Uploaded files get a random name, so the address never gives away the condition. Tick Must be played to the end to lock the answers until the clip has been heard, and set a limit on plays if you want one. Add a transcript for participants who need it; whether they open it is recorded.
2. Arms. On Arms & blocks, add up to 12 arms. Each item can show each arm its own text or clip (Different per arm on the Items tab), which is how training arms are built. Blocks run in order; tick Shuffle this block to randomize within it.
3. The decision. On Decision & directory, write the question and two to four answers. With mouse tracking on, participants click Start at the bottom and move to an answer at the top; the export has the path measures (MAD, AUC, direction changes) and timing for every decision.
4. Phone calls. A call item rings with the caller ID you set. Write what the caller says at each step: the opening, the reason for the call, the request, two follow-ups if the participant hesitates, the answer to "can I call you back?", and the goodbyes. Mark the caller as an attacker or a legitimate caller (participants never see this). Participants talk (Chrome and Edge understand speech) or type, and can hang up at any time. After the call they can look up a number in the phone directory and call it: the organization's real number tells them the truth.
5. Recording. On Sign-up & consent, turn on Voice recording to ask for separate consent to record. People who decline still take part and type instead. Recordings are listed on the Recordings & calls tab (managers and owners), never copied to backups, and can be deleted in one step when your IRB says so.
6. Prolific and other panels. See Recruiting from Prolific, SONA, MTurk, or CloudResearch.
7. Before launch, preview every arm, run the self-test, and pilot with a few lab members on the computers and browsers your participants will use.

What gets saved for each call: every turn of the conversation (what the participant said, with numbers, emails, and street addresses removed before saving; what the caller said; the step the caller was on; response times), how the call ended (for example disclosed, verified, refused, declined), callbacks to the directory, and the recording when the participant agreed to one. The codebook lists every column.

Recruiting from Prolific, SONA, MTurk, or CloudResearch

On Study content, Sign-up & consent, turn on Online panel and pick your panel. The lab fills in the settings that panel needs and shows the study link to paste into it. Participants arrive already signed in: their panel ID links their sessions, and is kept only in the payment list, never in the research data.

1. Codes. There are three: the completion code (every session done), the screened-out code (a screening question, a failed check, or a full quota ended the study), and the returned code (the participant clicked "I want to stop taking part"). Copy them from your panel, or click Generate and enter the same codes in the panel. Participants see a code only when they reach that point, so nobody can read it from the page.
2. The address back. Write {code} where the code goes and {pid} where the participant's panel ID goes. Prolific: the preset fills in Prolific's completion address, ending in ?cc={code}. SONA: paste the "client-side completion URL" from SONA's study page and put {pid} where SONA shows XXXX; SONA then grants credit by itself. MTurk needs no address: workers paste the code into the HIT. The last page shows the code with a Copy button and a button back to the panel. It never sends people away on a timer.
3. Study and session IDs. Prolific's STUDY_ID and SESSION_ID (or MTurk's hitId and assignmentId) are saved with the participant and appear in the exports. They are not identifying. Two session IDs for one participant means the same person opened the study twice.
4. Quotas. Under Quotas, cap how many people can sign up for the whole study and for each condition. A condition that is full stops receiving people, and when the whole study is full, newcomers see "This study is full" and get the screened-out code (so the panel pays or releases them as you set it up). People you add yourself are never turned away but do count. Set the cap a little above your target if you will exclude people later.
5. Test it. Open the study link yourself with a made-up ID (for example ?PROLIFIC_PID=test123&STUDY_ID=s&SESSION_ID=x), finish the study, and check that the button takes you to the panel's page with the right code. Delete that test participant afterwards.

Timing-critical studies

On Study content, Devices & timing, tick Load every picture, sound, and video before the session starts. Participants see a progress bar, and the session starts only when every file is ready, so a slow connection cannot delay a stimulus. Upload timing-critical files to the study rather than linking to another website. Every trial records onset_ms (the first display frame after the item appeared) and frame_ms (the screen's refresh interval; 16.67 ms is 60 Hz). For response times from onset, use rt_ms minus onset_delay_ms.

To check the computers you test on, open the timing self-check: it measures the screen's refresh rate, frame jitter, dropped frames, timer lag, and the delay from speaker to microphone, and saves the numbers as a file. Studies can also measure each participant's screen before every session (Measure each participant's display on Devices & timing).

Checking participants' setup

Also on Devices & timing: allow computers only, require Chrome or Edge for studies where people talk, check the microphone with a level meter, run a short headphone check in session 1 (the Huggins-pitch test of Milne et al., 2021, which people on loudspeakers fail), and ask for full screen. You choose whether a failed microphone or headphone check ends the study for that person or is only recorded. Leaving full screen never stops a session, but every exit is recorded. The language setting tells screen readers and the phone-call voices and speech recognizer which language the study is in; you write the text yourself.

Importing a survey

A study of the type "Vignettes and surveys" can start from a survey you already have. On the study's admin page, open Study content, then Import a survey, choose the file, and click Preview. The lab console (owners) can also create a new study straight from a file. Nothing is saved by the preview. You read the report, load the result into the editor as an unsaved draft, and click Save like any other edit.

Limits: files up to about 5 MB and 500 questions. Every import produces a report that lists what was mapped, what was approximated, and what could not be reproduced, each with its source ID. Read it before you save. A display rule the importer cannot express is left out, so that question is shown to everyone. The imported consent text is a marked placeholder, not approved text.

Running economic games

Create the study with the type "Economic game", then use the Presets tab to start from a standard game and edit it. The Payoffs tab has a "Check the design" button that tests your formulas before you save.

Live games

Participants are matched with each other in real time and decide round by round. Because groups form from whoever is present, plan sessions the way you would for a lab session:

• Choose a start time and tell participants to arrive within a window of a few minutes. Groups form in multiples of the group size, so invite a few more people than a multiple and let the leftover fallback handle the rest (a participation fee, or asynchronous play if stored decisions exist).
• Participants must keep the page open and visible. Closing it counts as not responding.
• Have one person on the Game operations tab during the session. It shows groups, rounds, who is waiting or has left, and lets a manager close a stalled group.
• Do not edit the study while anyone is waiting or playing.

Asynchronous games

Each participant decides alone, at any time, and is matched afterward with a stored decision of another participant. There is nothing to schedule. A later mover can state a response for every possible earlier move (the strategy method). Run "Resolve pending matches" after the data are in, and download payments again if any were pending.

Cost note

Live games poll the server, and every poll is one request to the lab's Netlify functions. At the default of one poll every 2 seconds, a participant uses about 35 requests per minute, so 100 participants for 20 minutes is about 70,000 requests. Asynchronous play uses about the number of rounds plus 6 requests per participant. Check the plan's request allowance before a large live session, and prefer asynchronous play when live matching is not essential.

The self-test

On the Participants tab (owner), the self-test plays a whole simulated session on the real store, including a participant who stops responding, checks groups, moves, payoffs, and exports, and deletes everything it created. Run it after every change to the design and before you recruit. Its time is simulated, so it does not test real waiting. Pilot with real people too.

Payments are exported as a payments CSV (and a Prolific bulk-bonus file). Preview totals first.

Piloting

1. Under Students, create pilot access codes marked as test, with a gap of 0 days between sessions. Test participants are counted separately and never mix with real ones, and in games they only meet each other.
2. Complete the study yourself, on a phone as well as a laptop. Ask two or three colleagues to do the same and say what confused them.
3. Open the Pilot report (not every study type has the same sections). It shows how items and sessions are performing, how long a session takes compared with the minutes in your consent text, and where people drop out.
4. When you are done, use "Delete all test participants" (owner) so test data never reaches your analysis.

Exports and codebooks

• Trials CSV: one row per participant and item (or per round and stage in a game). Surveys CSV: one row per participant and survey. Full JSON for archiving. Raw behavior traces (JSONL) only if you need event-level data.
• Wide CSV: one row per participant, ready for SPSS, Stata, or a regression. Survey answers are named s1_pre_age (session, survey, question) and item measures s1_vm_bank_decision (session, item, measure). Items are named by their ID, not by the order shown, because that order is shuffled for each participant. It also says who finished, who was screened out or stopped, and the panel's study and session IDs.
• This study's codebook (CSV or Markdown, Participants tab): every column of the downloads, with its kind, allowed values and their labels, the exact question text, and the conditions, sessions, and display rules it applies to. It is generated from the study content, so download it again whenever you change a question, and keep the copy that matches the version you ran. It is ready to put next to the data in a repository.
• Reproducibility bundle (Study content tab, under Study design file): one .zip with the design, the codebook, the exact platform version and code commit, every content version with its date, a citation file, and the fields for a Zenodo upload, which gives the materials a DOI. It has no participant data. Make it from the version you ran, and deposit it with the paper.
• Games add a groups CSV and the payments files.
• A "real participants only" box leaves out test data. The test column marks it otherwise.
• Column definitions: the codebook and the games codebook.
• Exports never contain email addresses. Participants appear under a study ID. The extra-credit list, which does contain emails, is a separate download for managers and owners.

Downloads are assembled in your browser from small batches, so they work at any study size. Save them to storage your IRB protocol approves.

Data quality without a CAPTCHA

AI agents can now pass attention checks (Westwood 2025, PNAS), so every study also records signals that software shows and people rarely do: a hidden box on each survey page that only form-filling software fills in, a browser that says it is run by automation, pasted text, text that appears without typing, typing faster than people type, a session finished in under a quarter of the expected time, and the same panel ID arriving twice. Nothing is blocked and no text is kept, only counts. The pilot report lists how many participants raise each flag and their study IDs, and the wide CSV has one dq_ column per flag. Decide (and preregister) how you will use them before you look at the results.

IRB and data checklist

Complete this before the first real participant. It is a checklist, not legal advice; your IRB decides.

• The IRB-approved consent text is in the study, and "IRB-approved" is ticked. The consent describes what the lab records, including behavior traces if the study type captures them.
• The protocol names where study data are stored. Study data live in Netlify Blobs. Confirm that Georgia State University has approved that storage for your data. Do not recruit before it has.
• Access to the Netlify site is access to identifiable data. A participant's study ID is derived from their email with a secret key kept in the same store, and the extra-credit list holds the emails. Anyone with Netlify access can therefore re-identify participants. List who has Netlify access in your data management plan.
• Backups: the lab copies each study to a backup store every hour. For a copy outside Netlify, the study owner downloads the complete backup and saves it to GSU-approved storage on a schedule. It contains emails, so treat it as identifiable.
• Withdrawals: the owner can delete a participant, which removes answers, traces, plans, and email, and removes them from the backup store. Copies you downloaded earlier are not changed, so record withdrawals so you can remove those people from saved files.
• Studies that record voices: the consent describes recording, the IRB has approved a deletion schedule for audio, and someone owns deleting it on time (Recordings & calls tab). Voice is biometric data in some states (for example Illinois, Texas, and Washington); on Prolific, exclude those states or get counsel's sign-off first.
• Studies with simulated scam calls: the consent says calls "may attempt to obtain information" without saying which, all caller details are fictional, and the debrief explains the deception.
• Run the self-test, pilot, and delete the test participants.
• Decide before data collection how you will handle participants who are not paid or credited because a group stalled.

Questions

Can participants use the lab on a phone?

Yes. Pages are built for phones as well as computers. Pointer-based measures mean little on touch screens, so keep that in mind when you analyze them.

Can I change a study after participants have started?

You can edit, and every save is a new version. Each session plan is fixed when a participant starts that session, so edits only affect sessions not yet started. Do not edit a live game while anyone is waiting or playing.

Who can see my study?

The lab owner, and the researchers who have a role on it. Each study has its own data store, so studies never share participants or data. A study you have no role in does not appear on your My studies page.

I forgot my password.

Email the lab from the sign-in page. The lab owner sends a reset link that lasts 24 hours and works once.

I lost my authenticator and my recovery codes.

Email the lab. The owner can remove the authenticator from your account after checking it is you, and you set up a new one.

My account is locked.

After 10 wrong attempts in 15 minutes an email address is locked for 15 minutes. Wait, or ask the lab owner for help.

Can I ask for a feature or report a problem?

Contact the lab. Say what you were doing and what happened. Do not include participant data in the message.