RetailAgent | Liuyuan Jiang

Summary

Aggregate return combines a stock’s overall movement with an agent’s choice of holding intervals. RetailAgent separates these components and finds negative within-stock timing across the evaluated configurations. Saved-action shuffles attenuate the effect, while self-authored memory changes persistence and conditional timing.

Exposure-matched timing separates the return of constant exposure from the agent’s allocation of that exposure across intervals on the same stock-day. Equations 1–2
All 14 standard configurations report negative mean timing among stock-days containing both actions. The configurations vary input modality, decision interval, account state, and model family. Table 1
The principal 10-minute text condition scores −45.7 bps per stock-day. Same-day shuffling attenuates the estimate to −8.7 bps while preserving each day’s number of long actions. Table 1
With a consistency directive, one earlier self-authored note is associated with more negative main-period timing and fewer position changes in the separate later-period panel. Table 3

What does aggregate return conceal about an agent’s decisions?

Exposure and timing are distinct

An agent can earn positive action-weighted return by remaining invested during a rising stock-day. That outcome still leaves open whether its chosen holding intervals were relatively favorable. RetailAgent compares the schedule with constant exposure on the same realized path.

Sequential decisions under controlled inputs

The framework records anonymized price observations, permitted state, actions, optional self-authored memories, and subsequently revealed returns. The principal numerical-text protocol supplies completed-interval history. Representation and state conditions examine how the policy changes when its input or carried state changes.

The empirical question

The study tests whether negative timing recurs across configurations and whether that pattern depends on the ordering of actions. It then varies visible self-authored memory to examine the relationship between persistence and conditional timing.

RetailAgent architecture with text and vision inputs, a self-authored memory route, and a GRU-and-adapter soft-token branch connected to a frozen LLM and a long-or-flat action. — **Figure 2 · Observation, action, and self-conditioning**
Numerical-text, visual, memory, and learned soft-token pathways into RetailAgent’s frozen LLM. Numerical-text histories use completed intervals. The chart and encoder branches use archived reconstructed one-minute inputs, whose provenance is documented in Appendix A.1.
Original paper source

Observation pathways. Figure 2 routes numerical text and visual observations through the frozen Qwen3.5-9B model. Joint prompts contain both input representations.

Recursive policy state. Position, ledger, account summaries, and self-authored notes depend on earlier decisions. The dashed memory route carries the agent’s own language into later prompts.

Learned continuous state. The optional branch maps seven price features through a VICReg-pretrained GRU and rank-64 adapter into two soft tokens on a 92-word embedding basis. Its optimization stage is separate from the inference-only standard and narrative conditions.

Observations, recursive state, and self-conditioning

Sequential observation and action. The principal numerical protocol supplies completed-interval price history and conceals stock identity, date, absolute price, news, fundamentals, and cross-sectional ranks. Each of the 23 ten-minute decisions precedes its return label.

Endogenous account state. Stateful prompts add position and entry, a trade ledger, or an account summary. These variables arise from earlier decisions, creating a recursive policy on the same historical path.

Narrative and latent conditioning. The narrative protocol first generates a note and then chooses an action using that note and up to w earlier notes. Standard and narrative conditions use inference only. The optional learned channel trains a GRU encoder and adapter to supply two price-dependent soft tokens to a frozen LLM.

Centering exposure isolates interval selection

Subtracting the constant-exposure comparator removes the contribution of the stock-day’s total return at the agent’s average long fraction. The remaining term measures how the agent allocates exposure across intervals. An always-long or always-flat schedule has zero centered timing, and complementing a schedule reverses its timing sign.

Exposure-matched timing and evaluation population

Notation

$i,\;d$: Stock and trading-date indices identifying a stock-day.
$t,\;T$: Decision-interval index and number of scored intervals on the stock-day.
$p_{idt}$: Binary action selected before interval t: 1 for long and 0 for flat.
$r_{idt}$: Relative price change over the subsequent interval, used as its research-return label.
$\bar p_{id}$: Mean long exposure across the T scored intervals.
$A_{id}(p,r)$: Exposure-matched within-stock timing in Equation 1. Reported means are in basis points per scored stock-day.
$w$: Number of earlier self-authored notes supplied in addition to the current-step note.

A_{id}(p,r)=\sum_t\bigl(p_{idt}-\bar p_{id}\bigr)r_{idt}

\bar p_{id}=T^{-1}\sum_t p_{idt}

The indices i and d identify a stock-day, t indexes its T scored intervals, $p_{idt}$ is the binary long-or-flat action, and $r_{idt}$ is the subsequent interval return. Mean exposure $\bar p_{id}$ is the fraction of scored intervals spent long. $A_{id}(p,r)$ is measured in basis points per stock-day, where one basis point equals 0.01%.

\sum_t p_{idt}r_{idt}=\bar p_{id}\sum_t r_{idt}+A_{id}(p,r)

Equation 2

An exact decomposition on each stock-day

Action-weighted return equals mean exposure times the sum of interval returns, plus centered timing. Always-long and always-flat schedules have zero timing. Exchanging every long and flat action negates the timing score.

Equation 2 in the paper

Appendix A.5

The population measured by timing

Standard and narrative timing means condition on stock-days containing both actions. Confidence intervals resample trading dates. Cross-configuration magnitude comparisons use the recorded condition-specific sampling frames.

Appendix A.5 in the paper

Return decomposition

Equation 2 follows by adding and subtracting mean exposure inside the action-weighted return. The first term is the return of constant exposure $\bar p_{id}$ on the same stock-day. The second, $A_{id}(p,r)$ , records alignment between deviations from that exposure and subsequent returns.

What the shuffle comparison identifies

Same-day shuffling preserves each stock-day’s number of long actions while breaking their interval alignment. Global shuffling preserves the overall long fraction in the saved archive. Their attenuated timing scores locate much of the intact estimate in the temporal arrangement of recorded actions.

Scoring population and inference

Standard and narrative timing summaries retain stock-days with both actions. Narrative scoring first removes failed intervals and requires at least five valid decisions. Confidence intervals resample trading dates, so stock-days from the same date stay in the same bootstrap cluster.

Assumptions behind the analysis

Equations 1–2 give an exact decomposition of the action-weighted return into passive exposure and centered timing. Complementing every long-or-flat action reverses the timing sign. The experiments estimate mean timing among stock-days containing both actions and examine its sensitivity to observation and policy state.

Equations 1–2 · Exposure and within-stock timing in the paper

Further details

Which trajectories contribute to the reported mean?

Standard and narrative scorers retain stock-days containing both actions. For narrative conditions, failed intervals are removed and at least five valid decisions are required. Table A1 records the flow from 1,500 attempted stock-days to each scored sample.

What is counted by the memory window?

The two-pass protocol generates a current-step note before the decision. The window w counts earlier notes. Thus $w=0$ uses the current note alone and $w=1$ adds one earlier note.

Related projects

SURF studies coverage of objective trade-offs in multi-objective learning. RetailAgent studies the temporal allocation of a sequential policy’s decisions.

SURF

Illustration

Exposure and interval choice on the same return path

Same exposure, different timing

Toggle whether a toy agent is long or flat before each interval. Returns are fixed at −20, +15, −10, +25, −15, and +5 basis points.

The paper’s timing formula is evaluated on six synthetic returns. Inverting the schedule negates its centered timing score.

Interval allocation on a fixed path

The synthetic schedule uses the paper’s timing definition to separate average exposure from interval allocation. In the archived memory-conditioned example in Table 6, positive long-interval return of +359.4 bps coexists with timing of −867.3 bps because the flat intervals capture +1,404.0 bps of the same stock-day’s movement.

Sequential alignment and memory-conditioned persistence

Intact actions and exposure-preserving shuffles

Table 1 · Qwen3.5-9B numerical text · 10-minute intervals
Condition	Stock-days	Timing (bps)	95% interval
Intact schedule	13,710	−45.7	[−49.4, −41.8]
Global shuffle	13,716	−3.5	[−5.4, −1.7]
Same-day shuffle	13,716	−8.7	[−10.4, −7.1]

Result

The intact schedule’s timing is −45.7 bps per stock-day. Same-day shuffling gives −8.7 bps after preserving daily action counts and disrupting interval alignment. These saved-action controls show that the temporal arrangement of the actions carries much of the negative timing.

Experiment setup and comparison details

Outcomes are subsequent non-overlapping research-return labels on historical equity paths. Means are conditional on scored stock-days containing both actions. The 95% percentile intervals use 3,000 trading-date bootstrap draws. The saved shuffle archive reports 13,716 stock-days, compared with 13,710 in the intact row.

Read the original result

Memory visibility under the consistency directive

Table 3 · main-period conditional timing · 10-minute intervals
Visible earlier memories	Scored N	Timing (bps)	95% interval
0 (current memory only)	1,194	−62.8	[−69.7, −56.0]
1	1,054	−74.1	[−81.5, −66.6]
5	1,154	−66.7	[−73.9, −59.7]
12	1,184	−67.1	[−75.0, −59.6]

Result

One earlier note gives the most negative main-period estimate of these four conditions, −74.1 bps per scored stock-day. In the separate later-period panel, this condition has 1.7 position changes per stock-day, compared with 2.5 for the current note alone. Longer windows change timing non-monotonically.

Experiment setup and comparison details

Each main-period arm attempts 1,500 stock-days across 150 dates. Narrative scoring removes failed intervals, requires at least five valid decisions, and retains stock-days containing both actions. The 95% date-clustered intervals use 2,000 bootstrap draws. Switch counts come from the later-period panel, which extends the evaluated dates and was included in the broader sample used to select the consistency directive.

Read the original result

Scored trajectories and date-resampled uncertainty

The reported timing means condition on stock-days containing both long and flat actions. Standard-arm confidence intervals use 3,000 date-resampled bootstrap draws and narrative intervals use 2,000. Modality and model-family rows have distinct sampling frames, so their repeated negative signs support the configuration-level finding.

Cite this work

@inproceedings{zhang2026retailagent,
  title = {RetailAgent: Structured Adverse Timing in Self-Conditioned Multimodal LLM Trading Agents},
  author = {Yupeng Zhang and Liuyuan Jiang and Hongyi Huang and Bingheng Li and Lisha Chen},
  booktitle = {NeurIPS 2026 Workshop on Interpreting Agent Behavior (IAB)},
  year = {2026}
}

Summary

What does aggregate return conceal about an agent’s decisions?

Exposure and timing are distinct

Sequential decisions under controlled inputs

The empirical question

Observations, recursive state, and self-conditioning

Centering exposure isolates interval selection

Exposure-matched timing and evaluation population

Notation

An exact decomposition on each stock-day

The population measured by timing

Return decomposition

What the shuffle comparison identifies

Scoring population and inference

Further details

Related projects

Exposure and interval choice on the same return path

Same exposure, different timing

Interval allocation on a fixed path

Sequential alignment and memory-conditioned persistence

Intact actions and exposure-preserving shuffles

Memory visibility under the consistency directive

Scored trajectories and date-resampled uncertainty

Cite this work

Research sources