Seed note: this methodology page is launch-ready in structure, but it should evolve as the publication accumulates real labs and editorial standards.

Source hierarchy

AgentScope uses a clear source order.

1. Official product and company sources establish what actually changed.
2. Public benchmarks and evaluation assets provide comparable task structure.
3. AgentScope labs test how those claims survive real engineering tasks.
4. Community evidence helps track trust, confusion, praise, and rising complaints.

That order matters. A loud social reaction is not enough on its own, and a benchmark score is not enough on its own.

Lab rules

Every lab report should record the task definition, versions or release context, environment, rubric, reviewer notes, and the difference between:

• product quality,
• model quality,
• and workflow outcome quality.

Those layers often move independently. A product can improve because the workflow got safer even if the underlying model stayed similar. A model can look strong in fixed prompts while still causing scope creep in a real repo task.

Community briefs

Community pulse coverage is manual in v1 by design. The goal is not to publish a fake precision score. The goal is to identify what people are actually reacting to, whether that reaction is broad, and whether it deserves retesting.

Representative posts should be cited, theme-clustered, and framed with editorial caution. Sarcasm, dogpiling, and product misunderstanding are common sources of noise.

Limitations

This publication starts with seeded pilot reports and a limited corpus. Readers should treat early entries as format demonstrations until the site accumulates a larger archive of real comparative work.

The right launch discipline is depth over volume:

• fewer task replays,
• stronger evidence trails,
• clearer caveats,
• and visible failures.