Pianist VS Bot

The Acoustic Turing League

Acoustic Turing League — Frequently Asked Questions

Why We Designed the League This Way

The Acoustic Turing League is a public human-versus-robot piano blind-testing league. Its purpose is to collect honest audience feedback in an entertaining format, while creating opportunities for robotics teams to learn how their systems are perceived by human listeners.

The League is not designed to eliminate every variable or claim to measure every dimension of robotic capability. Its rules reflect a different philosophy: open participation, meaningful public judgment, transparent scoring, proportionate verification, and continuous improvement.


1. What exactly is the Official Turing Score measuring?

The Official Turing Score measures how often robotic performances receive a majority audience judgment of “Human,” relative to the total number of qualifying human and robotic performances in the scoring period.

A robotic performance earns one point if more than 50% of eligible final judgments identify it as Human. Otherwise, it earns zero points. Human performances always earn zero points, regardless of the audience’s judgment.

The formula is:

Official Turing Score (%) = Robotic points earned ÷ Total qualifying human and robotic performances × 100

The score is a public perception indicator. It is not a comprehensive measure of robotic intelligence, musical quality, dexterity, safety, or general technical maturity.

2. Why does the denominator include human performances instead of only robot performances?

Because the League evaluates the complete program of a participant’s human-versus-robot challenge, not merely the success rate of selected robotic performances.

Consider a simple example:

  • A participant presents 10 qualifying performances, including both human and robotic performances.
  • Three robotic performances receive a majority judgment of Human.
  • The Official Turing Score is 3 ÷ 10 × 100 = 30%.

The denominator includes every qualifying performance, not just the robot’s attempts.

This design makes the composition of the entire program relevant to the published score. A participant cannot interpret the score properly without also examining the number of human performances, the number of robotic performances, and the number of points earned.

A robot-only denominator would answer a different question: what proportion of qualifying robotic performances received a majority judgment of Human? That can be a useful supplementary statistic, but it is not the League’s chosen Official Turing Score.

The League deliberately publishes the full counts so that audiences and researchers can understand what the score represents rather than treating a single percentage as the whole story.

3. Doesn’t including human performances in the denominator allow participants to manipulate their scores by changing the human-to-robot ratio?

The composition of the program does affect the score. The League acknowledges this openly.

For example, adding more human performances without earning additional robotic points increases the denominator and lowers the score. That is a mathematical consequence of the chosen formula, not something the League claims to have eliminated.

Why accept this trade-off?

The League is designed as an open participation format in which teams can develop their own presentation strategies. It does not impose a universal human-to-robot ratio and then pretend that all participants have identical programs.

Instead, the League publishes the underlying performance counts alongside the score. The Official Turing Score should be interpreted in the context of the participant’s actual program, not as a pure robot-only imitation rate.

The principle is transparency: define the metric clearly, preserve the records needed to reproduce it, and do not claim that it measures something it was not designed to measure.

4. Why not require every participant to use the same human-to-robot ratio, number of performances, duration, or difficulty?

Because the League is not intended to be a laboratory benchmark with every experimental variable held constant.

Participants may choose their own human-to-robot ratio, number of performances, performance length, and musical difficulty. They must follow the published rules, but they are not required to adopt a single presentation strategy.

This freedom is intentional.

A team may choose to demonstrate a broad range of performances. Another may concentrate on a smaller, more demanding program. Different teams may experiment with different strategies and discover how audiences respond.

These choices are part of the public competition.

The trade-off is equally important: results from different programs are not automatically equivalent to results from a standardized controlled experiment. The published score must be read alongside the underlying performance record.

The League values open participation and strategic diversity over the appearance of laboratory-level comparability.

5. If participants control their own formats, isn’t the League less scientifically rigorous?

It depends on what is being claimed.

The League is designed to collect real audience judgments under a shared set of public rules. It is not designed to establish a universal, controlled measurement of every robot’s capabilities.

Its value comes from combining several elements:

  • A recognizable blind-testing format.
  • A common voting and scoring procedure.
  • Publicly documented performance records.
  • Declared technical configurations.
  • Proportionate technical review.
  • The possibility of studying audience judgments alongside authorized technical data.

This can produce useful observations and research questions. It does not automatically establish causation or guarantee that differences between participants are attributable to a single technical factor.

Where controlled engineering comparisons are needed, researchers should use appropriate additional methods. The League can complement such work without pretending to replace it.

6. Why are human performances included if only robot performances can earn points?

Human performances are an essential part of the human-versus-robot format. They provide the audience with genuine human performances to judge, rather than asking people to evaluate robotic performances in isolation.

They also make the overall program a direct comparison between human and robotic performance.

Under the Official Turing Score, a human performance never earns a point. Its role is not to compete for robotic points but to form part of the complete challenge program and its published denominator.

This is a deliberate distinction between the role of a performance in the event and the role of a performance in the scoring formula.

7. Why not make the competition completely cheat-proof before launching it?

Because no practical public competition can guarantee the elimination of every possible form of manipulation before it begins.

The League instead adopts a staged approach to verification. It establishes basic rules, records relevant events, identifies what can be checked, and improves its procedures as experience and resources grow.

The current integrity measures may include:

  • Declaring the system responsible for a robotic performance.
  • Keeping the performer concealed until voting closes.
  • Locking final judgments before revealing the performer.
  • Recording performance and voting events.
  • Applying reasonable rate limits and duplicate-voting controls.
  • Conducting on-site technical reviews under the published rules.
  • Distinguishing suspected anomalies from verified violations.

These measures reduce particular risks, but they do not establish perfect security.

The League will describe the verification actually performed rather than imply that every performance has been exhaustively authenticated.

8. What prevents a participant from secretly using a human to control a robot?

The League’s intended integrity framework is designed to check whether a performance was produced by the declared robotic system rather than by hidden human teleoperation or another undeclared method.

Depending on the published rules and the available review arrangements, this may involve inspecting the relevant equipment, control enclosures, wired controllers, and media paths.

The scope of review should be sufficient to investigate the declared setup without requiring participants to disclose unrelated proprietary source code or confidential algorithms.

A review report should distinguish among:

  • Observed fact: something directly established during the review.
  • Suspected anomaly: something that warrants further examination.
  • Verified violation: evidence sufficient to support a finding that a published rule was breached.

The absence of evidence is not always proof of compliance. Equally, an unexplained anomaly is not automatically proof of misconduct.

The League records what it can establish and handles unresolved cases under its published procedures.

9. Why use peer review instead of relying entirely on the organizer?

Independent scrutiny can improve confidence in a distributed competition where participants arrange their own venues, equipment, and broadcasts.

Under the intended peer-review procedure, the next scheduled contender nominates one or two technical delegates for the designated review process. If the contender cannot provide delegates, the organizer may arrange an independent technical unit as a substitute, according to the applicable rules.

Reviewers operate within an agreed scope, respect venue safety and confidentiality, and do not receive unrestricted access to proprietary technology.

Peer review is not a guarantee that every form of misconduct will be detected. It is one layer of a broader integrity framework.

The goal is to make the verification process more credible and transparent without turning every participant’s technical system into an open-source project.

10. Could someone use fake accounts, multiple IP addresses, or voting bots to distort the result?

The League recognizes that online voting systems face this risk.

IP addresses and other network signals can help identify duplicate submissions or suspicious activity, but an IP address does not correspond perfectly to one person. Multiple legitimate voters may share a network, and one person may use multiple networks.

Consequently, the League should not claim that network-based controls prove the identity of every voter.

Instead, it may apply reasonable rate limits, duplicate prevention, anomaly detection, and documented incident procedures. Any material changes to these controls should be recorded.

If an incident materially affects a result, the League may classify the affected performance as disrupted or unverified, exclude the affected result where permitted, or apply another proportionate remedy under the published rules.

A technical anomaly should trigger an appropriate review, not an automatic accusation.

11. Why not impose severe penalties for every suspicious voting pattern?

Because suspicious activity and verified misconduct are not the same thing.

A shared university network, a public venue, a temporary connectivity problem, or a legitimate concentration of viewers may create unusual voting patterns without proving deliberate manipulation.

The League therefore distinguishes operational incidents, unverified performances, and verified rule violations.

Responses should reflect the evidence and the effect on the result. This helps protect the integrity of the competition without unfairly penalizing participants because a system behaved unexpectedly.

The objective is not to promise that abuse is impossible. It is to create a credible process for identifying, documenting, and responding to material problems.

12. Why allow participants to choose their own schedules, equipment, and broadcast platforms?

The League is intended to be a decentralized participation framework.

Participants arrange their own venues, pianos, robotic systems, human performers, local technical setups, and broadcasts. They may use their own websites or lawful third-party streaming platforms, subject to the published rules.

This allows teams with different resources and infrastructures to participate without requiring the organizer to own or operate every venue.

The trade-off is that the organizer cannot guarantee every participant’s broadcast availability, reach, or bandwidth. The rules therefore need clear procedures for schedule changes, technical failures, and performances that cannot be evaluated reliably.

Decentralization is not the absence of rules. It is a division of responsibilities under shared rules.

13. Why is the audience allowed to make the final judgment?

Because the League’s central question concerns human perception.

The audience is not being asked to certify a robot’s technical design. It is being asked to judge whether a performance sounds or appears human enough to be mistaken for a human performance under the conditions of the event.

The judgment is subjective by nature. That is part of the phenomenon being observed.

The League can preserve the voting process, aggregate judgments, and study patterns in the results. It should not claim that a majority judgment proves a robot is artistically superior or technically equivalent to a human pianist in every respect.

The score records a specific kind of public perception, under a specific set of rules.

14. Why collect technical telemetry if the score is based on audience votes?

Because the score tells us what happened, but not necessarily why.

If participants voluntarily share suitable technical records, researchers may be able to examine how aspects of robotic control relate to audience perception. Depending on the system, relevant records might include joint positions, velocities, torque, contact forces, timing deviations, and corrective actions.

For example, researchers might ask whether certain timing patterns or force-control characteristics are associated with performances that audiences more often identify as Human.

These are research questions, not established conclusions.

Different robotic systems may record different fields, units, and sampling rates. Data from different systems should not be treated as directly comparable without a sound technical basis. Missing information must remain identifiable as missing.

The long-term ambition is to build useful evidence connecting robotic control and human perception. The League does not claim that a validated model or universal benchmark already exists.

15. Who owns the technical data, and can the organizer use it to train AI models?

Data ownership, integrity auditing, research use, model training, commercial use, and public release are separate matters.

A participant may retain ownership of its raw technical logs while authorizing specified uses under clear data terms. Permission to inspect a log for competition integrity does not automatically imply permission to publish it, use it for commercial model training, or transfer it to third parties.

The League should state in advance what data it collects, how it is retained, who can access it, and what uses are authorized.

Participants should also understand what audience data they can receive for their own performances and how other participants’ confidential data and voter privacy are protected.

The long-term research value of the League depends on participants understanding the exchange and trusting that the agreed boundaries will be respected.

16. Does a high Official Turing Score mean a robot is better than a human pianist?

No.

A high score means that, under the League’s scoring rules, a participant earned more robotic points relative to its total qualifying human and robotic performances.

It does not establish that the robot is a better musician, more intelligent, safer, more dexterous, or more technically mature than a human pianist.

A robot may be difficult to distinguish from a human in one musical context and remain readily distinguishable in another. A perception score cannot summarize every dimension of performance.

The League deliberately separates its measured outcome from broader claims about capability.

17. Why not treat the Official Turing Score as a definitive scientific benchmark?

Because a benchmark requires more than a formula.

The score has a defined calculation, but comparisons also depend on the performance program, the audience, the voting conditions, the validity of the records, and the integrity of the declared setup.

The League can make its score reproducible and its limitations explicit. Researchers may later investigate whether additional standardized conditions or supplementary metrics would improve particular kinds of comparison.

Until those methods have been developed and validated, the League should describe the Official Turing Score as its own public perception metric, not as a definitive or universal measure of robotic performance.

18. Why does the League allow its verification procedures to evolve?

Because the available technical and organizational capabilities may change over time.

A public competition can begin with clear rules and a reasonable verification process, then strengthen its procedures as more participants, reviewers, and resources become available.

This is similar to the broader history of sports: advanced officiating technologies can improve the administration of a competition without being prerequisites for the sport’s existence.

However, evolving verification does not mean that rules can be changed arbitrarily. Material changes to scoring, eligibility, voting, data use, or awards should be announced before they apply to a new scoring period, with a version history maintained.

The League must be transparent both about the rules it applies and about the limits of its current verification capabilities.

19. What happens when a stream fails, voting is interrupted, or a record is missing?

The League distinguishes operational incidents from suspected misconduct and verified violations.

Depending on the circumstances and the published rules, a performance may be restarted, rescheduled, marked as affected or unverified, or excluded from the affected result.

The decision and its effect on the published score should be documented. Participants may request a review within the applicable deadline.

A stream failure does not automatically establish misconduct. Nor should a missing or corrupted record be silently treated as a normal, valid result.

The objective is consistent treatment of uncertainty, rather than pretending that every event will proceed perfectly.

20. Why offer recognition or showcase opportunities on violinist.ai?

The League aims to build a public record of participation and progress, not merely a sequence of isolated performances.

Showcase pages can help participating teams present their robotic systems, explain their work, and direct interested audiences toward their own projects.

Rankings and showcase pages serve different purposes. A team’s presence on a showcase page should not be represented as a certification of its safety, technical maturity, or superiority.

Any ranking-based awards, dedicated display rights, exclusivity, and duration must be stated clearly in the applicable season rules before they become commitments.

The broader objective is to give the League and its participants a recognizable public identity while keeping the meaning of each form of recognition precise.

21. What is the League ultimately trying to achieve?

The League seeks to create a recurring public setting in which human listeners judge human and robotic piano performances under shared rules, while robotics teams have an incentive to participate, learn, and improve.

Its long-term value may come from several sources working together:

  • A recognizable public format.
  • A growing network of participating teams.
  • Repeatable voting and scoring procedures.
  • Transparent records of performances and outcomes.
  • Technical review proportionate to available resources.
  • Authorized technical data linked to audience judgments.
  • Research that develops from accumulated evidence.
  • A durable public identity shared by the League and its participants.

The League does not need to solve every research or verification problem before its first trial. It does need to state honestly what it measures, what it verifies, what remains uncertain, and how its rules can improve.


The Principle Behind the Rules

The competition produces observations.
The records preserve evidence.
The accumulated evidence may help explain what the score alone cannot.

The Acoustic Turing League is designed to make public perception observable, participation possible, and technical learning worth pursuing.

It does not require perfect auditing before a competition can begin. It requires clear rules, transparent records, proportionate verification, and an honest distinction between what has been established and what remains to be learned.