radiocheckboxfreetextestimateradiosradio, except that it returns the entire distribution.checkboxescheckbox, except that it returns the entire distribution.freetextsfreetext, except that it returns the entire distribution.def) to pass the imgUrl parameter through to the estimate constructor. See our sample apps for additional examples.Outcome. Although you can call toString on such return values to obtain a simple, printable string, you should probably pattern-match on the result value returned by calling answer (or answers, depending on the question) on the returned Outcome object. Each question type has a different set of possible return values. We describe them in the next section.answer (or answers, depending on the question type) will block execution until the task is done running, which may be a substantial amount of time. Be sure that you want blocking behavior when you call answer.toString method for Outcome calls answer internally, which means that it blocks!mturk DSL here.options are the selection options seen by the user, along with optional images. Options can be created using one of the following choice constructors:choice(key: Symbol, text: String) or choice(key: Symbol, text: String, image_url: String)key denotes a stable identifier for a choice (e.g., kermit) not shown to the worker, text is the text label shown to the worker, and image_url is a url of an image shown beside the text label.text is the text of the question shown in a HIT and, by default, also as the task title. You can override the title by setting the title parameter.Answer[Symbol]: An object that represents a selected radio button, where each possible Symbol was defined with the key parameter of the choice constructors described above. This object has the following fields:value: the answer (Symbol).cost: the total cost (BigDecimal)confidence: the final confidence value (Double).distribution: raw sample responses (Array[Response[Symbol]])LowConfidenceAnswer, which has the same fields as Answer but which indicates that a quality-controlled response has a confidence lower than the desired threshold.OverBudgetAnswer, which indicates that a specified task cannot run at all due to insufficient funds. This object has the following fields:need: the funds needed to start a job (BigDecimal)have: the funds at hand (BigDecimal)NoAnswer, which indicates that an unexpected runtime error occurred.options are the selection options seen by the user, along with optional images. Options can be created using one of the following choice constructors:choice(key: Symbol, text: String) or choice(key: Symbol, text: String, image_url: String)key denotes a stable identifier for a choice (e.g., kermit) not shown to the worker, text is the text label shown to the worker, and image_url is a url of an image shown beside the text label.text is the text of the question shown in a HIT and, by default, also as the task title. You can override the title by setting the title parameter.Answer[Set[Symbol]]: An object that represents a set of selected checkboxes, where each Symbol was defined with the key parameter of the choice constructors described above. This object has the following fields:value: the answer (Set[Symbol]).cost: the total cost (BigDecimal)confidence: the final confidence value (Double).distribution: raw sample responses (Array[Response[Set[Symbol]]])LowConfidenceAnswer, which has the same fields as Answer but which indicates that a quality-controlled response has a confidence lower than the desired threshold.OverBudgetAnswer, which indicates that a specified task cannot run at all due to insufficient funds. This object has the following fields:need: the funds needed to start a job (BigDecimal)have: the funds at hand (BigDecimal)NoAnswer, which indicates that an unexpected runtime error occurred.pattern is a COBOL-style picture clause pattern that states what inputs are valid. AutoMan uses this pattern to perform probability calculations. A matches an alphabetic character, B matches an optional alphabetic character, X matches an alphanumeric character, Y matches an optional alphanumeric character, 9 matches a numeric character, and 0matches an optional numeric character. For example, a telephone number recognition application might use the pattern 09999999999.text is the text of the question shown in a HIT and, by default, also as the task title. You can override the title by setting the title parameter.freetext-specific:allow_empty_pattern means that the empty string is a valid worker response.
default: falsebefore_filter is not currently used.pattern_error_text is a helpful message that is displayed to the user when their input does not match a pattern. It is not mandatory but it is highly recommended that you use this setting.pattern_error_text for freetext questions as the default MTurk help message is not helpful. This parameter gives you an opportunity to provide an error explanation in non-technical terms.Answer[String]: An object that represents a response string. This object has the following fields:value: the answer (String).cost: the total cost (BigDecimal)confidence: the final confidence value (Double).distribution: raw sample responses (Array[Response[String]])LowConfidenceAnswer, which has the same fields as Answer but which indicates that a quality-controlled response has a confidence lower than the desired threshold.OverBudgetAnswer, which indicates that a specified task cannot run at all due to insufficient funds. This object has the following fields:need: the funds needed to start a job (BigDecimal)have: the funds at hand (BigDecimal)NoAnswer, which indicates that an unexpected runtime error occurred.confidence_interval lets you denote the confidence interval of an estimate. The options are:UnconstrainedCI() which will only even perform one round of tasks using the default sample size, returning the SymmetricCI(err: Double) which returns the err with confidence level confidence.AsymmetricCI(lerr: Double, herr: Double) which returns the -lerrr and +herr with confidence level confidence.text is the text of the question shown in a HIT and, by default, also as the task title. You can override the title by setting the title parameter.Estimate: An object that represents a "best estimate". This object has the following fields:value: the estiamte (Double).low: the low bound of a confidence interval's estimate (Double).high: the high bound of a confidence interval's estimate (Double).cost: the total cost (BigDecimal)confidence: the final confidence value (Double).distribution: raw sample responses (Array[Response[Double]])LowConfidenceEstimate, which has the same fields as Estimate but which indicates that a quality-controlled response has a confidence lower than the desired threshold.OverBudgetEstimate, which indicates that a specified task cannot run at all due to insufficient funds. This object has the following fields:need: the funds needed to start a job (BigDecimal)have: the funds at hand (BigDecimal)NoEstimate, which indicates that an unexpected runtime error occurred.checkboxes constructor here, but freetexts and radios are similar. There is also a buggy multiestimate constructor that should probably not be used at the moment.sample_size is the size of the sample.options are the selection options seen by the user, along with optional images. Options can be created using one of the following choice constructors:choice(key: Symbol, text: String) or choice(key: Symbol, text: String, image_url: String)key denotes a stable identifier for a choice (e.g., kermit) not shown to the worker, text is the text label shown to the worker, and image_url is a url of an image shown beside the text label.text is the text of the question shown in a HIT and, by default, also as the task title. You can override the title by setting the title parameter.budget is the total amount of money to be spent by a given human question function call. Note that this means that each function call has its own budget.
default: $5.00dont_reject, when set to true, will always accept completed assignments and pay workers for their work. This is useful when work is difficult and errors are likely, or when you just don't want to deal with the hassle of reputation management.
default: falsedry_run, when set to true, will not actually post jobs on MTurk.
default: falseimage_alt_text adds an HTML ALT annotation to the IMG tag created by the image_url parameter.
default: none (null)image_url adds an image to a question. Such images should be hosted someplace publically-accessible, such as Amazon S3 or a personal website.
default: none (null)initial_worker_timeout_in_s is the amount of time permitted to a worker in the initial round of tasks. Note that the actual time permitted depends on the number of rounds and is determined by the quality control policy. The default policy uses the formula initial_worker_timeout_in_s, minimum_spawn_policy states what the smallest number of assignments for a given HIT are on MTurk. This is necessary because MTurk has two totally boneheaded policies:UserDefinableSpawnPolicy(n) where n is the number you want.
default: 10
note: I am actively unhappy about this and am thinking of ways to simplify it. Suggestions welcome.mock_answers sets AutoMan to be used in mock mode for testing purposes. This is used interally by AutoMan for testing. You should not change this.
default: nullpay_all_on_failure controls whether workers are paid when a task runs out of money. Setting this to false means that workers will not be paid when an OverBudget result is returned, which generally makes workers unhappy.
default: truequestion_timeout_multiplier controls how much time a HIT exists on MTurk before it is timed out. Note that this is a distinct timeout from the amount of time a worker is given to complete a task, which is controlled by the initial_worker_timeout_in_s parameter. A HIT's total time is determined by the formula initial_worker_timeout_in_s, question_timeout_multiplier.wage controls the base wage for a worker. The actual reward paid depends on how much time a worker is given to do a task. The default policy uses a maximum likelihood estimate of the probability that a task is accepted in order to compute a wage that disincentivzes wage gaming behavior. It is complicated enough that if you want to know its inner workings, you should read our 2016 CACM article. Generally you should think of the reward as "probably doubling."
default: the U.S. Federal Minimum wage, or $7.25/houra is an initialized AutoMan platform adapter. Typically this will be an implicit variable that you return from a platform initializer expression like mturk. When marked implicit, you do not need to pass the parameter yourself; Scala will find it in the environment and pass it, simplifying human function calls. AutoMan needs this information in order to bind a human function call to a given crowdsourcing platform.database_path = "/path/to/your/database". The format of the database is H2.apps directory. Apps can also be built using pack. E.g.,apps/[the app]/target/pack/bin/.