Tutorial: Write and Run Your First App

An AutoMan app always has at least four parts.

Part 1. Import AutoMan and Create Main Class

As is typical in most programming languages, we have to tell Scala that we want to use AutoMan. Paste the following import statement into the top of your source file:

import org.automanlang.adapters.mturk.DSL._

Specifically, we're telling Scala that we want to use AutoMan to manage Amazon Mechanical Turk (mturk) jobs using its MTurk domain-specific language (DSL).

If you have not yet created an account for Amazon Mechanical Turk, now would be a good time.

We also need to create the main class for our application. Scala makes this a little easier than Java. Paste the following into your editor below your import statement.

object MyFirstAutoManApp extends App {
  println("Hello world!")
}

An object in Scala is just a special kind of class that does not need to be instantiated. It is also sometimes referred to as a singleton class, since you can only have one of them.

extends App tells Scala to augment the MyFirstAutoManApp with a public and static main method that takes a String[]. Any code you write inside the class will be interpreted as being a main method implementation. Arguments are available via a variable called args. If you come from Java, this is a lightweight version of the public static void main(String[] args) which you have probably used many times.

The code above is a complete Scala hello-world app. As a little sanity check, before we dig into AutoMan, let's just try it out. Run the following in your terminal:

$ sbt run

The first time you run sbt, it will likely download large numbers of libraries, printing huge amounts of diagnostic information along the way. Do not be alarmed. Subsequent runs should produce substantially less output.

You should see output that looks a bit like this:

[info] welcome to sbt 1.3.13 (N/A Java 12.0.2)
[info] loading project definition from /home/dbarowy/my-first-automan-app/project
[info] loading settings for project my-first-automan-app from build.sbt ...
[info] set current project to my-first-automan-app (in build file:/home/dbarowy/my-first-automan-app/)
[info] Compiling 1 Scala source to /home/dbarowy/my-first-automan-app/target/scala-2.12/classes ...
[info] running SimpleRadioProgram 
Hello world!
[success] Total time: 4 s, completed Oct 24, 2020, 3:34:12 PM

Notice, buried in all that output, that your program printed out Hello world!. If you see Hello world! in your output, move on to the next step. If not, read the sbt output carefully to diagnose and fix the problem.

Stack Overflow is a wonderful resource for Scala and SBT questions.

Part 2. Initialize the AutoMan Platform Adapter

Paste the following into your MyFirstAutoManApp class.

implicit val a = mturk (
    access_key_id = args(0),
    secret_access_key = args(1),
    sandbox_mode = true
)

The code above defines a variable a that stores an instance of mturk. AutoMan needs what we call a platform adapter in order to know which service to connect to. In this case, we are connecting to Mechanical Turk.

When you run this code, you will need to supply your MTurk access key and secret key on the command line. args is an argument array, and args(0) is the first element of that array (if you come from Java, note that arrays in Scala use () instead of []). These are the same credentials you downloaded in an earlier step of this tutorial.

Never embed your access_key_id or your secret_access_key in your source code! Doing so makes it easy to accidentally push your code to a public site like GitHub where they can be stolen and abused.

Mechanical Turk comes equipped with a sandbox mode. The sandbox lets you run test code without worrying about real MTurk workers or real money. When sandbox_mode is set true in the mturk adapter, you will run in sandbox mode.

Be sure that you really want to run your program before setting sandbox_mode to false. It will run real jobs and it will spend real money!

Part 3. Define a Human Function

Paste the following code below your platform adapter code.

def which_one() = radio (
  budget = 5.00,
  text = "Which one of these does not belong?",
  options = (
    choice('oscar, "Oscar the Grouch", "http://tinyurl.com/qfwlx56"),
    choice('kermit, "Kermit the Frog", "http://tinyurl.com/nuwyz3u"),
    choice('spongebob, "Spongebob Squarepants", "http://tinyurl.com/oj6wzx6"),
    choice('cookiemonster, "Cookie Monster", "http://tinyurl.com/otb6thl"),
    choice('thecount, "The Count", "http://tinyurl.com/nfdbyxa")
  )
)

The above defines a human function called which_one that takes no arguments. It is important to note that which_one is just an ordinary function in Scala, although it does behave in some special ways that we will describe in the next section.

This function creates a "radio button question" on Mechanical Turk by calling the radio constructor. "Radio button questions" allow MTurk users to select one of n options.

AutoMan has constructors for numerous question types. Refer to the AutoMan API Reference for more information.

The constructor above has many parameters which are set to "sane defaults," so that you do not need to specify a task in great detail. These defaults are designed to minimize your surprise.

The key elements in the question function above are:

• budget: This parameter specifies the maximum amount of money AutoMan will spend on this task. AutoMan always tries to spend less. If the cost of a task exceeds the budget you supply, AutoMan will shut down the task and return a "low-confidence answer."

• text: This parameter supplies the text of the question. You describe what you want workers to do here.

• options: This parameter supplies the valid options. Since this is a radio button question, each option will produce a radio button. The choice constructor takes three parameters:

  • A label of type Symbol. You can think of a Symbol as a special string designed for easy comparison. This parameter is not visible to MTurk workers.

  • A name, which is visible to MTurk workers.

  • An optional image_url, which is a link to an image hosted somewhere on the Internet, like this one.

• confidence (not shown): This parameter stands for the statistical confidence level and is a floating-point number between 0 and 1 (exclusive). A number approaching zero tells AutoMan that virtually any answer is fine. A number approaching one tells AutoMan that you want to be very certain that it is correct. Although the confidence parameter is not shown above, it is set to the default of 0.95, which is something of a standard threshold across empirical science.

Part 4. Call Your Human Function Inside an AutoMan Block

Now that you have a human function defined, you can call it. First, paste the following AutoMan block below your function definition.

automan(a) {
  // your code will go here
}

We are going to call our function inside that block. The purpose of an AutoMan block is to delineate when you are done using MTurk. AutoMan needs this information so that it knows you are ready to shut down your program.

Failing to tell AutoMan to shutdown will cause your program to hang.

Now we can call which_one() inside our AutoMan block.

automan(a) {
  println("Answer is: " + which_one())
}

The Complete Program

Here is the complete source code for our first AutoMan program (with the hello-world bit removed). You can find a copy of this program in the sample applications directory of AutoMan's GitHub repository, along with many other examples.

import org.automanlang.adapters.mturk.DSL._

object MyFirstAutoManApp extends App {
  implicit val a = mturk (
    access_key_id = args(0),
    secret_access_key = args(1),
    sandbox_mode = true
  )

  def which_one() = radio (
    budget = 5.00,
    text = "Which one of these does not belong?",
    options = (
      choice('oscar, "Oscar the Grouch", "https://tinyurl.com/y2nf2h76"),
      choice('kermit, "Kermit the Frog", "https://tinyurl.com/yxh2emmr"),
      choice('spongebob, "Spongebob Squarepants", "https://tinyurl.com/y3uv2oew"),
      choice('cookiemonster, "Cookie Monster", "https://tinyurl.com/y68x9zvx"),
      choice('thecount, "The Count", "https://tinyurl.com/y6na5a8a")
    )
  )

  automan(a) {
    println("Answer is: " + which_one())
  }
}

Notice that the above program does not store any private access keys in the source text! Instead, they must be passed in using command line arguments. For additional examples with more sophisticated command line parsing, see the sample applications directory.

Run Your First App

Back on the command-line, we can now run our app with:

$ sbt "run <your access key> <your secret key>"

Assuming you have not changed the mturk initializer, this program will post jobs to the MTurk sandbox. When you run this program in the sandbox, no work will get done, because only the live production site has active workers. You must simulate the job yourself. We describe the process of simulating a job in the section titled Pro Tip: Use the MTurk Sandbox.

After posted jobs are completed, somewhere in AutoMan's voluminous output, you should see something like:

Answer is: 'spongebob

If you want to see AutoMan's quality control algorithm in action, use the sandbox to supply worker responses that disagree. You should observe that after AutoMan obtains all of the responses for a given round, it will decide whether disagreement is strong enough to warrant asking the crowd for more responses.

The program above works, but it's a little simplistic. Without some special configuration, MTurk will not let you post tasks with fewer than 10 HITs. For an easy task like this, which may only need a few responses, this program is a little inefficient. Also, if something goes wrong, this program hides that fact from you.

We provide additional example programs that work around these issues in the apps directory of our GitHub repository. An efficient, error-handling program that defines essentially the same task can be found here.

If something goes wrong as your job runs, be sure to read AutoMan's (voluminous) output. AutoMan provides a great deal of detail to make diagnosing problems easy.