Deploying models

1. Define your entry script and dependencies

To deploy a model, you must provide an entry script (also referred to as the scoring script) that accepts requests, scores the requests by using the model, and returns the results. The entry script is specific to your model. It must understand the format of the incoming request data, the format of the data expected by your model, and the format of the data returned to clients. If the request data is in a format that is not usable by your model, the script can transform it into an acceptable format. It can also transform the response before returning it to the client.

The entry script must contain an init() method that loads your model and then returns a function that uses the model to make a prediction based on the input data passed to the function. Azure ML runs the init() method once, when the Docker container for your web service is started. The prediction function returned by init() will be run every time the service is invoked to make a prediction on some input data. The inputs and outputs of this prediction function typically use JSON for serialization and deserialization.

# Example when the model is a file
model_path <- file.path(Sys.getenv('AZUREML_MODEL_DIR'), 'my_model.rds')

# Example when the model is a folder containing a file
model_path <- file.path(Sys.getenv('AZUREML_MODEL_DIR'), 'my_model_folder', 'my_model.rds')

# Example when the model is a file, and the deployment contains multiple versions of the same model
first_model_path <- file.path(Sys.getenv('AZUREML_MODEL_DIR'), 'my_model', '1', 'my_model.rds')

second_model_path <- file.path(Sys.getenv('AZUREML_MODEL_DIR'), 'my_model', '2', 'my_model.rds')

library(jsonlite)

init <- function()
{
  # Get the path to the model location of the registered model in Azure ML
  model_path <- Sys.getenv("AZUREML_MODEL_DIR")
  
  # Load the model
  model <- readRDS(file.path(model_path, "model.rds"))
  message("logistic regression model loaded")
  
  # The following method will be called by Azure ML each time the deployed web service is invoked
  function(data)
  {
    # Deserialize the input data to the service
    vars <- as.data.frame(fromJSON(data))
    
    # Evaluate the data on the deployed model
    prediction <- as.numeric(predict(model, vars, type="response")*100)
    
    # Return the prediction serialized to JSON
    toJSON(prediction)
  }
}

2. Define your inference environment

You will also need to provide an Azure ML environment (r_environment()) that defines all the dependencies required to execute your scoring script. You can create a new environment for deployment, or use a previously instatiated environment or registered environment.

Then define the inference configuration, which consists of the entry script, the environment, and optionally the directory that contains all the files needed to package and deploy your model (such as helper files for the entry script). See the reference documentation for inference_config().

myenv = get_environment(ws, name = 'myenv', version = '1')

inference_config = inference_config(entry_script = 'score.R',
                                    source_directory = './my_scoring_folder',
                                    environment = myenv)

Note that if you specify the source_directory parameter, the entry script file must be located in that directory, and the value to entry_script should be the relative path of the file inside that directory.

3. Define your deployment configuration

Before deploying your model, you must define the deployment configuration. The deployment configuration is specific to the compute target that will host the web service. For example, when you deploy a model locally, you must specify the port where the service accepts requests.

The following table provides examples for creating the deployment configuration for each compute target:

1. Local deployment

To deploy a model locally, you need to have Docker installed on your local machine. If you are deploying locally from a compute instance, Docker will already be installed.

For an example of local deployment, see the deploy-to-local sample.

2. ACI deployment

For an example of deploying to ACI, see the train-and-deploy-to-aci vignette.

3. AKS deployment

To deploy a model to AKS, you will first need an AKS cluster for the deployment compute target. You can either

• create a new AKS cluster with create_aks_compute()
• or attach an existing AKS cluster if you already have one in your Azure subscription with attach_aks_compute()

You can instead also create or attach an AKS cluster via the CLI or studio UI.

For an example of deploying to AKS, see the deploy-to-aks vignette.

Troubleshooting deployment

If your service deployment fails, you can use get_webservice_logs() to inspect the detailed Docker engine log messages from your web service deployment. Note that if your initial deployment fails and you want to attempt a new deployment, you will first need to delete the original web service if you want to use the same web service name. You can use the delete_webservice() method.

For a more detailed guide on working around or solving common deployment errors, see Troubleshooting AKS and ACI deployments.

Web service authentication

The easiest way to authenticate to deployed web services is to use key-based authentication, which generates static bearer-type authentication keys that do not need to be refreshed.

AKS deployments additionally support token-based auth.

The primary difference is that keys are static and can be regenerated manually, and tokens need to be refreshed upon expiration.

deployment_config <- aci_webservice_deployment_config(cpu_cores = 1, 
                                                      memory_gb = 1,
                                                      auth_enabled = TRUE)

# Generate the primary auth key
primary_key <- generate_new_webservice_key(service, key_type = "Primary")

# Generate the secondary auth key
secondary_key <- generate_new_webservice_key(service, key_type = "Secondary")

deployment_config <- aks_webservice_deployment_config(cpu_cores = 1, 
                                                      memory_gb = 1,
                                                      token_auth_enabled = TRUE)

aks_service_access_token <- get_webservice_token(service)

# Get the JWT
jwt <- aks_service_access_token$access_token
# Get the time after which token should be refreshed
refresh_after <- aks_service_access_token$refresh_after