Introduction

We provide a VirtualBox Virtual Machine file as a our artifact. The compressed tar gzip file is available for download here.

Our artifact contains installed Ubuntu 14.04 with OpenWhisk and SPL already deployed. In addition we have also already deployed SPL and OpenWhisk Conductor Actions that we used for the evaluation. Users can directly invoke these actions to compare the performance. Our artifacts supports has following two benchmarks:

  • Concurrent Invocations experiment: SPL is consistently faster than OpenWhisk Conductor with increasing concurrent invocations.
  • Request Size experiment: SPL is consistently faster than OpenWhisk Conductor for increasing request size.

Getting Started

Prerequisites: Virtual Box 6.0. 60 GB Disk space. x86 Intel CPU with 4 cores or more with at least 8 GB of RAM.

Setting up VirtualBox VM: Following steps will extract and add the Virtual Machine.

  • Extract the openwhisk.tar.gz to folder openwhisk.
  • Open VirtualBox 6.0.
  • Click on Tools.
  • Click on + button in the toolbar.Add openwhisk-1.16-linux.vbox in openwhisk folder.

Starting OpenWhisk:

  • Start the VM.
  • Open a terminal in host machine and ssh into the machine:
    • ssh -p 3022 abhinav@127.0.0.1
    • Type abhinav as password
  • Start OpenWhisk by executing:
    • sudo chown -R abhinav:abhinav /tmp/wsk*
    • cd openwhisk/ansible
    • ansible-playbook -i environments/local openwhisk.yml -e mode=clean
    • ansible-playbook -i environments/local openwhisk.yml
  • To test if OpenWhisk is running, execute:
    • wsk -i namespace list
      •
    • The output should display guest namespace
  • To test if OpenWhisk contains all required actions, execute:
    • wsk -i action list
      •
    • The output should display several available actions
  • To test if OpenWhisk Conductor action is running, invoke the deployed action by executing:
    • cd ~/openwhisk-scripts/
    • wsk -i action invoke conductorCost --param-file param-test.js -b -r
    • The output should show a json with "test" value field "text".
    To test if SPL actions is running execute, invoke the deployed action by executing:
    • cd ~/openwhisk-scripts/
    • wsk -i action invoke Program_kQw1UFK8u2 --param-file param-test.js -b -r
      •
    • The output should show a json with "test" value field "text".

Step by Step Instructions

After setting up the machine in Getting Started Guide. You can run above mentioned experiments.

To run Concurrent invocations experiment:

  • Follow instructions in Getting Started guide to start the machine and OpenWhisk
  • Change to the directory containing scripts
    • cd ~/openwhisk-scripts/
  • Execute this experiment with 1 invocation to 28 concurrent invocations with each experiment running 5 times. Please note that execution will take around 1 hour. All logs generated by this experiment except any Exception thrown can be safely ignored.
    • python experiments.py 10 10 5 28 conductor,spl load
  • All data is stored in ~/openwhisk-scripts/data/function-times.csv
  • To produce the graph execute:
    • cd R-scripts
    • R -f ./whisk-results.R
      •
  • You can now view the resulting graph in ~/openwhisk-scripts/whisk-results.pdf. This graph will show that SPL is consistently faster than OpenWhisk conductor with increasing concurrent invocations.
  • Since our experiments generates a lot of data, the logs produced by OpenWhisk can occupy significant space. Before powering off the system, we need to clean OpenWhisk logs. To cleanup OpenWhisks logs:
    • rm -rf /tmp/wsklogs/*
      •
  • Poweroff the system using:
    • sudo poweroff

To run Response time vs. request size experiment:

  • Follow instructions in Getting Started guide to start the machine and OpenWhisk
  • Change to the directory containing scripts
    • cd ~/openwhisk-scripts/
  • To execute this experiment with parameter sizes of 0 KB to 1 MB with each experiment running 5 times. Please note that execution will take around 1 hour. All logs generated by this experiment except any Exception thrown can be safely ignored.
    • python experiments.py 10 10 5 1 conductor,spl param
  • All data is stored in ~/openwhisk-scripts/data/whisk/parameter-size/1-invocation.csv
  • To produce the graph execute:
    • cd R-scripts
    • R -f ./param-whisk-results.R
      •
  • You can now view the resulting graph in ~/openwhisk-scripts/param-whisk-results.pdf. This graph will show that SPL is consistently faster than OpenWhisk conductor with increasing concurrent invocations.
  • Since our experiments generates a lot of data, the logs produced by OpenWhisk can occupy significant space. Before powering off the system, we need to clean OpenWhisk logs. To cleanup OpenWhisks logs:
    • rm -rf /tmp/wsklogs/*
      •
  • Poweroff the system using:
    • sudo poweroff