The beta release has been completely rewritten in c# to take advantage of azure sdk 2.2. Alas, the python sdk has not kept pace with many of the new features and functions. The python code is still available anyway.
With the beta release, we provide a series of command line utilities to submit jobs, query status, cancel jobs. We have also implemented 3 queues for prioritization purposes.
It is now possible to configure the size of the worker VMs in the settings of the autoscaler. The autoscaler itself is a PaaS cloud service, while the worker remain VMs for compatibility with any Windows application.
By default the worker VMs will run as many tasks as they have cores.
If one task needs the full resources of the VM (e.g. it is multi-threaded, uses up the RAM), it is possible to specify that it should be run exclusively.
For instance:qsub highq exclusive "myprocess.exe par1 par2"
will submit a task to the high priority queue to be run exclusively on whatever node picks it up. The agent will spawn myprocess.exe and pass it parameters par1 and par2.
The other threads monitoring the queues will just wait for the task to finish.
Note that the solution does not handle yet authentication and authorization. Anyone with access to the utilities can submit jobs. Jobs will execute under whatever credentials you have set for the agent when building the worker VM template. We suggest local user or local service.
Feel free to contribute an authentication and authorization module :-)
Please remember to enter the appropriate values for your subscription in the files instead of the <string> placeholders before compiling.
You can find a short demo here: http://youtu.be/7OlpcS3A_OMProject Description
Geres is a simple batch job manager for Azure, written in Python for general applicability.
From a Python script (or interactive console, e.g. iPython worksheet) users will be able to queue jobs for computation, query their status.
Computation happens in Azure virtual machines where a Python interpreter has been deployed, but is not limited to python scripts. The interpreter will call whatever executable is specified in the job description.
The number of computing nodes is managed automatically to optimize waiting times before a job is started.
Microsoft Developer & Platform Evangelism
Copyright (c) Microsoft Corporation. All rights reserved.
THIS CODE AND INFORMATION ARE PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND,
EITHER EXPRESSED OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND/OR FITNESS FOR A PARTICULAR PURPOSE.
The example companies, organizations, products, domain names,
e-mail addresses, logos, people, places, and events depicted
herein are fictitious. No association with any real company,
organization, product, domain name, email address, logo, person,
places, or events is intended or should be inferred.
THIS IS SAMPLE CODE. BUGS ARE EXPECTED. SECURITY IS NOT.