{"id":2104,"date":"2018-08-24T11:21:17","date_gmt":"2018-08-24T10:21:17","guid":{"rendered":"https:\/\/research.reading.ac.uk\/act\/?post_type=kbe_knowledgebase&p=2104"},"modified":"2022-03-10T10:20:42","modified_gmt":"2022-03-10T10:20:42","slug":"running-matlab-scripts-as-batch-jobs","status":"publish","type":"kbe_knowledgebase","link":"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/running-matlab-scripts-as-batch-jobs\/","title":{"rendered":"Running Matlab Scripts as Batch Jobs"},"content":{"rendered":"

[vc_row][vc_column][vc_column_text]Matlab jobs can run in batch on the Reading Academic Computing Cluster (RACC) and the met-cluster, without the need for an interactive session or the Matlab user interface. We suggest you use the RACC as met-cluster is being decommissioned!<\/strong>\u00a0In order to do that, an additional job script needs to be written, which calls the execution of the Matlab script after loading the Matlab module and finding the directory of the Matlab script.<\/p>\n

The examples below show a batch job scripts that call a Matlab script (called testscript here), for the RACC:<\/p>\n

#!\/bin\/bash\n\n#SBATCH --ntasks=1\n#SBATCH --cpus-per-task=1  #(some Matlab jobs might benefit from getting more CPUs)\n#SBATCH --output=logfile.txt\n#SBATCH --mem=6G   #(let's specify memory for the whole thing rather than per CPU)\n#SBATCH --time=120:00  # 2 hours\n\nmodule load matlab\nunset DISPLAY\ncd \/path\/to\/your\/matlab\/script\nmatlab -nodisplay < testscript.m > outfile.txt<\/pre>\n
and for met-cluster (legacy)<\/p>\n
#!\/bin\/bash\n\nmodule load matlab\ncd \/path\/to\/your\/matlab\/script\nmatlab -nodisplay < testscript.m > outfile.txt<\/pre>\n
The ‘matlab’ command needs to be run with the ‘-nodisplay<\/em>‘ flag in order to suppress the user interface. You can redirect the output into a file (called outfile.txt here), so that the progress of the Matlab job can be tracked easily, and is is separate from other output. But, if you don’t it will work fine too, and you get Matlab output in the job’s output file (logfile.txt). In Slurm, we need to redirect the input file in, ‘-r’ option, used in the met-cluster version of this article, might not work.<\/strong> Please note that the Matlab script itself should have the command ‘exit’ at the end so that Matlab exits successfully after it runs the code.<\/p>\n
One known issue with the above script is that if you write\/edit it through Windows, invisible extra characters tend to be introduced which stop the script from running. It’s a good idea to write this in the Unix environment instead.<\/p>\n
You can execute the above script locally first before submitting it (remember to set the permissions of the script to ‘rx’ first).<\/p>\n
On the RACC you submit the script with the command:<\/p>\n
sbatch batch_job.sh<\/pre>\n
You will have to experiment to find how much memory the job will require. Allocating too much will be wasteful (and bad for your queuing priority), while allocating too little will get the job killed when it runs out of memory. Unfortunately Matlab often allocates a lot of memory even if your actual data does not use that much, and in Slurm you need to reserve memory for the whole thing. (Maybe we should test how it works with Matlab compiler at some point) It is safe to request just 1 CPU for the whole job (–cpus-per-task=1); if Matlab wants to run multi-threaded it will see how many CPUs it got and it will not attempt that (on met-cluster it is more tricky).<\/p>\n
On met-cluster, the batch script can be submitted with the ‘qsub<\/em>‘ command. To submit the job to e.g. the limited queue, you can use<\/p>\n
qsub -pe smp 4 -l limited -j y -o $PWD\/logfile.txt batch_job.sh<\/pre>\n
The flag ‘-j y -o $PWD\/logfile.txt<\/em>‘ pipes the system output from the batch jobs into one file in your run directory, which is useful for error tracking if your job crashes or fails to run. We added ‘-pe smp 4’. It is not needed for all Matlab jobs, but we mention it here because many Matlab jobs currently running on met-cluster are multi-threaded, while users are not aware that some Matlab functions used are are trying to parallelize the computations. Typically those jobs tend to use around 4 CPU cores,<\/p>\n
For more information on how to use the clusters, please see the met-cluster documentation<\/a> and Reading Academic Computing Cluster User’s Guide<\/a>.<\/p>\n
Compiled Matlab<\/h2>\n
It is also possible to compile your matlab scripts using the Matlab Compiler<\/a>\u00a0for the appropriate version of Matlab. This can considerably reduce the amount of memory required to run the script.<\/p>\n
You can compile you matlab code using mcc<\/a>. To access the compiler you need to setup MATLAB. This can be done, as before, with the command<\/p>\n
module load matlab<\/pre>\n
If the file highlight.m contains the following lines<\/p>\n
function result = highlight(a); result = strcat('**', a, '**')<\/pre>\n
Then it can be compiled with the command<\/p>\n
mcc -R '-nodisplay,-nojvm,-singleCompThread' -m highlight.m<\/pre>\n
The options are:<\/p>\n
-R - Runtime Matlab Options\n-nodisplay - Do not try to connect to an X11 display\n-nojvm - Do not use Java Virtual Machine\n-singleCompThread - Only a single thread, this allows PBS to run multiple copies of the job on one node.\n-m - Compile the code into a standalone executable\n<\/pre>\n
More information about the options can be found on the mcc<\/a> page.<\/p>\n
You can run the script using the command<\/p>\n
.\/highlight $(hostname)<\/pre>\n
In your batch submission script you will need to load the same matlab module as you used to compile the code so that the environment is setup correctly.<\/p>\n
#!\/bin\/bash\n\n#SBATCH --ntasks=1\n#SBATCH --cpus-per-task=1 #(some Matlab jobs might benefit from getting more CPUs)\n#SBATCH --output=logfile.txt\n#SBATCH --mem=1G #(let's specify memory for the whole thing rather than per CPU)\n#SBATCH --time=120:00 # 2 hours\n\nmodule load matlab\nunset DISPLAY\ncd \/path\/to\/your\/matlab\/compiled\/script\n.\/highlight $(hostname)<\/pre>\n
 [\/vc_column_text][\/vc_column][\/vc_row]<\/p>\n","protected":false},"excerpt":{"rendered":"
[vc_row][vc_column][vc_column_text]Matlab jobs can run in batch on the Reading Academic Computing Cluster (RACC) and the met-cluster, without the need for an interactive session or the Matlab user interface. We suggest you use the RACC as met-cluster is being decommissioned!\u00a0In order to do that, an additional job script needs to be written, which calls the execution<\/p>\n","protected":false},"author":361,"featured_media":2108,"template":"","meta":{"_acf_changed":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"__cvm_playback_settings":[],"__cvm_video_id":"","_links_to":"","_links_to_target":""},"kbe_taxonomy":[61,91],"kbe_tags":[],"class_list":["post-2104","kbe_knowledgebase","type-kbe_knowledgebase","status-publish","has-post-thumbnail","hentry","kbe_taxonomy-academic-cluster-articles","kbe_taxonomy-software"],"acf":[],"yoast_head":"\nRunning Matlab Scripts as Batch Jobs - Academic Computing Team<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/running-matlab-scripts-as-batch-jobs\/\" \/>\n<meta property=\"og:locale\" content=\"en_GB\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Running Matlab Scripts as Batch Jobs - Academic Computing Team\" \/>\n<meta property=\"og:description\" content=\"[vc_row][vc_column][vc_column_text]Matlab jobs can run in batch on the Reading Academic Computing Cluster (RACC) and the met-cluster, without the need for an interactive session or the Matlab user interface. We suggest you use the RACC as met-cluster is being decommissioned!\u00a0In order to do that, an additional job script needs to be written, which calls the execution\" \/>\n<meta property=\"og:url\" content=\"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/running-matlab-scripts-as-batch-jobs\/\" \/>\n<meta property=\"og:site_name\" content=\"Academic Computing Team\" \/>\n<meta property=\"article:modified_time\" content=\"2022-03-10T10:20:42+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/research.reading.ac.uk\/act\/wp-content\/uploads\/sites\/2\/Unorganized\/81042-1.png\" \/>\n\t<meta property=\"og:image:width\" content=\"256\" \/>\n\t<meta property=\"og:image:height\" content=\"231\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Estimated reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/running-matlab-scripts-as-batch-jobs\/\",\"url\":\"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/running-matlab-scripts-as-batch-jobs\/\",\"name\":\"Running Matlab Scripts as Batch Jobs - Academic Computing Team\",\"isPartOf\":{\"@id\":\"https:\/\/research.reading.ac.uk\/act\/#website\"},\"datePublished\":\"2018-08-24T10:21:17+00:00\",\"dateModified\":\"2022-03-10T10:20:42+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/running-matlab-scripts-as-batch-jobs\/#breadcrumb\"},\"inLanguage\":\"en-GB\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/running-matlab-scripts-as-batch-jobs\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/running-matlab-scripts-as-batch-jobs\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/research.reading.ac.uk\/act\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Knowledgebase\",\"item\":\"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Running Matlab Scripts as Batch Jobs\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/research.reading.ac.uk\/act\/#website\",\"url\":\"https:\/\/research.reading.ac.uk\/act\/\",\"name\":\"Academic Computing Team\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/research.reading.ac.uk\/act\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/research.reading.ac.uk\/act\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-GB\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/research.reading.ac.uk\/act\/#organization\",\"name\":\"University of Reading\",\"url\":\"https:\/\/research.reading.ac.uk\/act\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-GB\",\"@id\":\"https:\/\/research.reading.ac.uk\/act\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/research.reading.ac.uk\/act\/wp-content\/uploads\/sites\/2\/2017\/08\/cropped-University_of_Reading_shield-1.png\",\"contentUrl\":\"https:\/\/research.reading.ac.uk\/act\/wp-content\/uploads\/sites\/2\/2017\/08\/cropped-University_of_Reading_shield-1.png\",\"width\":512,\"height\":512,\"caption\":\"University of Reading\"},\"image\":{\"@id\":\"https:\/\/research.reading.ac.uk\/act\/#\/schema\/logo\/image\/\"}}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Running Matlab Scripts as Batch Jobs - Academic Computing Team","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/running-matlab-scripts-as-batch-jobs\/","og_locale":"en_GB","og_type":"article","og_title":"Running Matlab Scripts as Batch Jobs - Academic Computing Team","og_description":"[vc_row][vc_column][vc_column_text]Matlab jobs can run in batch on the Reading Academic Computing Cluster (RACC) and the met-cluster, without the need for an interactive session or the Matlab user interface. We suggest you use the RACC as met-cluster is being decommissioned!\u00a0In order to do that, an additional job script needs to be written, which calls the execution","og_url":"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/running-matlab-scripts-as-batch-jobs\/","og_site_name":"Academic Computing Team","article_modified_time":"2022-03-10T10:20:42+00:00","og_image":[{"width":256,"height":231,"url":"https:\/\/research.reading.ac.uk\/act\/wp-content\/uploads\/sites\/2\/Unorganized\/81042-1.png","type":"image\/png"}],"twitter_card":"summary_large_image","twitter_misc":{"Estimated reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/running-matlab-scripts-as-batch-jobs\/","url":"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/running-matlab-scripts-as-batch-jobs\/","name":"Running Matlab Scripts as Batch Jobs - Academic Computing Team","isPartOf":{"@id":"https:\/\/research.reading.ac.uk\/act\/#website"},"datePublished":"2018-08-24T10:21:17+00:00","dateModified":"2022-03-10T10:20:42+00:00","breadcrumb":{"@id":"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/running-matlab-scripts-as-batch-jobs\/#breadcrumb"},"inLanguage":"en-GB","potentialAction":[{"@type":"ReadAction","target":["https:\/\/research.reading.ac.uk\/act\/knowledgebase\/running-matlab-scripts-as-batch-jobs\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/running-matlab-scripts-as-batch-jobs\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/research.reading.ac.uk\/act\/"},{"@type":"ListItem","position":2,"name":"Knowledgebase","item":"https:\/\/research.reading.ac.uk\/act\/knowledgebase\/"},{"@type":"ListItem","position":3,"name":"Running Matlab Scripts as Batch Jobs"}]},{"@type":"WebSite","@id":"https:\/\/research.reading.ac.uk\/act\/#website","url":"https:\/\/research.reading.ac.uk\/act\/","name":"Academic Computing Team","description":"","publisher":{"@id":"https:\/\/research.reading.ac.uk\/act\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/research.reading.ac.uk\/act\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-GB"},{"@type":"Organization","@id":"https:\/\/research.reading.ac.uk\/act\/#organization","name":"University of Reading","url":"https:\/\/research.reading.ac.uk\/act\/","logo":{"@type":"ImageObject","inLanguage":"en-GB","@id":"https:\/\/research.reading.ac.uk\/act\/#\/schema\/logo\/image\/","url":"https:\/\/research.reading.ac.uk\/act\/wp-content\/uploads\/sites\/2\/2017\/08\/cropped-University_of_Reading_shield-1.png","contentUrl":"https:\/\/research.reading.ac.uk\/act\/wp-content\/uploads\/sites\/2\/2017\/08\/cropped-University_of_Reading_shield-1.png","width":512,"height":512,"caption":"University of Reading"},"image":{"@id":"https:\/\/research.reading.ac.uk\/act\/#\/schema\/logo\/image\/"}}]}},"_links":{"self":[{"href":"https:\/\/research.reading.ac.uk\/act\/wp-json\/wp\/v2\/kbe_knowledgebase\/2104","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/research.reading.ac.uk\/act\/wp-json\/wp\/v2\/kbe_knowledgebase"}],"about":[{"href":"https:\/\/research.reading.ac.uk\/act\/wp-json\/wp\/v2\/types\/kbe_knowledgebase"}],"author":[{"embeddable":true,"href":"https:\/\/research.reading.ac.uk\/act\/wp-json\/wp\/v2\/users\/361"}],"version-history":[{"count":8,"href":"https:\/\/research.reading.ac.uk\/act\/wp-json\/wp\/v2\/kbe_knowledgebase\/2104\/revisions"}],"predecessor-version":[{"id":6776,"href":"https:\/\/research.reading.ac.uk\/act\/wp-json\/wp\/v2\/kbe_knowledgebase\/2104\/revisions\/6776"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/research.reading.ac.uk\/act\/wp-json\/wp\/v2\/media\/2108"}],"wp:attachment":[{"href":"https:\/\/research.reading.ac.uk\/act\/wp-json\/wp\/v2\/media?parent=2104"}],"wp:term":[{"taxonomy":"kbe_taxonomy","embeddable":true,"href":"https:\/\/research.reading.ac.uk\/act\/wp-json\/wp\/v2\/kbe_taxonomy?post=2104"},{"taxonomy":"kbe_tags","embeddable":true,"href":"https:\/\/research.reading.ac.uk\/act\/wp-json\/wp\/v2\/kbe_tags?post=2104"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}