Divide and conquer

Development blog of the Flamenco project

Flamenco Manager 2.3 and Worker 2.2

by Sybren A. Stüvel on Friday, January 11, 2019

Yesterday Flamenco Manager 2.3 and today Flamenco Worker 2.2 have been released. To use all the new goodness, Flamenco Server should be updated to 2.2 or newer (which is naturally already the case on Blender Cloud).

Most notable new features are:

  • Better control over logging, and as a result much less network traffic between Manager and Server.
  • It's now possible to render videos from frames using FFmpeg.
  • The Manager dashboard has had a major revamp.
  • Workers can be automatically blacklisted when they fail to render a job, so that other Workers can take over. For example, a worker with too little memory to render a certain scene won't cause failures.

Manager New Stuff

  • Upgraded the web interface to use Bootstrap 4.
  • Dashboard now uses Vue.js for a dynamic interface.
  • Dashboard drops support for Internet Explorer 11 due to it not supporting modern JavaScript.
  • Workers can be selected and sent actions with one button click.
  • Built with Golang 1.11
  • Worker sleep time can now be managed from the Manager dashboard.
  • Workers that fail multiple tasks of the same type on the same job (threshold is configurable, defaults to 3) are blacklisted. This means that they cannot perform tasks of that type on that job any more, and previously failed tasks are requeued so that they become available to other workers.
  • Workers can be requested to lazily change their state; such state changes are only applied after the currently running task is finished.
  • Added ability to compress & upload task log files to Flamenco Server. This requires Flamenco Server version 2.2 or newer.

Manager Fixes

  • Fix an issue where a task could time out when its Worker goes to sleep or shuts down.
  • Other smaller bugfixes.

Worker New Stuff

  • Added support for overriding Blender RNA properties for render commands.
  • Added a new command create_video which uses FFmpeg to create a video after rendering an image sequence. It's up to Flamenco Server to include (or not) this command in a render job.
  • Added a pre_task_check section to the configuration file, which can contain write.N and read.N keys (where N can be anything to make the keys unique). Every value is a path to be checked for writability or readability. Note that write checks are lossy, and bytes are appended to any existing file used to check writability. When such a check fails, the Worker will go to status error and sleep for 10 minutes before trying again.
  • Subprocess commands now write the spawned process PID in a text file, and refuse to run if there already is such a file with an alive PID. This helped in tracking down doubly-running Blender instances.

Worker Fixes

  • Upgraded Python from 3.5 to 3.7.
  • Moved from pip-installing requirements.txt to Pipenv.
  • Restored a missing file required for progressive rendering.
  • Prevent outgoing queue saturation by not fetching a new task when the queue is too large.
  • Fixed bug where task updates would be sent in an infinite loop when the Manager didn't know the task, blocking all other task updates.
  • Many smaller bug fixes.

For more details, checkout the Manager changelog and Worker changelog

Download Flamenco Manager and Worker and get started today!