Use Slicebot to monitor one or more directories for new files. Once a file is detected, it is automatically sliced and uploaded to the cloud.
Slicebot is included with the Slicer installation.
Slicebot is not officially supported on Windows.
Quick Start
Setting up Slicebot consists of the following steps:
- Create a configuration file that defines the directories to be monitored and how content will be processed.
- Start Slicebot by running the following command:
./slicebot /ConfigurationFilePath
Basic Information
This section explains:
- The available configuration methods.
- How to set up the main Slicebot configuration file.
- How to set up a file-specific configuration file.
- How to define configuration settings within a file name.
- How to test out a configuration.
- How to end a Slicebot session.
Configuration
Configuration settings may be defined through the following mechanisms:
-
- Defines a default configuration.
- Identifies the set of directories to be monitored.
- Defines directory-specific configurations.
-
File-specific Configuration File: Defines how a specific file will be processed.
-
File Name: Defines how a specific file will be processed.
Settings will be applied in the following order:
- [Global] section of the main configuration file.
- Directory-specific sections in the main configuration file.
- File-specific configuration file.
- File name.
As a result, settings defined within a file name will take precedence over any conflicting settings defined elsewhere.
Main Configuration File
Slicebot requires a configuration file that contains settings defining:
- How and when source media will be processed.
- The directories that will be monitored for source media.
Key Information:
- Upon starting Slicebot, it must be informed where this configuration file is located.
- Create this configuration file using a text editor.
- The file extension for this file should be “cfg” in lower-case (e.g.,
slicebot.cfg
). - This configuration file may be saved in any directory that is accessible by Slicebot.
- The file extension for this file should be “cfg” in lower-case (e.g.,
Categorize settings into the following sections:
-
Global: Defines a default configuration for all monitored directories.
-
Directory-specific Settings: A section must be created for each desired monitored directory.
-
The header for this type of section should be set to the full path to the desired directory.Example:ini1[~/sales/videos/2016/]
-
-
The settings defined in this type of section:
- Only apply to that directory.
- Override any conflicting settings defined in the Global section.
-
A section may be left blank. A blank section indicates that the specified directory should be monitored and processed according to the configuration defined in:
- The
Global
section of the main configuration file. - A file-specific configuration file.
- A file-name.
- The
Define settings using the following format:
Name = Value
Sample Configuration
1[global]2enc_wait = 13mail_host = smtp.gmail.com:5874done_emails = sys@mycompany.com, me@mycompany.com5fail_emails = me@mycompany.com6[~/t/slice/a/]7[~/t/slice/b/]8enc_wait = 09username = me@email.com10apikey = abcDEFghiJKLmnoPQRtuvWXYz123ABCdefGHIJKL
File-Specific Configuration File
Instructions on when and how a specific file is processed may be defined by creating a configuration file that contains the desired configuration settings.
Although most configuration settings may be defined within a file-specific configuration, there are exceptions (e.g., halt_on_error and ignore). The description associated with each setting indicates any limitations on where it may be implemented.
Key information
-
File-specific settings may be defined in a text file that meets the following conditions:
- It must be stored in the same directory as the desired source media.
- It must be named after the desired source media file.
- The file extension for this file should be “cfg” in lower-case (e.g., myvideo.mp4.cfg).
Use thehelper_noext
setting to exclude the source media’s file extension (e.g., mp4). -
This type of configuration file should not contain section headers (e.g., [global] or [~/folder/]).
-
File-specific settings override conflicting settings defined in the main configuration file.
-
The recommended best practice is to ensure that a file-specific configuration file is present in a monitored directory before the corresponding media file is copied into it. This ensures that the media file will be processed according to the configuration defined in the file-specific configuration file.
-
Define the require_config setting within the main configuration file to require a file-specific configuration before processing a file.
A sample configuration file is provided below.
1enc_wait = 02done_emails = marketing@mycompany.com, joe@mycompany.com3fail_emails = jane@mycompany.com
File Name Configuration
Instructions on when and how a specific file is processed may be defined by including the desired settings within the source media’s file name.
Slicebot will only honor settings defined within a file name when the
allow_inline
setting has been enabled via either the main or a file-specific configuration file.Although most configuration settings may be defined within a file name, there are exceptions (e.g., halt_on_error and ignore). The description associated with each setting indicates any limitations on where it may be implemented.
Syntax:
FileName^Setting1=Value1^Setting2=Value2^SettingN=ValueN.ext
Example
Original file name:
marketing.mp4
New file name:
marketing^max_slices=3^external_id=1FA1272b.mp4
Follow these guidelines when defining a setting within a file name:
-
Each setting should be appended to the file name.
-
Each setting should be prepended with a caret (^) symbol.
-
A file name should never contain spaces.
-
Convert space characters within a setting’s value to the plus (+) symbol.Example:
my_movie^description=Standard+Trailer.mp4
In the above example, the asset’s name will be set “Standard Trailer.”
-
-
A matching file name must still be assigned to any supporting files (e.g., closed captioning or file-specific configuration file).Closed Captioning Example:
my_movie^max_slices=3^external_id=1FA1272b.mp4.ttml
Thehelper_noext
setting determines whether source media’s file extension (e.g., mp4) needs to be included when naming supporting files. -
Settings defined within a file name override conflicting settings defined in the Slicebot or a file-specific configuration file.
-
Avoid creating file names longer than those supported by the OS.
Use file-specific configuration files to avoid OS limitations.
Testing
Use
--dry-run
option to test your configuration without processing files../slicebot --dry-run slicebot.cfg
End a Slicebot Session
Slicebot runs indefinitely until one of the following conditions is true:
- It is manually interrupted. The recommended method of interrupting Slicebot is through the SIGINT signal. This signal allows Slicebot to finish the current slicing job before shutting down. Pass this signal to Slicebot by running the following command:
kill -SIGINT SlicebotPID
- Slicebot encounters a fatal error causing it to exit with a non-zero code. This may occur when there is an issue with the configuration file.
Configuration Settings
This section describes the supported configuration settings that can be used to control how slicebot processes files.
Key information
-
The config file format considers lines starting with # to be comments, so they are ignored.
-
Although most settings can be used anywhere (global section, directory sections, or per-file configs), you will find that some settings are more naturally used in some places more than others. For example, the global section will typically hold things like username, API key, and email settings, although they are legal in other places as well.
-
Setting values support basic value substitution:[FILENAME] - replaced with the current source filename, minus the path. Example, with a source file of /mnt/files/ad.mp4:
external_id=foo[FILENAME]bar
would become
external_id=fooad.mp4bar[FILENAME_SHORT] - replaced with the current source filename, minus the path and file extension. Example, with a source file of /mnt/files/ad.mp4:
external_id=foo[FILENAME_SHORT]bar
would become
external_id=fooadbar
The following configuration settings may be leveraged when defining:
- The main configuration file.
- A file-specific configuration file.
- Configuration settings by file name.
Setting | Description | Example |
---|---|---|
allow_inline | Directory Monitoring Determines whether the source media’s file name may contain configuration settings. Learn more. Valid values are:
| allow_inline = 0 |
apikey Required | Requires Slicer version 17111500 or higher Authorization Sets the secret API key through which Slicebot authenticates to the CMS. Bypass this requirement by setting the SLICER_USER and the SLICER_APIKEY environment variables.username setting.Find API keys by navigating to Settings > Integration Keys > API Keys from the Integration Keys page Syntax: apikey = APIKey | apikey = abc... | |
auto_replace | Asset Replacement (Library) Determines whether asset(s) with the same external ID will be automatically replaced. Valid values are:
| auto_replace = 1 |
autoexpire | Asset Retention (Library) Marks an asset for deletion once the specified number of hours have elapsed. The sample configuration, shown to the right, will delete assets 48 hours after they have been encoded. | autoexpire = 48 |
breaks | Ad Breaks (Asset) Inserts an ad break for each specified time range. Key information:
breaks = StartTime-StopTime Multiple Ad Breaks Syntax: breaks = StartTime1-StopTime1, StartTime2-StopTime2, StartTimeN-StopTimeN | breaks = 10.1-30.58,105-151.332 | |
bug | Asset Overlays an image on the video at the specified start time for the given duration. Key information:
bug = PNGFile,StartTime,Duration The sample configuration, shown to the right, will add overlay1.png at the start of the asset for 5 seconds and overlay2.png five seconds before the end of the asset. If these two overlay effects overlap, then overlay2.png would appear on top of overlay1.png. | bug = overlay1.png,0,5000,,, overlay2.png,-5000,5000 |
description | Asset (Library) Assigns a name to the asset generated from the source media. This setting is typically used when defining file-specific configurations. By default, assets are named after the source media’s file name. | description = Marketing Event 2016 |
done_directory | Post-Processing Defines the directory to which the source media will be moved after it has been successfully processed. Key information:
| done_directory = ~/media/success |
done_emails | Notifications Defines a comma-separated list of email addresses to which an email notification will be sent when a file is successfully processed. This setting requires the mail_host setting. | done_emails = fred@gmail.com, george@example.com |
done_url | Notifications Defines the URL to which a HTTP POST request will be submitted when a file is successfully processed.Learn more. | done_url = http://myserver.net/myservice |
enc_wait | Slicebot Determines whether Slicebot will wait until after the cloud finishes encoding an asset before it starts processing the next source media file. Valid values are:
| enc_wait = 1 |
external_id | Asset (Library) Assigns an external ID to the asset generated from the source media. By default, the same external ID may be assigned to multiple assets. Override this behavior by adding the “_replace:” prefix to the desired external ID. This will cause the new asset to replace existing assets with the same external ID. | external_id = ep20115e_12 external_id = _replace:ep20115e_12 |
fade | Asset Fades audio and/or video in/out from the specified start time. Key information:
fade = Method,MediaType,StartTime,Duration The sample configuration, shown to the right, will fade in audio/video at the start of the asset for 5 seconds and then fade out 5 seconds before the end of the asset. | fade = in,both,0,5000,,, out,both,-5000,5000 |
fail_directory | Post-Processing Defines the directory to which the source media will be moved when it is not successfully processed. Key information:
| fail_directory = ~/media/failure |
fail_emails | Notifications Defines a comma-separated list of email addresses to which an email notification will be sent when a file is unsuccessfully processed. This setting requires the mail_host setting. | fail_emails = ops@company.org |
fail_url | Notifications Defines a URL to which a HTTP POST request will be submitted when a file is unsuccessfully processed.Learn more. | fail_url = http://myserver.net/ myservice2?status=fail |
force_aspect_ratio | Asset Forces the video to the specified aspect ratio. Syntax: force_aspect_ratio = WidthRatio:HeightRatio | force_aspect_ratio = 16:9 |
halt_on_error | Source Media Processing Determines whether processing will stop when an issue occurs. This setting may only be defined within the main configuration file. Valid values are:
| halt_on_error = 1 |
helper_noext | Directory Monitoring Determines whether supporting files must include the source media’s file extension. Valid values are:
Source media’s file name: marketing.mp4 File-specific configuration file name: marketing.mp4.cfg | helper_noext = 1 |
ignore | Configuration File Determines whether a directory will be monitored for new files. This setting should only be defined under a directory-specific section within the main configuration file. Valid values are:
| ignore = 1 |
ignoreName | Directory Monitoring Ignores files in a monitored directory with the specified file name. Key information: File names are case-insensitive. Specify multiple files by using a comma to delimit each one. | ignoreName = thumbs.db,.DS_Store |
ignoreSuffix | Directory Monitoring Ignores files in a monitored directory with the specified file extension(s). Key information: File extensions are case-insensitive. Specify multiple file extensions by using a comma to delimit each one. | ignoreSuffix = db,txt,jpg |
libraries | Asset (Library) Adds an asset to one or more shared libraries. Key information: A library may be specified either by name or GUID. Library names must be an exact match and are case-sensitive. Add an asset to multiple libraries by using a comma to delimit each one. Add to Library by Name Syntax: libraries = LibraryName Add to Library by GUID Syntax: libraries = LibraryGUID Add to Multiple Libraries Syntax: libraries = Library1,Library2,LibraryN | libraries = 357…,8b0… |
mail_from | Notifications Determines the sender email address (i.e., from) for email notifications. | mail_from = automation@company.org |
mail_host | Notifications Defines the hostname for the mail server through which email notifications will be sent. A port number may be appended to the specified host. Mail servers typically use the following ports:
Syntax: mail_host = Hostname:Port | mail_host = smtp.company.com:587 |
mail_password | Notifications Defines the password for the user defined by the mail_username setting. This setting is only required when a mail server requires authorization before sending emails. | mail_password = G183hIU39331f |
mail_username | Notifications Defines the name of the user account on the mail server through which email notifications will be sent. This setting is only required when a mail server requires authorization before sending emails. | mail_username = robo-emails@example.com |
max_slices | Slicing Limits the number of temporal slices to generate from the source media. Key information:
| max_slices = 20 |
meta | Asset (Library) Adds metadata to the asset. Key information:
meta = Field=Value An alternative method for assigning metadata is through a JSON file. Learn more. | meta = rating=TV-13 meta = rating=TV-13,,, air_date=20160105 |
meta_int | Asset (Library) Adds metadata containing an integer value to the asset. Key information:
meta.int = Field=Integer | meta_int = is_ad=1 meta_int = is_ad=1,,,year=2013 |
mix_atracks | Asset Mixes multiple audio tracks together into a single track. A sample use case is when the source audio has separate left and right audio tracks instead of a single stereo audio track. Key information:
mixatracks = Track1=Channel1,Track2=Channel2=,TrackN=ChannelN | mix_atracks = 0=L,1=R,2=C, 3=RL,4=RR,5=LFE |
multibot | Directory Monitoring Indicates whether multiple instances of Slicebot are monitoring the same directory. Valid values are:
Each instance of Slicebot should run on a separate computer. Default Value: 1 | multibot = 1 |
notify_retries | Notifications Determines the maximum number of attempts to send out a HTTP notification before proceeding to the next file. Notifications are defined through the following settings:
| notify_retries = 10 |
notify_retry_wait | Notifications Determines the number of seconds that must pass after a failed notification before reattempting to send out a notification. Default Value: 10 | notify_retry_wait = 30 |
password Deprecated | This setting and the SLICER_PASS environment variable have been deprecated and should not be used. Please use the apikey setting or the SLICER_APIKEY environment variable instead.Learn more. Defines the password that corresponds to the user defined by the username setting. | password = 1@D208PneG63f |
poster | Asset Generates a poster image from the video frame corresponding to the specified time. This parameter is ignored when poster_file has been defined. Syntax: poster = Time | poster = 5000 |
poster_file | Asset Sets the asset’s poster image to the specified file. Syntax: poster_file = FullPath | poster_file = /images/poster.png |
poster_size | Asset Sets the size of the poster image. Key information:
The following sample scenarios assume that the poster image is being generated from a 1280x720 video: poster_size: 1000x500, actual poster image size: 888x500; poster_size: 1500x1500, actual poster size: 1280x720 Syntax: poster_size = WidthxHeight | poster_size = 512x512 |
require_captions | Processing Prerequisite Determines whether a closed captioning file is required. Learn more. Valid values are:
| require_captions = 1 |
require_config | Processing Prerequisite Determines whether a file-specific configuration file is required. Learn more. Valid values are:
| require_config = 1 |
skip_drm | Asset Determines whether the asset’s playback URLs must be signed. Valid values are:
| skip_drm = 1 |
start_url | Defines the URL to which a HTTP POST request will be submitted whenever slicing is initiated on a file. Learn more. | start_url = http://myserver.net/myservice |
thumbnails | Asset Adds one or more thumbnails. Key information:
thumbnails = Prefix:WidthxHeight, Prefix2:Width2xHeight2,PrefixN:WidthNxHeightN | thumbnails = tiny:50x50,med:400x400 |
timedmeta | Asset Adds timed metadata at the specified time. Single Field Syntax: timedmeta = Time:Field=Value Multiple Fields Syntax: timedmeta = Time1:Field1= Value1,Time2:Field2=Value2,TimeN:FieldN=ValueN | timedmeta = 0:TIT2=MyTitle, 2000:TCOM=Myself |
trim | Asset Trims the content from the source media that occurs prior to the specified start time. Optionally, duration may be specified to trim content that exceeds the specified time period. Specify StartTime and Duration in milliseconds. The source media’s file type may affect video frame accuracy by a few milliseconds. Trim Content Prior to Start Time Syntax: trim = StartTime Trim Content by Start Time and Duration Syntax: trim = StartTime,Duration The sample configuration, shown to the right, will exclude the first 3 seconds of the source media and truncate it so that the asset’s total duration is 52.5 seconds. | trim = 3000,52500 |
username Required | Authorization Defines the user name corresponding to the account to which the asset will be uploaded. Bypass this requirement by setting the SLICER_USER and SLICER_APIKEY environment variables. | username = joe@example. |
Closed Captions
You can provide slicebot with closed caption information by placing closed caption data in a TTML file in the same directory as the media file. TTML files are simple text files that can be created with third party tools or with any text editor:
xml
1<?xml version="1.0" encoding="utf-8"?>2<tt xml:lang="en" xmlns="http://www.w3.org/2006/04/ttaf1" xmlns:tts="http://www.w3.org/2006/04/ttaf1#styling">3 <head>4 <styling>5 <style id="defaultSpeaker" tts:fontSize="24" tts:fontFamily="Arial" tts:fontWeight="normal" tts:fontStyle="normal" tts:textDecoration="none" tts:color="white" tts:backgroundColor="black" tts:textAlign="center" />6 <style id="defaultCaption" tts:fontSize="24" tts:fontFamily="Arial" tts:fontWeight="normal" tts:fontStyle="normal" tts:textDecoration="none" tts:color="white" tts:backgroundColor="black" tts:textAlign="center" />7 </styling>8 </head>9 <body style="defaultCaption" id="thebody">10 <div xml:lang="en">11 <p begin="00:00:01.420" end="00:00:02.620"><metadata ccrow="14" cccol="3" /><span tts:fontStyle="italic">THIS IS SOME TEXT...</span></p>12 <p begin="00:00:06.620" end="00:00:09.790"><metadata ccrow="13" cccol="1" />(Doorbell rings)</p>13 <p begin="00:00:11.100" end="00:00:13.000"><metadata ccrow="14" cccol="8" />♪ LA LA LA SINGING ♪</p>14 <p begin="00:00:20.460" end="00:00:22.170"><metadata ccrow="14" cccol="0" />THIS IS MORE TEXT</p>15 </div>16 </body>17</tt>
The
require_captions
config setting can be used to ensure that slicebot will skip a file if it does not have a corresponding TTML file. As with other settings, it can be used in the global, directory, or per-file context.The TTML files use a naming convention similar to config files: take the name of the associated media file and add a “.ttml” extension to the filename. For example, if slicebot is processing a file named /tmp/foo/film.mp4, it will look to see if there is a file named /tmp/foo/film.mp4.ttml and, if so, use it for the source of closed caption data. If no corresponding TTML file exists, the video will not have any closed caption data. Note that the helper_noext setting can alter the naming convention for TTML files.
Only hh:mm:ss.sss time formats are currently supported in the TTML file.
Metadata Files
You can assign metadata key-value pairs to an asset using individual settings; for larger amounts of metadata, however, this can be cumbersome, so it may be more convenient to make use of a metadata file. This file is a JSON text file that has the metadata you want associated with the media file. As with caption and config files, the name of the metadata file is the media filename with an additional extension: .meta.json. For example, if a media file is named video123.mp4, then an associated metadata file for it would be named video123.mp4.meta.json.
The contents of the metadata file must be a valid JSON object, or processing the file will produce an error. Structured metadata is supported, although presently it is not fully displayed or editable in the CMS UI.
1{2 "genre" : "comedy",3 "network" : "ZNN",4 "season" : 1,5 "episode" : 136}
In general, a given media file will use individual metadata settings or a metadata file, but not both. However, in the event that both are specified for a particular file, metadata from a file is applied first, followed by any metadata settings.
HTTP Notifications
HTTP notifications provide an additional way to integrate a system with your content production workflow. This type of notification allows Slicebot to submit a HTTP POST request whenever one of the following events takes place:
- Slicing Starts (start_url)
- Slicing Completes (done_url)
- Slicing Error (fail_url)
Enable HTTP notifications by setting one or more of the above settings to the URL that will be requested.
Email notifications may also be sent whenever slicing completes (done_emails) or when an error occurs (fail_emails).
HTTP POST Request
Slicebot will submit a HTTP POST request using the default Internet media type (i.e., application/x-www-form-urlencoded). The body of this request will consist of a single line with the following parameters:
Parameter | Description |
---|---|
filename | Indicates the full path and the filename of the file. |
asset_id | Indicates the system-generated unique ID of the new asset. |
external_id | Indicates the external ID defined in the configuration. If one has not been defined, then it will be set to an empty string. |
description | Indicates the file’s description defined in the configuration. If one has not been defined, then it will be set to an empty string. |
status | Indicates the result of the slicing operation. Valid values are either “ok” or “error.” |
In addition to the above parameters, any custom query string parameters defined in the configuration setting (i.e., start_url, done_url, and fail_url) will be automatically stripped from the request and inserted into the request body. This allows custom parameters to be passed to the remote server as needed.
Custom query string parameters whose names match one of the above system-defined parameters (e.g., filename or description) will be ignored.
To allow for future expansion, the server process that receives the HTTP POST should silently ignore unrecognized parameters.
Notification Issues
Slicebot will log an error under the following conditions:
- It has been configured to request an invalid URL.
- A valid URL has been provided, but the server is unreachable.
- A valid URL has been provided, but it generates a non-200 response.
After which, it will reattempt the HTTP notification several times as determined by the notify_retries setting. After each attempt, it will pause for a few seconds as determined by the notify_retry_wait setting. If it is unable to successfully submit the HTTP POST request, then it will move on to the next file.
Sample HTTP Notification Scenario
This sample HTTP notification scenario assumes that the done_url setting is configured to:
http://www.example.com/slicing/handler?custom=1234&other=5678
The following sample HTTP POST request will be submitted once a file called “video.m4v” has been successfully sliced.
1POST /slicing/handler HTTP/1.12Host: www.example.com3Accept-Encoding: identity4Content-Length: 1615Content-Type: application/x-www-form-urlencoded6Connection: close7User-Agent: upLynk-slicebot/1.08status=ok&other=5678&description=The%20file&filename=%2Fopt%2Ft%2Fmyvideos%2Fvideo.m4v&external_id=dummy_id&custom=1234&asset_id=a9119d7afc5242319d028794ae283ea7
Notice that the custom query string parameters (i.e., custom and other) defined in the done_url setting were automatically stripped from the request and inserted into the request body.