Yumda
Yum for AWS Lambda
Install / Use
/learn @lambci/YumdaREADME
yumda – yum for Lambda
A Linux distro of software packages that have been recompiled for an AWS Lambda environment, with a yum configuration to install them (requires Docker).
Contents
- Quickstart
- AWS SAM Example
- Serverless Framework Example
- No
develPackages? - Requesting Packages to Add
- Building/Hosting Your Own Packages
Quickstart
Usage:
docker run lambci/yumda:<version> yum <yum-args>
For newer Amazon Linux 2 Lambda runtimes use lambci/yumda:2. For older runtimes (python2.7, python3.6 ,python3.7, ruby2.5,
java8, go1.x, dotnetcore2.1 or provided) use lambci/yumda:1.
Eg, to see what packages are available for Amazon Linux 2 runtimes:
$ docker run --rm lambci/yumda:2 yum list available
Loaded plugins: ovl, priorities
Available Packages
GraphicsMagick.x86_64 1.3.32-1.lambda2 lambda2
GraphicsMagick-c++.x86_64 1.3.32-1.lambda2 lambda2
ImageMagick.x86_64 6.7.8.9-15.lambda2.0.2 lambda2
OpenEXR.x86_64 1.7.1-7.lambda2.0.2 lambda2
OpenEXR-libs.x86_64 1.7.1-7.lambda2.0.2 lambda2
alsa-lib.x86_64 1.1.4.1-2.lambda2 lambda2
apr.x86_64 1.6.3-5.lambda2.0.2 lambda2
# etc...
To install a dependency (eg, ghostscript) into a local directory (which could be zipped up into a
layer):
$ mkdir -p gs-layer
$ docker run --rm -v "$PWD"/gs-layer:/lambda/opt lambci/yumda:2 yum install -y ghostscript
Loaded plugins: ovl, priorities
Resolving Dependencies
--> Running transaction check
---> Package ghostscript.x86_64 0:9.06-8.lambda2.0.5 will be installed
--> Processing Dependency: urw-fonts >= 1.1 for package: ghostscript-9.06-8.lambda2.0.5.x86_64
--> Processing Dependency: lcms2 >= 2.6 for package: ghostscript-9.06-8.lambda2.0.5.x86_64
--> Processing Dependency: poppler-data for package: ghostscript-9.06-8.lambda2.0.5.x86_64
--> Processing Dependency: libtiff.so.5(LIBTIFF_4.0)(64bit) for package: ghostscript-9.06-8.lambda2.0.5.x86_64
--> Processing Dependency: libpng15.so.15(PNG15_0)(64bit) for package: ghostscript-9.06-8.lambda2.0.5.x86_64
# etc...
# Then you can zip it up and publish a layer
$ cd gs-layer
$ zip -yr ../gs-layer.zip .
$ cd ..
$ aws lambda publish-layer-version --layer-name gs-layer --zip-file fileb://gs-layer.zip --description "Ghostscript Layer"
Full example with AWS SAM
Let's say you want to create a Lambda function that needs to clone a git repository and then manipulate an image using GraphicsMagick. For fun, we'll also convert it to ASCII art and log it.
The example we'll walk through below uses nodejs10.x runtime (and hence lambci/yumda:2). The code for this example lives in the examples/nodejs10.x directory, but we'll walk through the steps of creating it from scratch. For older runtimes, see examples/python3.7 and just replace any usage below of lambci/yumda:2 with lambci/yumda:1.
Start off by creating a new SAM app:
sam init --runtime nodejs10.x --name yumda-example
cd yumda-example
We'll edit the function code in hello-world/app.js to run the commands we want:
const { execSync } = require('child_process')
const shell = cmd => execSync(cmd, { cwd: '/tmp', encoding: 'utf8', stdio: 'inherit' })
exports.lambdaHandler = async (event, context) => {
shell('git clone --depth 1 https://github.com/lambci/yumda')
shell('gm convert ./yumda/examples/sam_squirrel.jpg -negate -contrast -resize 100x100 thumbnail.jpg')
// Normally we'd perhaps upload to S3, etc... but here we just convert to ASCII:
shell('jp2a --width=69 thumbnail.jpg')
}
These binaries (git, gm, jp2a) don't exist on Lambda, so we'll need to install them – this is where yumda comes in:
# Assume we're still in the yumda-example directory
mkdir -p dependencies
docker run --rm -v "$PWD"/dependencies:/lambda/opt lambci/yumda:2 yum install -y git GraphicsMagick jp2a
Now we have the binaries (and their dependencies) in a local directory that can be deployed as a layer alongside our function.
We can declare our layer in template.yaml, so the whole app looks like this:
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
HelloWorldFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: hello-world/
Handler: app.lambdaHandler
Runtime: nodejs10.x
Layers:
- !Ref DependenciesLayer
DependenciesLayer:
Type: AWS::Serverless::LayerVersion
Properties:
ContentUri: dependencies/
You can use the AWS SAM CLI to test it:
$ sam local invoke --no-event
Invoking app.lambdaHandler (nodejs10.x)
DependenciesLayer is a local Layer in the template
Image was not found.
Building image...
Requested to skip pulling images ...
Mounting /tmp/sam-app/hello-world as /var/task:ro,delegated inside runtime container
START RequestId: 6908f297-72af-143f-dd79-2b6128c5c428 Version: $LATEST
Cloning into 'yumda'...
....'',,,''....
.';codddddddddddddddddddol;'.
.,ldxdddddddddddddddddodddooddddolc;.
.codddddoddddddddddddddddddddddddddddolllc'
.;odddddddddddddddddddddddddddddddddddddddollll:.
:doddddddddddddddddddddddddddddddddddddoddddolooodc
'ddodooooddddoooddddddddddddddddddddddddddddddddlldlod'
cddodxkOOOOOOOOOxdddddddddddddddddddddddoddddddddooloood:
:ddxO00OOOOOO0OOO0Oxdddddoddddddddoddoddoodddddddddooooooxl
,xxk00OOO00OOOOO0OOO0OdoxdxoclllllllddddddddodddddddoooooooO;
dOKkdoodk0OOO00OOOO00Oko:c:,..''.'.';ccldddddddddddddooooookO
.O:. .lO0k00O0kc,::;,''''''''.''''',;;';lodddddddooooodkK'
'0OOOk:'....'',,.'.'''.''',,''....'cddodddoloood0K;
oKx:''....','','''.,.''''',,''. .'',ldddooooookOK:
ld;'.'.,:;'',,''.,0Wx.'''',,,,;;'.''':oddoooox0OK;
'o;,';:;;,,,,,,'''.okNx..'',,,,,,;:;,,;:oolood00kK.
.x:;;;;',''',,''''.oWclNO,.'',',''',;;;;;lloodO0O0d
:0x;;;;,'''',,,,,''.:,'',;'',,;,,'''',;;;;:oodOO0kK.
.x00d;;;:;;::::::::;c::c::c:::::c:::::::;;;;:lkOOOOK;
.l0OO0x;;::;;,,,,,,,,,,,,,,',,,,,,',,,,,,;;::;cOOO0kKc
;O0O0OOdol.''..''...''...''''.'...''....''..',xx0O0O0:
.x0OOOkdodO''..'.,;:;;;;;,..''...';:;;;;;;'..''.d00k0k'
cKO00kxoddxd..'.':'. ..'..'.. .;,'...:0OKl
.d0O0kddodddkl.'.'. ... ..''.;0l.
.OO0OxodddddoOl''.. . ;l. .. .l: ..'..
.O0Oxoddddddddxx'.'. .0WWN. lXWWc .''
xOOdooddddddodkl.''. . 'od' .cd: ..''. . x,x
;Kxodddddddood',...... . . ........ ...... 'lkOlo.
Oxoddddoddoddl . ..,'..,'.
'Ododdddodddxxl. . .':;.. .
:kooddddddddodk; . .. .. ..
:ddddddodddddddxd. . .........'.. ...
:xdddddddddddddddxxo. . ...'... . ......
'kodddddddddddddddx' ..''...................'.'. . ........
.dddodddddddddddoxl ...'...'...'....'''''......''............
,ddododdddddddddx: ......'''.... ....''.'..'..........
cxoxooddddddddddo .......... ...'..''....'..
cxlxdddddddododdc.......... . . ...''..'''.
:xdoodddddddxll:............ .:x. ..'..'..
.okxdddddd:............... .OXlx; .'.......
.okOxxx,.........'. .O;x. .'.........
;xOd..........'. . ...........
.'............. . ............
.............. ..............
...........''. ..'...........
...........''..... ....'............
..........''.''........''''..........
......'...''''''''''''.''........
...'..'.. ..'..''..
END RequestId: 6908f297-72af-143f-dd79-2b6128c5c428
REPORT RequestId: 6908f297-72af-143f-dd79-2b6128c5c428 Duration: 530.74 ms Billed Duration: 600 ms Memory Size: 128 MB Max Memory Used: 41 MB
null
Packaging and deploying
To package and deploy our Lambda, we can also use sam, but there's currently a bug packaging any layers that contain symlinks.
You can work around that by creating the layer zip yourself:
cd dependencies
zip -yr ../dependencies.zip .
cd ..
``
