This Project is Deprecated

NOTE: This project is being deprecated in favor of Waffle. The new project is based off this project meaning all of the same configuration and APIs will continue to work as expected (at the time of writing this).

Migrating to Waffle

Update your dependency

If you are using Mix, simply change your dependency from arc to waffle. Waffle uses the same version tags, so there is no need to modify the version in your mix file.

Because Waffle is still Arc under the hood, any Arc-based storage providers, such as arc_gcs, will continue to work without any additional changes.

Update your configs

Replace any instances of config :arc to config :waffle. All configuration options remain the same (as of version 0.11.0) and do not need to be updated.

Update your code

Change any instances of Arc to Waffle except where it concerns the third-party storage providers. For example, Arc.Storage.GCS would continue to use Arc as the root namespace because that's defined outside the core library. If you wish to standardize the libraries, you can use an alias in your files to prepare for an eventual namespace conversion, e.g.

# "Renames" the Arc namespace to Waffle
alias Arc, as: Waffle

# Use the new library name as your normally would
Waffle.Storage.GCS

Arc

Arc is a flexible file upload library for Elixir with straightforward integrations for Amazon S3 and ImageMagick.

Browse the readme below, or jump to a full example.

Installation

Add the latest stable release to your mix.exs file, along with the required dependencies for ExAws if appropriate:

defp deps do
  [
    arc: "~> 0.11.0",

    # If using Amazon S3:
    ex_aws: "~> 2.0",
    ex_aws_s3: "~> 2.0",
    hackney: "~> 1.6",
    poison: "~> 3.1",
    sweet_xml: "~> 0.6"
  ]
end

Then run mix deps.get in your shell to fetch the dependencies.

Configuration

Arc expects certain properties to be configured at the application level:

config :arc,
  storage: Arc.Storage.S3, # or Arc.Storage.Local
  bucket: {:system, "AWS_S3_BUCKET"} # if using Amazon S3

Along with any configuration necessary for ExAws.

Storage Providers

Arc ships with integrations for Local Storage and S3. Alternative storage providers may be supported by the community:

Rackspace - https://github.com/lokalebasen/arc_rackspace
Manta - https://github.com/onyxrev/arc_manta
OVH - https://github.com/stephenmoloney/arc_ovh
Google Cloud Storage - https://github.com/martide/arc_gcs
Microsoft Azure Storage - https://github.com/phil-a/arc_azure

Usage with Ecto

Arc comes with a companion package for use with Ecto. If you intend to use Arc with Ecto, it is highly recommended you also add the arc_ecto dependency. Benefits include:

Changeset integration
Versioned urls for cache busting (.../thumb.png?v=63601457477)

Getting Started: Defining your Upload

Arc requires a definition module which contains the relevant configuration to store and retrieve your files.

This definition module contains relevant functions to determine:

Optional transformations of the uploaded file
Where to put your files (the storage directory)
What to name your files
How to secure your files (private? Or publicly accessible?)
Default placeholders

To start off, generate an attachment definition:

mix arc.g avatar

This should give you a basic file in:

web/uploaders/avatar.ex

Check this file for descriptions of configurable options.

Basics

There are two supported use-cases of Arc currently:

As a general file store, or
As an attachment to another model (the attached model is referred to as a scope)

The upload definition file responds to Avatar.store/1 which accepts either:

A path to a local file
A path to a remote http or https file
A map with a filename and path keys (eg, a %Plug.Upload{})
A map with a filename and binary keys (eg, %{filename: "image.png", binary: <<255,255,255,...>>})
A two-tuple consisting of one of the above file formats as well as a scope object.

Example usage as general file store:

# Store any locally accessible file
Avatar.store("/path/to/my/file.png") #=> {:ok, "file.png"}

# Store any remotely accessible file
Avatar.store("http://example.com/file.png") #=> {:ok, "file.png"}

# Store a file directly from a `%Plug.Upload{}`
Avatar.store(%Plug.Upload{filename: "file.png", path: "/a/b/c"}) #=> {:ok, "file.png"}

# Store a file from a connection body
{:ok, data, _conn} = Plug.Conn.read_body(conn)
Avatar.store(%{filename: "file.png", binary: data})

Example usage as a file attached to a scope:

scope = Repo.get(User, 1)
Avatar.store({%Plug.Upload{}, scope}) #=> {:ok, "file.png"}

This scope will be available throughout the definition module to be used as an input to the storage parameters (eg, store files in /uploads/#{scope.id}).

Transformations

Arc can be used to facilitate transformations of uploaded files via any system executable. Some common operations you may want to take on uploaded files include resizing an uploaded avatar with ImageMagick or extracting a still image from a video with FFmpeg.

To transform an image, the definition module must define a transform/2 function which accepts a version atom and a tuple consisting of the uploaded file and corresponding scope.

This transform handler accepts the version atom, as well as the file/scope argument, and is responsible for returning one of the following:

:noaction - The original file will be stored as-is.
:skip - Nothing will be stored for the provided version.
{executable, args} - The executable will be called with System.cmd with the format #{original_file_path} #{args} #{transformed_file_path}.
{executable, fn(input, output) -> args end} - If your executable expects arguments in a format other than the above, you may supply a function to the conversion tuple which will be invoked to generate the arguments. The arguments can be returned as a string (e.g. – " #{input} -strip -thumbnail 10x10 #{output}") or a list (e.g. – [input, "-strip", "-thumbnail", "10x10", output]) for even more control.
{executable, args, output_extension} - If your transformation changes the file extension (eg, converting to png), then the new file extension must be explicit.

ImageMagick transformations

As images are one of the most commonly uploaded filetypes, Arc has a recommended integration with ImageMagick's convert tool for manipulation of images. Each upload definition may specify as many versions as desired, along with the corresponding transformation for each version.

The expected return value of a transform function call must either be :noaction, in which case the original file will be stored as-is, :skip, in which case nothing will be stored, or {:convert, transformation} in which the original file will be processed via ImageMagick's convert tool with the corresponding transformation parameters.

The following example stores the original file, as well as a squared 100x100 thumbnail version which is stripped of comments (eg, GPS coordinates):

defmodule Avatar do
  use Arc.Definition

  @versions [:original, :thumb]

  def transform(:thumb, _) do
    {:convert, "-strip -thumbnail 100x100^ -gravity center -extent 100x100"}
  end
end

Other examples:

# Change the file extension through ImageMagick's `format` parameter:
{:convert, "-strip -thumbnail 100x100^ -gravity center -extent 100x100 -format png", :png}

# Take the first frame of a gif and process it into a square jpg:
{:convert, fn(input, output) -> "#{input}[0] -strip -thumbnail 100x100^ -gravity center -extent 100x100 -format jpg #{output}", :jpg}

For more information on defining your transformation, please consult ImageMagick's convert documentation.

Note: Keep this transformation function simple and deterministic based on the version, file name, and scope object. The transform function is subsequently called during URL generation, and the transformation is scanned for the output file format. As such, if you conditionally format the image as a png or jpg depending on the time of day, you will be displeased with the result of Arc's URL generation.

System Resources: If you are accepting arbitrary uploads on a public site, it may be prudent to add system resource limits to prevent overloading your system resources from malicious or nefarious files. Sin

Arc

Install / Use

README