Rails API & React Frontend for Image Uploads to Amazon S3

Goal

To spin up a new Rails API with a React frontend that allows for a single image upload that is sent back to the Rails API for processing with storage on Amazon S3.

Initial Setup

> mkdir image_upload_example && cd image_upload_example

# Rails API Setup - Rails 6.1
> rails new backend --api --database=postgresql && cd backend && bin/setup && cd ..

# React Frontend Setup
> yarn create react-app frontend --template typescript

Now that we have a base set for our frontend and backend let’s get a Procfile set up that will allow us to boot these two application together. I like using the Procfile in conjunction with overmind, I highly recommend it.

Procfile

In the root of your entire project create a new file Procfile and add the following.

backend: cd backend && bin/rails server
frontend: cd frontend && yarn start

From theis same directory you can now run: overmind start and it will go boot both applications and apply a default port of 5000 to the Rails API and 5100 to the React frontend.

overmind initial setup

Rails API Backend Setup

We are going to use Active Storage to facilitate the uploading and storage of files to Amazon S3.

# From the backend directory
> bin/rails active_storage:install && bin/rails db:migrate

Next we will want to set up Active Storage to talk with Amazon S3.

In config/storage.yml we will want to declare an Amazon service. It is alrerady defined in this file you will need to uncomment it.

...

amazon:
  service: S3
  access_key_id: <%= Rails.application.credentials.dig(:aws, :access_key_id) %>
  secret_access_key: <%= Rails.application.credentials.dig(:aws, :secret_access_key) %>
  region: your_region
  bucket: your_own_bucket
...

Then in config/environments/development.rb we need to tell Active Storage which service to use. For the purposes of this demo and what I wanted to see locally I set this to the amazon service instead of the local disk storage.

# Change the following line
config.active_storage.service = :local
# to
config.active_storage.service = :amazon

You will also need to add, gem "aws-sdk-s3", require: false to your Gemfile.

Another gem to install while in there is the rack-cors. It is already listed in the Gemfile you will need to uncomment it.

For the sake of this demo we will take the easiest route to get cors setup to allow the React frontend talk to the backend.

In config/initializers/corrs.rb you can uncomment the code there to make it look like the below.

Rails.application.config.middleware.insert_before 0, Rack::Cors do
  allow do
    origins '*'

    resource '*',
      headers: :any,
      methods: [:get, :post, :put, :patch, :delete, :options, :head]
  end
end

We are going to now add a very basic model and controller for Recipes.

> bin/rails g model Recipe name:string --no-test-framework
> bin/rails g controller recipes --no-helper --no-assets --no-test-framework
> bin/rake db:migrate

To get an image attached to this new Recipe object we need to add has_one_attached :image

class Recipe < ApplicationRecord
  has_one_attached :image
end

In addition to adding the image attribute to the model we need to modify the controller to create the recipe and allow an image attribute through the strong params.

class RecipesController < ApplicationController
  def create
    recipe = Recipe.create!(recipe_params)
  end

  private

  def recipe_params
    params.require(:recipe).permit(:name, :image)
  end
end

React Frontend Setup

First let’s install a few dependancies we want to use in this demo.

# Formik for default form behavior
> yarn add formik
# React Dropzone for image uploading
> yarn add react-dropzone
# Styled Components for some base styling of React Dropzone
> yarn add styled-components @types/styled-components

For this we are going to take over the default App page and put all the functionality there.

The App.tsx will look something like below:

import React, { useState, useCallback } from "react";
import { Formik, Field, Form, FormikHelpers } from "formik";
import { DropzoneRootProps, FileWithPath, useDropzone } from "react-dropzone";
import styled from "styled-components";

interface Values {
  name: string;
  image?: FileWithPath;
}

const getColor = (props: DropzoneRootProps) => {
  if (props.isDragAccept) {
    return "#00e676";
  }
  if (props.isDragReject) {
    return "#ff1744";
  }
  if (props.isDragActive) {
    return "#2196f3";
  }
  return "#eeeeee";
};

const AppContainer = styled.div`
  margin: 50px;
`;

const FieldContainer = styled.div`
  margin: 5px;
`;

const Container = styled.div`
  flex: 1;
  display: flex;
  flex-direction: column;
  align-items: center;
  padding: 20px;
  border-width: 2px;
  border-radius: 2px;
  border-color: ${(props) => getColor(props)};
  border-style: dashed;
  background-color: #fafafa;
  color: #bdbdbd;
  outline: none;
  transition: border 0.24s ease-in-out;
`;

const ImageContainer = styled.img`
  max-width: 200px;
  max-height: 200px;
`;

const App = () => {
  const [imagePreview, setImagePreview] = useState('');
  const [file, setFile] = useState('');
  const onDrop = useCallback((acceptedFile) => {
    setFile(acceptedFile[0]);
    setImagePreview(URL.createObjectURL(acceptedFile[0]));
  }, []);
  const {
    getRootProps,
    getInputProps,
    isDragActive,
    isDragAccept,
    isDragReject,
  } = useDropzone({
    multiple: false,
    accept: "image/*",
    onDrop,
  });

  const preview = (
    <div>
      <ImageContainer src={imagePreview} alt="recipe" />
    </div>
  );

  const onSubmit = (
    values: Values,
    { setSubmitting }: FormikHelpers<Values>
  ) => {
    let formData = new FormData();

    formData.append("recipe[name]", values.name);
    formData.append("recipe[image]", file);

    const requestOptions = {
      method: "POST",
      body: formData,
    };

    fetch("http://localhost:5000/recipes", requestOptions);
  };

  return (
    <AppContainer>
      <h1>Homebrew Recipe</h1>
      <Formik
        initialValues={{
          name: "",
        }}
        onSubmit={onSubmit}
      >
        {({ setFieldValue }) => (
          <Form>
            <FieldContainer>
              <label htmlFor="name">Name</label>
              <Field id="name" name="name" placeholder="IPA" />
            </FieldContainer>
            <div className="container">
              <Container
                {...getRootProps({
                  isDragActive,
                  isDragAccept,
                  isDragReject,
                })}
              >
                <input {...getInputProps()} />
                <p>Drag 'n' drop an image here, or click to select an image</p>
              </Container>
              {imagePreview && (
                <div>
                  <h4>Image Preview</h4>
                  {preview}
                </div>
              )}
            </div>

            <button type="submit">Submit</button>
          </Form>
        )}
      </Formik>
    </AppContainer>
  );
};

export default App;

A main point to notice in the above code is that with Formik and React Dropzone during the onSubmit you have to take the collected values and create a FormData object to get the image sent back to the backend.

Here is a minimal working version getting an image uploaded from the frontend to the backend to aws.

working example

Rails & Postgres - Selectively Restoring From a Compressed pg_dump File

Problem

You have a backup of a postgres database from the pg_dump command that was output via the custom format resulting in a compressed file.

Now you want to take this file to another environment and don’t need certain tables fully restored with data.

Solution

namespace :db do
  desc "Selectively remove passed in tables data from being imported on restore"
  task selective_restore: :environment do |_t, args|
    raise("Don't run in production") if Rails.env.production?

    config = ActiveRecord::Base.configurations[Rails.env]
    database = config['database']
    user = config['username'] || nil
    password = config['password'] || nil
    port = config['port'] || 5432
    host = config['host'] || 'localhost'
    db_dump_filepath = Rails.root.join('tmp', 'db.dump')
    db_list_filepath = Rails.root.join('tmp', 'db.list')
    db_bak_filepath = Rails.root.join('tmp', 'db.list.bak')s

    `pg_restore -l #{db_dump_filepath} > #{db_list_filepath}`

    args.extras.each do |arg|
      `sed -i.bak '/TABLE DATA public #{arg.downcase}/ s/^/;/' #{db_list_filepath}`
    end

    pg_restore_command = [].tap { |rc|
      rc << "PGUSER=#{user}" if user
      rc << "PGPASSWORD=#{password}" if password
      rc << "pg_restore -h #{host} -d #{database}"
      rc << "--verbose --clean --no-acl --no-owner"
      rc << "-L #{db_list_filepath} #{db_dump_filepath}"
    }.join(' ')

    `#{pg_restore_command}`

    `rm #{db_list_filepath} #{db_bak_filepath}`
  end
end

Details (If you’re interested…)

Here is what a compressed pg_dump file looks like.

compressed pg_dump file

We want to take this compressed file and push it through the pg_restore command with the -l flag to create a .list file.

`pg_restore -l #{db_dump_filepath} > #{db_list_filepath}`

This will create a new file db.list and this file is a readable version that lists the contents of the archive.

db list file

Now we can take this newly created list file and run it through linux’s sed command to comment out each table passed in via the args to prevent that data from being imported on restore.

This is being done in the code block here:

args.extras.each do |arg|
  `sed -i.bak '/TABLE DATA public #{arg.downcase}/ s/^/;/' #{db_list_filepath}`
end

All this command is doing is using regex to look through the file to find the table passed in and if it is found add a ; to the beginning of the line. This comments the line out.

The sed command needs the -i.bak which creates a backup of the file before making these changes. Since this is used locally or in a non-production environment I’m not concerned with this being overwrote after each iteratiton. I delete this file once the task is complete.

When the db.list file is done being modified we can pass that file in to the pg_restore command with the -L option with the original db.dump file.

This is an example of what the command looks like that is built via the [].tap code block.

pg_restore -h localhost -d selective_db_restore_development --verbose --clean --no-acl --no-owner -L ./db.list ./db.dump

pg_restore will use that list file to only restore the elements and tables listed in that file.

Example Code

I have an example repo up on github if you wanted to check it and see this work.

In this repo I seeded a database with a bunch of data via the faker gem.

You can find the compressed db dump file lib/tasks folder.

You can run the following and it will restore all the tables.

bin/rake db:selective_restore

Then if you want to see it exclude tables a command like

the following can be run.

bin/rake db:selective_restore[hops,fermentables]

You will see when looking at the database after running this no data in the hops or fermentables table.