Part 4: Add S3 event source

So far, we have been manually invoking the Lambda function ourselves in order to detect objects in the image and add the information to our database. However, we can automate this workflow using Lambda event sources so that the Lambda function is invoked every time an object is uploaded to the S3 bucket.

For this section, we will be doing the following:

Add Lambda event source for S3 object creation event

Change the Lambda function to be invoked whenever an object is uploaded to a S3 bucket via the on_s3_event decorator.

Instructions

1. In the app.py file, change the detect_labels_on_image signature to be named handle_object_created that accepts a single event parameter:

   def handle_object_created(event):
   

2. Update the decorator on handle_object_created to use the app.on_s3_event decorator instead and have the Lambda function be triggered whenever an object is created in the bucket specified by the environment variable MEDIA_BUCKET_NAME:

   @app.on_s3_event(bucket=os.environ['MEDIA_BUCKET_NAME'],
                    events=['s3:ObjectCreated:*'])
   def handle_object_created(event):
   

3. Add the tuple _SUPPORTED_IMAGE_EXTENSTIONS representing a list of supported image extensions:

   _SUPPORTED_IMAGE_EXTENSIONS = (
       '.jpg',
       '.png',
   )
   

4. Update the handle_object_created function to use the new event argument of type S3Event and only do object detection and database additions on specific image file extensions:

   @app.on_s3_event(bucket=os.environ['MEDIA_BUCKET_NAME'],
                    events=['s3:ObjectCreated:*'])
   def handle_object_created(event):
       if _is_image(event.key):
           _handle_created_image(bucket=event.bucket, key=event.key)
   
   
   def _is_image(key):
       return key.endswith(_SUPPORTED_IMAGE_EXTENSIONS)
   
   
   def _handle_created_image(bucket, key):
       labels = get_rekognition_client().get_image_labels(bucket=bucket, key=key)
       get_media_db().add_media_file(key, media_type=db.IMAGE_TYPE, labels=labels)
   

Validation

1. Ensure the contents of the app.py file is:

import os

import boto3
from chalice import Chalice
from chalicelib import db
from chalicelib import rekognition

app = Chalice(app_name='media-query')

_MEDIA_DB = None
_REKOGNITION_CLIENT = None
_SUPPORTED_IMAGE_EXTENSIONS = (
    '.jpg',
    '.png',
)


def get_media_db():
    global _MEDIA_DB
    if _MEDIA_DB is None:
        _MEDIA_DB = db.DynamoMediaDB(
            boto3.resource('dynamodb').Table(
                os.environ['MEDIA_TABLE_NAME']))
    return _MEDIA_DB


def get_rekognition_client():
    global _REKOGNITION_CLIENT
    if _REKOGNITION_CLIENT is None:
        _REKOGNITION_CLIENT = rekognition.RekognitonClient(
            boto3.client('rekognition'))
    return _REKOGNITION_CLIENT


@app.on_s3_event(bucket=os.environ['MEDIA_BUCKET_NAME'],
                 events=['s3:ObjectCreated:*'])
def handle_object_created(event):
    if _is_image(event.key):
        _handle_created_image(bucket=event.bucket, key=event.key)


def _is_image(key):
    return key.endswith(_SUPPORTED_IMAGE_EXTENSIONS)


def _handle_created_image(bucket, key):
    labels = get_rekognition_client().get_image_labels(bucket=bucket, key=key)
    get_media_db().add_media_file(key, media_type=db.IMAGE_TYPE, labels=labels)

Redeploy the Chalice application

Deploy the updated Chalice application.

Instructions

1. Run chalice deploy:

   $ chalice deploy
   Creating deployment package.
   Creating IAM role: media-query-dev-handle_object_created
   Creating lambda function: media-query-dev-handle_object_created
   Configuring S3 events in bucket media-query-mediabucket-fb8oddjbslv1 to function media-query-dev-handle_object_created
   Deleting function: arn:aws:lambda:us-west-2:123456789123:function:media-query-dev-detect_labels_on_image
   Deleting IAM role: media-query-dev-detect_labels_on_image
   Resources deployed:
     - Lambda ARN: arn:aws:lambda:us-west-2:123456789123:function:media-query-dev-handle_object_created
   

Validation

1. Upload the othersample.jpg image to the S3 bucket:

   $ aws s3 cp ../chalice-workshop/code/media-query/final/assets/othersample.jpg s3://$MEDIA_BUCKET_NAME
   

2. Use the get-item CLI command to ensure the othersample.jpg data was automatically populated in the DynamoDB table:

   $ aws dynamodb get-item --table-name $MEDIA_TABLE_NAME \
       --key '{"name": {"S": "othersample.jpg"}}'
   {
       "Item": {
           "name": {
               "S": "othersample.jpg"
           },
           "labels": {
               "L": [
                   {
                       "S": "Human"
                   },
                   {
                       "S": "People"
                   },
                   {
                       "S": "Person"
                   },
                   {
                       "S": "Phone Booth"
                   },
                   {
                       "S": "Bus"
                   },
                   {
                       "S": "Transportation"
                   },
                   {
                       "S": "Vehicle"
                   },
                   {
                       "S": "Man"
                   },
                   {
                       "S": "Face"
                   },
                   {
                       "S": "Leisure Activities"
                   },
                   {
                       "S": "Tourist"
                   },
                   {
                       "S": "Portrait"
                   },
                   {
                       "S": "Crowd"
                   }
               ]
           },
           "type": {
               "S": "image"
           }
       }
   }
   

   If the item does not appear, try running the get-item command after waiting for ten seconds. Sometimes, it takes a little bit of time for the Lambda function to get triggered.