Part 5: Add S3 delete event handler

Now that we are automatically importing uploaded images to our table, we need to be able to automatically delete images from our table that get deleted from our bucket. This can be accomplished by doing the following:

Add Lambda function for S3 object deletion

Add a new Lambda function that is invoked whenever an object is deleted from the S3 bucket and if it is an image, removes the image from the table.

Instructions

1. In the app.py file add a new function handle_object_removed that is triggered whenever an object gets deleted from the bucket and deletes the item from table if it is an image:

@app.on_s3_event(bucket=os.environ['MEDIA_BUCKET_NAME'],
                 events=['s3:ObjectRemoved:*'])
def handle_object_removed(event):
    if _is_image(event.key):
        get_media_db().delete_media_file(event.key)

Verification

1. Ensure the contents of the app.py file is:

import os

import boto3
from chalice import Chalice
from chalicelib import db
from chalicelib import rekognition

app = Chalice(app_name='media-query')

_MEDIA_DB = None
_REKOGNITION_CLIENT = None
_SUPPORTED_IMAGE_EXTENSIONS = (
    '.jpg',
    '.png',
)


def get_media_db():
    global _MEDIA_DB
    if _MEDIA_DB is None:
        _MEDIA_DB = db.DynamoMediaDB(
            boto3.resource('dynamodb').Table(
                os.environ['MEDIA_TABLE_NAME']))
    return _MEDIA_DB


def get_rekognition_client():
    global _REKOGNITION_CLIENT
    if _REKOGNITION_CLIENT is None:
        _REKOGNITION_CLIENT = rekognition.RekognitonClient(
            boto3.client('rekognition'))
    return _REKOGNITION_CLIENT


@app.on_s3_event(bucket=os.environ['MEDIA_BUCKET_NAME'],
                 events=['s3:ObjectCreated:*'])
def handle_object_created(event):
    if _is_image(event.key):
        _handle_created_image(bucket=event.bucket, key=event.key)


@app.on_s3_event(bucket=os.environ['MEDIA_BUCKET_NAME'],
                 events=['s3:ObjectRemoved:*'])
def handle_object_removed(event):
    if _is_image(event.key):
        get_media_db().delete_media_file(event.key)


def _is_image(key):
    return key.endswith(_SUPPORTED_IMAGE_EXTENSIONS)


def _handle_created_image(bucket, key):
    labels = get_rekognition_client().get_image_labels(bucket=bucket, key=key)
    get_media_db().add_media_file(key, media_type=db.IMAGE_TYPE, labels=labels)

Redeploy the Chalice application

Deploy the updated Chalice application with the new Lambda function.

Instructions

1. Run chalice deploy:

   $ chalice deploy
   Creating IAM role: media-query-dev-handle_object_removed
   Creating lambda function: media-query-dev-handle_object_removed
   Configuring S3 events in bucket media-query-mediabucket-fb8oddjbslv1 to function media-query-dev-handle_object_removed
   Resources deployed:
     - Lambda ARN: arn:aws:lambda:us-west-2:123456789123:function:media-query-dev-handle_object_created
     - Lambda ARN: arn:aws:lambda:us-west-2:123456789123:function:media-query-dev-handle_object_removed
   

Verification

1. Delete the uploaded othersample.jpg object from the previous part:

   $ aws s3 rm s3://$MEDIA_BUCKET_NAME/othersample.jpg
   

2. Use the scan CLI command to ensure the object is no longer in the table:

   $ aws dynamodb scan --table-name $MEDIA_TABLE_NAME
   {
       "Items": [
           {
               "name": {
                   "S": "sample.jpg"
               },
               "labels": {
                   "L": [
                       {
                           "S": "Animal"
                       },
                       {
                           "S": "Canine"
                       },
                       {
                           "S": "Dog"
                       },
                       {
                           "S": "German Shepherd"
                       },
                       {
                           "S": "Mammal"
                       },
                       {
                           "S": "Pet"
                       },
                       {
                           "S": "Collie"
                       }
                   ]
               },
               "type": {
                   "S": "image"
               }
           }
       ],
       "Count": 1,
       "ScannedCount": 1,
       "ConsumedCapacity": null
   }
   

   If the item still appears, try running the scan command after waiting for ten seconds. Sometimes, it takes a little bit of time for the Lambda function to get triggered. In the end, the table should only have the sample.jpg item.