Fifth & Harbor at Sunrise, San Diego

Experience the streets of downtown San Diego in this mash up of video, music, and tech.

Fifth & Harbor at Sunrise, San Diego


I've been thinking of a cool (and maybe useful) application that uses image recognition software. There are videos on YouTube that feature chill music and night time cruising through different cities. I thought this was a creative way of showing a city to those who don't live there and city streets are a perfect environment to test image recognition. I captured a long take of the downtown area of San Diego that features an uninterrupted experience. This creates the feeling you are flying through the streets of San Diego. In this Crafty Cloud article, I decided to mash video, music, and cloud tech together to create something exhilarating and beautiful.

The stack

(1) HD video was captured of the planned route using a GoPro Hero 7. I took advantage of the image stabilization software built in and didn't need a gimbal.
(2) Once the route was captured, I exported the individual frames into image files (.jpg), so they could be run through Amazon Rekognition.
(3) For reliability and speed of processing, I ran the image processing code on an EC2 instance.
(4) Each frame/image was uploaded to S3.
(5) I used the Python SDK (boto3) to run the images through Amazon Rekognition and saved the responses on the server. The results contained detected objects and scenes, along with confidence scores and bounding boxes.
(6) Finally, we come full circle and merge the frames back into a single video file with the bounding boxes displayed.
(7) Open Shot is an open source video editing tool and was used to create the final video.

The craft

Lessons learned

I'm not a videographer by trade and learned a lot by trial and error. Here are some things I learned regarding this Crafty Cloud article:

  • For darker shots decrease the fps to allow more light to come in, and for day shoots to increase the fps to let less light in.
  • Plan your route and stick to it. I drove the route in the video several times before getting it right, and if one thing goes wrong in the long take, you have to start over.
  • Use the GoPro Hero 7, it was an impressive piece of technology to shoot with. It made it easy to shoot quality video and simplified the rig by replacing a gimbal with software-based stabilization.
  • When using Open Shot Video Editor convert your m4a to mp3 or it will cause the application to continually crash!

Shout out!

  • James Lawrence - Roomate
    Thank you James for keeping me company during the late night/early morning rides and providing feedback on the process (he also helped me figure out how to use the GoPro).

  • Open Shot Video Editor
    The team at Open Shot made an intutive UI so people new to video editing and experienced professionals can create quickly. Thank you for allowing us to benefit from your open mindset!