gitw

programmatic photography

programmatic photography and automated video production solution

Imagine sleeping peacefully while technology is hard at work programmatically photographing vivid stars and beautiful Northern Lights dancing across the sky as the Earth spins. When the aurora is highly active, you would be alerted to wake up and step outside and enjoy every minute of the show. The folks at Geeks in the Woods imagined this, then created the solution to trasnform the vision into reality. The timelapse videos below are some of the many that have been automatically generated thus far.

The mission was to take photographs programmatically all day and night, and automatically generate the resulting photos into timelapse videos that will stream to the webcam page. Sony Alpha mirrorless cameras with Sigma low light lenses are used, which are specifically optimal for capturing vivid stars and active Northern Lights.

first solution for programmatic astrophotography

While the aurora can be forecasted to be visible with a high KPI, it may not be active where viewers are physically located. In addition, the aurora typically becomes active early in the morning - usually from 2am to 6am. Waiting up until 2am every night the Northern Lights are forecasted to be visible isn't ideal - even for the Brown brothers who tend to stay up late working. Thus, they wanted a system that could detect if the Northern Lights were actually visible from the property in Valdez, Alaska, while they and their guests were sleeping. They also wanted to share this unique Nordic experience with their friends and family around the world.

There are traditional webcams designed for outdoor applications like the Nest Outdoor camera and aUniFi Video cameras. These traditional outdoor webcams even support low light, but do so by increasing the ISO and other night vision tactics. These traditional webcams do not have the capability for adjusting the shutter speed to capture the stars vividly on a clear night or the aurora dancing across the night sky. The shutter speed should be increased and the ISO should be as low as possible for ultimate clarity.

There are also video cameras that can be implemented to record the sky all night long. However, video cameras are not designed for long exposure frames since they generally capture a specific number of frames per second. They would work when the aurora is extremely bright and highly active, but they would not be able to capture stars using longer exposures. And capturing multiple frames per second over the course of hours and days creates lots of wasteful data when the objects are not moving at a rate that requires multiple frames per second.

The Brown brothers were intrigued by the ability to programmatically use Sony mirrorless cameras coupled with low light lenses to capture long exposure photographs, and thereafter, automatically generate timelapse videos from the long exposure photographs.

programmatic and automated solution

The solution to programmatically capture photographs using Sony Alpha cameras and then automatically turn those photographs into timelapse videos is a distributed system relying on IOT devices and serverless technologies. The photographs are streamed real-time to the web and the timelapse videos are made available on the web immediately after their automated production.

In order to automatically detect the aurora in photographs, an image recognition tagging service provided by Amazon Web Services is used. If the photographs are tagged with "aurora", then the system will promptly notify opted-in users via text message or a call, of the aurora's immediate activity.

system architecture for programmatic photography and automated video production

development history

  • twins taking photographs while hiking in Alaska

    first automated photography tests

    July 2018

    programmatic photography with Sony A6300 from automated python script in Alaska

    As lightness decreased and darkness returned, the Brown brothers were excited to continue experimenting with astrophotography. They wanted to build a system to capture the stars and aurora while they were sleeping. After discovering they had the ability to onnect to their Sony Alpha cameras programmatically, they created the first script and began testing out automated photography.

    Sony A6300 taking photographs from automated python script outside in Alaska

  • building first prototype of weatherproof camera box

    constructing first weatherproof camera box

    August 2018

    building weatherproof camera box with front window to protect Sony Alpha camera and low light lens

    WWith the vast range of weather in Alaska and its ability to change quickly from sun, to clouds, to snow - they knew they needed to protect their Sony Alpha camera from the elements. They devised a plan to build an insulated box with a window on the front to keep the camera warm and dry throughout the winter and guard against condensation on the lens.

    weatherproof camera box taking photographs automatically over lake in Alaska

    The insulation along with a small heater inside the box prevented the cold from impacting the camera's performance, and kept the front window dry and clear of moisture from rain and snow.

  • Lee building, iterating and learning for automated camera system

    success with automated photography

    September 2018

    insulated weatherproof outdoor camera box with heat to protect Sony A6300 mirrorless camera

    The Python automated photography scripts originally ran on an Apple MacBook with OSX. Since OSX comes standard with the ability to run Python, this was a frictionless way to build, test and iterate upon the prototype.

    Sony A6300 mirrorless camera programatically controlled

    The script was finally hardened over many days and nights of resolving issues and making enhancements, and can now reliably capture photographs and automatically stream them to the Webcam page.

  • user

    deploying script to Raspberry Pi

    October 6th, 2018

    Programming Raspberry Pi for automated photography and timelapse videos

    Since the automated photography script was successfully running on the MacBook, it was time to migrate it to a simplified device that would be less expensive to risk operating outside in Alaska's sometimes extreme weather (even with best efforts to make a quality weatherproof box.) Deploying and operating the script on Raspberry Pi was surprisingly easy.

  • Bradley building and 3D printing new weatherproof camerabox

    printing a 3D weatherproof camera box begins

    October 8th, 2018

    Bradley Pizzimenti at Gennaker Systems, located in Anchorage, designed a CAD model camera box specially for the Aurora.cam project. On this day, he began the 3D-printing process.

programmatic photography

The programmatic photography system is comprised of a Sony Alpha camera and a Raspberry Pi. Since the Raspberry Pi runs Debian linux, it comes standard with Python 2. Thus, the script responsible for the programmatic photography is written in Python and deployed to the Raspberry Pi device.

Additional systems of a Sony Alpha camera and Raspberry Pi with the programmatic photography script can be deployed for specific views. The diagram below shows two deployments: one positioned to capture the Northern Lights and the other positioned to capture the daytime views overlooking the lake.

The Python script is in charge of connecting to the camera, taking a photograph with optimal shutter speed, and then transferring the photograph from the camera to the Raspberry Pi. The script also provides a management interface accessible via a web browser that allows users to view the latest image and stop/start functionality for the timelapse of captures.

The Python programmatic photography script takes photographs at specified intervals. The photography interval is every 60 seconds. By default, the script tries to first take a photograph at 200 ISO. If the photograph is too dark, the script increases the shutter speed to reach a clear photograph at 200 ISO. If the shutter speed has to be increased greater than 20 seconds while ISO is at 200, then ISO is increased to an appropriate value. Since Sony Alpha mirrorless cameras have max shutter speeds of 20 seconds (for optimal astrophotography), if a shutter speed length above 20 seconds is required, then the script sets the camera to use Auto ISO. (Note that the max shutter speed for the Sony A6300 camera is actually 30 seconds, but any longer than 20 seconds will cause star trailing.)

The Python script uses Sony's Camera Remote API beta. The unfortunate news is that Sony will no longer be providing updates and active support for the Sony Alpha Remote Camera API. Thus, this programmatic photography script will only work with Sony A7, Sony A7 II, Sony A6300 and Sony A6000 mirrorless cameras.

The Raspberry PI is connected to a local Wifi network. The script deployed on the Raspberry Pi uses the device's internet connection to upload each photograph after it has been captured to AWS S3 for cloud storage and web accessibility. These programmatic photographs are made available in real-time on the Webcam page.

at www.geeksinthewoods.com

automated video production

The Automated Video Production system is responsible for taking the processed photographs and generating timelapse videos. The process runs three times per day creating four videos:

  • An hour after sunrise, the system creates two timelapse videos. The first production is a sunrise timelapse video, made up of photographs from one hour prior to sunrise to one hour after sunrise. The second production is a night timelapse video, comprised of photographs from just after sunset of the prior day to sunrise of the current day.
  • Ah hour after sunset, the system creates a sunset timelapse video, comprised of photographs from one hour prior to sunset to one hour after sunset.
  • Right after midnight, a full 24-hour cycle timelapse video is created, comprised of all photographs from the prior day.

The autogenerated timelapse videos are uploaded to AWS S3 for cloud storage and web accessibility. While all the autogenerated timelapse videos are posted on this website, the best videos are published to our Geeks in the Woods YouTube channel.

serverless architecture

Since the Brown brothers are entrepreneurial software engineers, they prefer focusing on creating solutions with code and not wasting precious time managing servers. Thus, they are excited with the evolution and growing trends of serverless architecture.

As you can see in the above diagram, Photo Processing, Autogenerated Video Production, the API, and Web App are deployed on AWS Fargate rather than on AWS's old standard of EC2 instances. While EC2 instances provide access to the VM, Fargate only allows access to the task. Thus, deploying to EC2 only abstracts to the hardware and requires software engineers or their IT ops team to deploy and manage servers before software can be deployed and managed. Fargate abstracts the hardware level and abstracts to the task level. Software engineers no longer need to provision and manage servers before deploying and releasing software. With Fargate, software engineers can focus on deploying and releasing the tasks their software was designed to achieve.

Lee Brown, co-founder of Geeks in the Woods, shares his experience with Serverless Architecture in October 2018 in the presentation below.

weatherproof camera box

Electronics are not usually built to operate in extreme weather conditions and cold temperatures. Since there were no commercially-available weatherproof boxes for cameras, the Brown brothers had to build their own. Their first weatherproof camera box prototype was made out of wood and insulation board, and it was quite large. This prototype wasn't perfect, but after several enhancements and iterations, they were able to get a better understanding of the specific requirements that would go into creating an optimal version, which are listed below

  • sealed completely to prevent rain and melting snow from leaking into the interior
  • crystal clear front glass window
  • dimensions to provide space for 1 Sony Alpha mirrorless camera, and 1 Raspberry Pi devide
  • a screw fixed to the floor to secure the camera, inset 0.25"
  • a heat pad installed on the floor to keep components warm
  • the ability to open the box to conduct maintenance on equipment

With an understanding of the optimal design, they worked with Bradley Pizzimenti to turn their concept into a reality. This perfect weatherproof camera box is being 3D-printed in Anchorage.