Last month, I wrote an open letter to you acknowledging the frequency of outages Frame.io had been experiencing and outlined the steps we were taking to improve our performance and stability.
In an effort to remain transparent, I wanted to update you on our progress—what we’ve accomplished, and what we’re tackling next. As I said in that first letter, we know how much you rely on Frame.io to do your work, and how unacceptable those interruptions have been.
We’re grateful for your patience and encouraged that our efforts are paying off, in that we’ve seen a significant improvement in the stability of our platform over the past 30 days. This is the result of a number of important steps we’ve taken.
What we’ve accomplished
- Assigned additional personnel dedicated to performance initiatives
- Created a dedicated task force to API performance
- Increased our observability and active monitors in systems where we’ve seen reliability challenges
- Partnered with subject matter experts and AWS to tune our database configuration and vacuum policies
- Released a substantial update to Storage Accounting, significantly reducing our database load
- Performance tuned our top API hot paths, in some cases improving performance up to 50 percent and resulting in better end-user experience
- Completed migration of 4 major subsystems onto our new job system
- Isolated our legacy event bus into a separate Kubernetes cluster, allowing more capacity for API requests from external users
Next up
We continue to closely examine any and all slowdowns in our service—even those that occur briefly—in order to ensure that our performance is always what you expect in terms of speed and reliability.
Our next steps include:
- Focus on API performance and efficiency of larger batch operations
- Database connection pooling, multiplexing, and caching configurations
- Begin work on the infrastructure that powers our new Activity and Notifications systems
- Continue to move async operations to our new job system
- Architect our multi-region approach for storage and data
Once again, we sincerely apologize for the inconvenience you’ve experienced, and want to reinforce how serious we are about the stability of our product. I deeply appreciate your ongoing patience as we continue to address your concerns and feedback.
