https://solutr.com/tags/aws/ Recent content in Aws on Solutr.com https://solutr.com/%3Clink%20or%20path%20of%20image%20for%20opengraph,%20twitter-cards%3E https://solutr.com/%3Clink%20or%20path%20of%20image%20for%20opengraph,%20twitter-cards%3E Hugo -- 0.147.9 en Fri, 13 May 2016 18:23:00 +0200 https://solutr.com/posts/2016-05-13-s3-api-too-slow-xargs-to-the-rescue/ Fri, 13 May 2016 18:23:00 +0200 https://solutr.com/posts/2016-05-13-s3-api-too-slow-xargs-to-the-rescue/ <p>My days as a BigData engineer grew darker in the past few days. Now almost all critical data that our aggregations, partners and customers want to access is located in a bucket on Amazon&rsquo;s S3. Since that gets horribly expensive for the vast amounts of data we create on a daily basis, we lean towards not keeping much more than a month of active objects in the bucket, to meet our SLA&rsquo;s regarding backup and recovery, we keep them for about a year in Glacier afterwards.</p>