krellor

That’s a good takeaway. AWS is the ultimate Swiss army knife, but it is easy to misconfigure. Personally, when you are first learning AWS, I wouldn’t put more data in then you are willing to pay for on the most expensive tier. AWS also gives you options to set price alerts, so if you do start playing with it, spend the time to set cost alerts so you know when something is going awry.

Have a great day!

So you just asked the most confusing thing about AWS service names due to how names changed over time.

Before S3 had an archival tier, there existed a separate service that AWS named AWS Glacier Storage, and then renamed to AWS S3 Glacier.

Around 2012 AWS started adding tiers to S3 which made the standalone service redundant. I received you look at S3 proper unless you have something like a Synology that can directly integrate with the older job based API used by the original glacier service.

So, let’s say I have a 1TB archival file, single tarball, and I upload it to a brand new S3 bucket, without version, special features, etc, except it has a life cycle policy to move objects from S3 standard to S3 Glacier instant access after 0 days. So effectively, I upload the file and it moves to Glacier class storage.

The S3 standard is ~$24/tb/month, and lets say worst case scenario our data sits on standard for one whole day before moving.

$0.77+$0.005 (API cost of the put)

Then there is the lifecycle charge to move the data from standard to glacier, with one request per object each way. Since we only have one object the cost is

$0.004 out of standard
$0.02 into glacier

The cost of glacier instant tier is $4.1/tb/month. Since we would be there all but one day, the cost on the first bill would be:

$3.95

The second month onwards you would pay just the $4.1/month unless you are constantly adding or removing.

Let’s say six months later you download your 1tb archive file. That would incur a cost of up to $30.

Now I know that seems complicated and expensive. It is, because it is providing services to me in my former role as director of engineering, with complex needs and budgets to pay for stuff. It doesn’t make sense as a large-scale backup of personal data, unless you also want to leverage other AWS services, or you are truly just dumping the data away and will likely never need to retrieve it.

S3 is great for complying with HIPAA, feeding data into a cdn, and generally dumping data around in performant way. I’ve literally dropped a petabyte off data into S3 and it just took it and did its thing.

In my personal AWS account I use S3 as a place to dump cache contents built by lambda functions and served up by API gateway. Doing stuff like that is super cheap. I also use private git repos (code commit), private container registry (ecr), and container host (ECS), and it is nice have all of that stuff just click together.

For backing up my personal computer, I use iDrive personal and OneDrive, where I don’t have to worry about the cost per object, etc. iDrive (not an Apple service) let’s you backup multiple devices to their platform and keeps them versioned.

Anyway, happy to help answer questions. Have a great day.

Thanks for posting. I just deployed to my container host in AWS ECS and it’s working well in my testing. Very easy deployment with docker.

It’s complicated. I gave the most expensive pricing, which is their fastest tier and includes stripping across three availability zones and guarantees 11 nines of data durability. Additionally, the easy integration with all other AWS services and the feature richness of S3 buckets makes it hard to do a fair apple to apple comparison unless you really have well defined needs. So I gave the highest price to keep it simple, and for someone who says they just have a few GB, any cost should be trivial.

AWS S3 has a free tier that covers the first 5Gb. I recommend it because the AWS cli is excellent, and gives you lots of options for how to sync your data. The pricing is $0.023/GB/month after the free tier. It can be overwhelming to get into AWS but it is worth it to have access to the ultimate IT service swiss army knife.

I run a lot of tech, containerized workloads in AWS, home firewalls running on protectli boxes for all my family around the country, wireless controllers to run APs for my family around the country, but as I got older one thing I stopped rolling my own instance of was data backups. My data backs up to OneDrive and iDrive, so two copies of my data. My wife has access to both via shared credentials in a 1password folder that she knows how to access and uses regularly.

As I got older and I had a family, the pictures of our kids, wills, financial records, insurance documents are all just too important. Every service that holds my data is paid annually for less than $200/year total and auto renews. She could call either company and prove ownership if she ever did need help getting access. Also, I can easily share folders to her.

It’s funny how getting older makes you think of the sorts of issues enterprise teams have. Don’t implement solutions where you will be one deep, have a succession plan, and complexity is the enemy. All the tech I run now is fun and helpful, but can be replaced with a trip to BestBuy. The data and pictures however must be easy to retrieve for her.

So I don’t have a good self hosted solution for you other than to say that at some point it’s ok to change your strategy. And if you are worried about privacy, you can encrypt subsets of your data locally before it is backed up.

One thing I’ll add is I often found it helpful to glide them in which helps straighten the wires, then pull them out and trim the ends to be even. Then put back in connector, and make sure all pins touch all wire ends.

That’s interesting. Any chance your ISP could have been qos’ing streaming video? Although Singapore would be about the one place where a VPN concentrator would help; it is pretty much the big fiber hub in that local region for East, West, North connectivity.

Fiber map

I’ve only ever used Oracle cloud in an enterprise environment, so I don’t know what features you have available. I’m also much more familiar with AWS. But you should be able to create a proxy endpoint in your present region, and traverse the cloud providers internal network. That would likely improve your streaming. You could also create a VPN endpoint in your current region and terminate your traffic inside your cloud providers network, but that would add protocol overhead.

I would look at tools like iperf to look at your packet loss because being further from your server will increase latency, but shouldn’t impact the streaming unless you also have packet loss.

I used to oversee WAN and peering operations for a large multi site. Residential ISPs almost never respond to reports of inefficient routes unless you are one of their peers, big business customers, or you really know your stuff and send in a detailed report showing asymetric routes, bad bgp info, etc.

As far as a VPN goes, that probably wouldn’t help either. You will probably increase the number of hops and latency. Your route will still egress your isp gateway, to your VPN provider, then travel over the Internet and to your remote server, while adding additional protocol overhead. Yes, it is remotely possible that there is an improved link from his regional VPN node to his remote provider, but unlikely from my experience with traffic engineering.