I've had it for about 5 years now. I like the size of it and it's surprisingly sturdy, but I'm not overly fond of the glass top, because it's so hard to keep clean.
If you genuinely help people and they think that help has given them some leverage in life then most (not quite all) will downplay your contribution. Anything else and they are diminishing their own self esteem.
None of your users will help you moneterise what you have - they want it for free. You have to do it yourself.
Physician - heal thyself.
Monetization of a website is a problem that a lot of businesses struggle with, even Facebook.
I took a look through the websites in your profile. Your peers might be afraid to tell you this, but I'll give it to you straight: they are awful. The color schemes are bizarre, the code doesn't follow any standards, and they don't provide value. At least that's how they appear to me.
Getting positive reviews and more downloads creates a viral effect as you get more prominence in the market.
But I'm just being pedantic. Congrats on the growth.
congrats on the success.
I mean I got that you have a great product and keep improving it and people will stay - but you have to get your users from somewhere.
Was it all just people who found the app organically in the app store? No PR or marketing of any kind?
In Android if you prefix a directory name with a dot, its content will not appear on Galleries and other common apps (some will let you choose to do so in settings). You can also create a .nomedia empty file in a directory to hide it.
Can you share a graph...would be interesting to see the growth velocity curve.
Btw, why the name of the iOS app is "Hide Pictures and Photos with KeepSafe", but the Android version is just "Hide pictures with KeepSafe"?
In my iPhone, when I search "hide photo" I can't find your app, so the keyword "Photos" is therefore useless?
But having worked at and close to major corporations they are often slow to make decisions, and they often choose something that is not best value for money or best tech.
If you really do have a disruptive solution you might want try to work together with (much) smaller companies. You will have an easier time to iterate your product with them since you are basically talking directly to (hopefully) passionate techies that understand the problems and threats and has some clue about possible solutions.
Grow with these customers. You can make joint case studies about how you succeed together. This is always valueable to show other possible customers.
Best of luck.
IMHO you only need your own email server for extensive logging of email traffic for regulatory reasons. Or if you have a corporate environment with people who need shared mailboxes/centrally managed email.
edit: IMAP move works fine btw. It can get a little weird with the way it handles multiple labels though.
edit2: Gmail presents labels as folders in IMAP. So multiple labels = same email in multiple folders. There are also Gmail specific IMAP folders like All Mail that are under a Gmail folder. I encourage you to take a look at it using Tbird or another IMAP client. You will have some loss of structure when moving away from tags and into folders so just make sure you know what you're losing and where.
It's been a couple years since i tried it, but I used <name>.tld for email like you're considering. I never received more than a few emails a day, but it was greatly outnumbered by spam. probably 100:1. gmail and Google Apps mail is the tits with spam. false positives (or negatives) are very rare on the order of 1 every month or 2... probanly 1:1000 emails.
If you use Google Apps for email, it syncs right in with android devices and is subject to the same security (good or bad) of gmail. Even if you're an experienced sysadmin (irrelevant of os) it will take an order of magnitude more time to setup and maintain the infrastructure than if you just use google.
yeah yeah i know, i'm suckin their junk pretty hard... but the beauty of it is that their stuff 'just works'. It's hard to beat.
Been thinking about it for a while though, and haven't yet made any decisions yet. sigh
* privacy and security: you don't have any with email. It moves in plaintext across the Internet. It'd be silly to pretend that your own mail server doesn't help some, but you might be better off feeling insecure. You'll keep sensitive data out of your email, and if you have to move very sensitive data across email, encrypt it. If you move unencrypted data across the Internet, assume it's become public information. Personally, I feel that Google is good with privacy and security. I'm happy with it. http://support.google.com/mail/bin/answer.py?hl=en&answe... I use two-factor authentication on my google account, and it alerts me if it's been accessed from unusual IPs/locations. Frankly, Google does a better job of protecting my privacy and security than I would. I wouldn't audit my log files. I believe there's a fair chance my privacy and security would go down by switching off of gmail.
So, I went to namecheap, registered a domain, and it forwards email to my existing email address. If Google locks me out of my account permanently, there's a layer of indirection with my gmail account. I'll point namecheap's mail forwarder at my new email account.
If you lack ideas(totally normal) think of small problems in your daily routine that could be improved/fixed and do it.
Good luck, and don't give up, seriously. When you start getting rewarded for your work this is one of the most emotionally(and maybe even financially) remunerative jobs.
Peter Seibel's Practical Common Lisp is available online (and as a book) and provides a lot good advice including a link to lispbox, which in the best "learn the hard way" tradition, requires you to finally learn emacs.
PG's own Ansi Common Lisp is more of a traditional textbook as well as a reference to the language. The exercises are thought provoking. It is available through Amazon including used copies.
Because Lisp originated as a way of describing Turing machines, the discussions tend to cover a lot of the theoretical background and deeper insight into how particular features may be used...or at least that's my impression.
1. Scratch the really high level stuff and start digging into some C or x86 assembly ( with GCC and binutils), read the coreutils and glib source and program some of your own system tools ( like find or ls ). This will give you a really solid understanding of memory and hardware internals which you can use to your advantage in other software apps.
hope this helps and good luck!
Since you already know some Python, sign on to Udacity's (free) CS 212 Design of Computer Programs by Peter Norvig. It has 7 weeks worth of classes and teaches you a tonne in that time. It is a class targeted specifically at "I know some basic programming but what do I do to get to the next level?" people.
The official version of the class is over, but nothing stops you from joining today, watching the videos, doing the assignments etc.
Since one of your aims is to 'learn from great coders' this should be right up your alley since Peter is one of the best developers in the world and you can see him developing multiple programs and refining them step by step till they are gems of elegant code, and you can trawl the forums to see questions people asked and answers to them.
Once you work through the class - should take you 7 weeks if you are disciplined -and you understand all the techniques demonstrated there, you'll be in good shape to start/join an open source project in a domain of your interest.
And oh prepare to have your mind blown. Peter Norvig is one seriously good developer.
Start S M A L L. You say you try to build things that are over your head. Don't do that. Build things that are just about your level, while still providing a challenge. You're not going to come out of the gate with something like Kenneth Reitz' "Requests" framework. You're going to have years of learning and experience ahead of you before you can produce work like that (unless you're a prodigy, which almost none of us are).
Most importantly, do not give up. You are not stupid or incapable of programming if you hit a wall and can't seem to get by it. We all hit these walls, at every level of our craft. The issue is recognizing when you have the ability to climb the wall, to blow a hole through it, walk around it, or just go back the way you came and pick another path.
You could try applying for unpaid (or lightly paid) internships as more of a learning experience. If companies don't think you'll be a drain on resources, I would imagine certain companies would love to have someone who is eager to learn hanging around to take minor tasks and to be mentored for a few hours a week. The key there is finding the right company with that kind of time/support.
If you are in the financial situation to do so (especially if you are still in school) find an (people won't like me saying this) unpaid internship. At 17, in high school, living in the middle of nowhere, I put up my resume (with very little web development skills) on HackerNews saying I would work for free, remotely, about 8 hours a week. I got 10 responses. I took one doing Ruby on Rails with an organization that did pair programming exclusively and I learned more than I did in years of Java classes. With this training, I was able to find a two month paid internship in Ruby on Rails this summer, and hopefully this chain will continue upward into jobs. I cannot explain the value of on-the-job learning and experience, and if an unpaid internship is what it takes to get you there, I really recommend it.
In the meantime:-Learn git inside and out if you don't know it-Pursue a web development framework-If you have programmer friends, get together and code with them. Productive people make you more productive.-Whatever you do, put yourself out there. The more you are out there, the more opportunities will come your way.
I think this is perhaps the first thing you should approach differently.Begin with simple projects that you know are within your reach, even if they are dead simple (make a tic-tac-toe in Django). Completing those projects will boost your morale so you will stay motivated.Also I would say that you need to build up your Github account into a professional signature that someone would look at and see where your interests are, whether you have potential as an intern / junior dev who may grow to become a good dev, with some coaching and mentorship.Also, the world is moving towards agile and you are going to impress a lot of people if your activity on Github shows that you have the ability to build software in an agile way.
also, start learning python and don't stop until you can actually build something you like. experiment.
"Start Small". But what does that mean really ? It means that focus on a specific topic and give yourself a task/side project out of it. For example, "Build a user registration system". Now pick the language/framework etc. You seem to be using Python/Django already so that could work in your case.
Keep hacking even if not finished. "It is about the journey not the destination" (cheap attempt at using a favorite quote).
I don't think signature campaigns will do any good since not even pressure from the US and Turkey has slowed down the bloodshed so do we really think they will care about a bunch of virtual signatures collected online?
There's a few more viable options that should be considered:
1) Leverage someone who has influence with the Syrian government to take up the cause. Given Bassel has been an open source contributor, maybe look at which tech companies are still contracted by the Syrian government and try to get them to lend a voice of support. Make sure to play up the positive PR that releasing Bassel will have on improving Syria's image.
2) Stop collecting signatures and start collecting money to work the back channels. Let's be honest here, corruption is rife (high corruption index on Transparency International study) and the situation pretty chaotic, if you really want to free Bassel, consider a pragmatic approach.
In fact, it would have been better if you had linked to https://creativecommons.org/weblog/entry/33119 which vouches for http://freebassel.org/
Creative Commons supports efforts to obtain the release of Bassel Safadi, a valuable contributor to and leader in the technology community. Bassel's expertise and focus across all aspects of his work has been in support of the development of publicly available, free, open source computer software code and technology. He pursues this not only through his valuable volunteer efforts in support of Creative Commons, but in all of his work in the technology field. Through his efforts, the quality and availability of freely available and open technology is improved and technology is advanced.
You wouldn't partner with a coder who can't code and is learning on the job so why would a technical founder want to partner with a non-technical "business" cofounder who doesn't know anything, worse, think he does?
This includes actual knowledge on things like knowing how to actually acquire customers and said metrics, not just be able to recite shit you read off HN or some blogs. Actual experience...
2. Without knowing how to code, at least be extremely knowledgeable about technical topics. Steve Jobs probably couldn't code squat but he understands enough. That's all I need too.
3. Extremely fast learner and can break habits quick. This one almost every person I've met failed at. Very few actually have been able to do this and they are gems.
- Fund Raising and Financing - Customer Acquisition, Marketing, Content Marketing, PR- Metrics Metrics Metrics
Basically - the Start; the Operations (customer facing, not dev obviously); the Intelligence (where the overlap is and the co-founders work closely together on sharing their perspectives).
Ideas are worthless without sales, marketing, money (earned or otherwise), and execution.
A -> Always be learning. Always.
B -> Books. Books are good. O'Reilly is great. Other books can be great too.
C -> Competence -> Comprehension -> Confidence
(ok, i'm cheating. this is four things)
4 -> in my opinion, competence is simply a process where you keep trying new things over time. you keep learning as you go. always.
2) Marketing experience in that same domain
3) Someone who is honest and genuine
During the night of 30.06.2012 to 01.07.2012 our internalmonitoring systems registered an increase in the level ofIT power usage by approximately one megawatt.
The reason for this huge surge is the additional switchedleap second which can lead to permanent CPU load on Linuxservers.
According to heise.de, various Linux distributions areaffected by this. Further information can be found at:http://www.h-online.com/open/news/item/Leap-second-Linux-can...
In order to reduce CPU load to a normal level again, arestart of the whole system is necessary in many cases.First, a soft reboot via the command line should beattempted. Failing that, you have the option of performinga hardware reset via the Robot administration interface.For this, select menu item "Server" and the "Reset" tabfor the respective server in the administration interface.
Please do not hesitate to contact us, should you have anyqueries.
Hetzner Online AGStuttgarter Str. 191710 Gunzenhausen / Germanyinfo@hetzner.dehttp://www.hetzner.com
During the night of 30.06.2012 to 01.07.2012 our internal monitoring systems registered an increase in the level ofIT power usage by approximately one megawatt.
The reason for this huge surge is the additional switched leap second which can lead to permanent CPU load on Linux servers.
According to heise.de, various Linux distributions are affected by this. Further information can be found at:http://www.h-online.com/open/news/item/Leap-second-Linux-can...
Please do not hesitate to contact us, should you have any queries.
VolScroll, tho . . . not sure about it. on my Gal S2, the vol buttons are really out of the way. I'll bet that the only time i'd use them with this app would be when i only could use one hand. even then, it's just as easy to hold with hand and thumb scroll...
And when's the only time you'd need 1-handed scrolling? Behind the wheel of the car. Neat idea, to be sure... just not sure if it's practical.
Fantastic job. You should be a Google dude.
Scale doesn't really affect recommendations either way if you're using Python/Django. You'll still want to use Celery, but scaling everywhere except DB is fairly prescriptive. Not knowing how the data is structured, be sure to look at Mongoengine. I've been using it with Django lately and loving the simplicity.
Ease of use for hosting = Dotcloud again. Super super simple. Promise. Integrates with github to version your app, no cost to get started, and deploys are as simple as 'dotcloud push . <appname>'.
Simple payments integration = Stripe [http://stripe.com]. My last app went from not having billing to doing recurring subscription billing with Stripe in about 20 minutes. It's hard to get easier than that. Similar terms to Paypal, supports major CCs.
For email processing, a lot of people recommend Sendgrid or Mailgun, but I've found that I actually really like Amazon SES for this. There's no free tier, but it's dirt cheap to use, and once you'd get into the paid tiers for the other services, I've found that SES still comes out cheaply.
For e-mail I recommend Sendgrid, you can get 200 e-mails a day free. Also easy to integrate in.
For payments I've used Braintree and I like it. I didn't set up the account but I was given a sandbox account to try it out and it should be possible to get one. Easy to use, especially if all you are doing are transactions (even though subscriptions aren't bad on Braintree either).
There's nothing wrong with details: all the little hacks that work for us that we can share and copy. They're fun and cool and can really help.
But make no mistake about it: details are not issues.
Examples of details:
- when I wake up - what I eat - how I exercise - my set-up - my preferences - my life hacks - how my team is organized - how we communicate - what I read - where we go - tips & tricks
- the value I produced for my paying customers
9:01 answer the Skype call, offer a quick "I'm here" before switching to mute so they won't hear the sounds of coffee brewing, much less the groaning.
9:12 carry around the laptop while starting laundry, then into the bathroom and out again. Nobody suspects a thing.
9:35 coffee's starting to work. A good thing too, have to say my piece. Short and sweet but not too gruff.
9:37 hangover's coming back for round 2. Get back in bed with Skype on low.
10:08 shower time. Really gotta buy drano soon. And toothpaste.
10:35 third cup o' joe. Start working.
# hours after post: 7 # comments in this thread only expressing how they thought the "double stealth" bit was funny: 10 # American Psycho references: 3 # top level comments providing absolutely no information or any point at all: 10 # of reddit-style "one-up" threads 5
: perhaps including this one?
: is there a name for this yet?
0550 Alarm (https://play.google.com/store/apps/details?id=com.mobitobi.a... -- I LOVE this app!!) goes off and I sneak out of the bedroom trying not to wake the 1 year old.
0551 Instant coffee in microwave & bio-needs-management. pull on shorts from previous day and clean t-shirt.
0600 prepare lunches & snacks for wife & kids
[0620-0630 optional time to catch up on email if I was adequately well-prepared in advance for food prep the night before]
0630 wake 1yo & wife. change diaper, dress, feed, and sunscreen 1yo.
0645 wake 3yo, usually with help from 1yo.
0705 wife leaves for work and i leave to drop off the kids at daycare/preschool.
0745-1630 lots of meetings, some real work, possibly some exercise, dinner prep, laundry, and possibly a couple of errands.
1630 pick up kids and proceed with the evening family craziness until putting them to bed around 2030.
2030-2230 fluctuates between doing the real work I couldn't get done during the work day and spending time with my wife. Usually more of the latter than the former, something I am extremely thankful for.
The point isn't the schedule. The point is that finding a work-life balance that accommodates productivity and supports happiness in both areas is a very personal thing, and as jgc's sarcasm shows it's ultimately about what one can accomplish and how one feels while working toward those accomplishments, not the self-congratulatory circle-jerk minutiae so many people focus on.
But seriously, I think "double stealth mode" deserves to officially enter startup lexicon.
To quote Louis CK's interview from Tuesday... it's WONDERFUL!
Likely because focusing on that is a lot easier than focusing on reality: It's hard work, takes extreme dedication and sacrifice. Much easier to pretend that certain types of yoga exercise are key factors....
I do notice a few small problems:
1) when you go to demo page and actually view the demo cam streaming the user information menu up top (Login/Register etc.) looks fine. But going from demo page to the home page once more somehow shows the menu as "logged in"--the menu says Logout, even though you shouldn't be logged in to see demo. In any case, it must be something to do with session or the simple loggedin/out logic.
2) In the demo page, dialog box on deleting cam or adding cam will actually be covered by flash. This is a pretty common issue and I believe the solution is to set the flash w-mode: opacity or something of that sort. It has been a while since I worked with drop down menu and flash :)
A suggestion: instead of control buttons of up/down/left/right-- you can definitely add immediate improvements to the experience in 2 levels:1) bound the keyboard shortcut which should be quite simple. Just trigger the click even on the appropriate button. Don't forget to have a small delay so you don't crash the darn device
2) enable dragging of the video: I'm "thinking"... when the video is loaded, cover the video with a low opacity div (1px gif of 0 opacity?) then bound a mouse listener to the box. Again, you can move the cam according to the mouse movement by checking mousemove and then on delay trigger the click even on your already bound control button. In fact, the easiest would probably have 9 div arranged in this manner:
if mouseenter 5 do nothing... if mouse enters 1 trigger left+up button click, 9-down+right, 8-down and so on.
I hope this makes sense. I'm simply looking at the demo and assume the real experience is probably the same and could be improved.
Other than that, things look great. Best of luck to the team!
How are you planning on competing with those?
Long story short, I quit and went freelance, best decision I ever made :)
My advice to you would be that (unfortunately) it will never be considered cost effective in management's eyes to rebuild the existing sites in a decent CMS / Framework. But you should at least be able to get them to use modern dev standards on any new builds, or complete rebuild of existing sites.
Make the security argument, a procedural system the size of CMS and site running on it ->could<- be riddled with security issues especially as developers are acknowledging that any addition is a bodged job due to global variable etc etc. If they are dealing with larger clients this should concern them enough.
Also if you're lucky enough to be deploying to PHP 5.3 using closures (http://php.net/manual/en/functions.anonymous.php) can save a lot of hassle when dealing with procedural code.
I found another developer with the same feelings about our (lack of) code quality as me and we put our desks opposite each other. This allowed us to exchange ideas and implement them in a positive environment. This worked well, and we quickly improved our deployment and testing practices.
But refactoring the code was harder. Developers that had been there 10 years knew that starting to pull at X would break Y, so they were afraid of change. Of course they had experience to avoid most problems, and this knowledge was what made them valuable to the company. At any other company they would have been sub-par developers.
Our improvement progress slowed down. We got tired of having to work really hard to get to the most basic industry standard, which were still far from stuff we could be really proud of.
In the end we both quit, and it was the right decision.
If you feel you have the support from other developers to make a change to your situation, then stay and do it with energy. But otherwise just move on, there is no point in wasting your time when there are so many better places.
The first and foremost thing you must understand is that each code base has a personality of its own. Code bases, like people, change over time but they rarely deviate from their core character. And the more money that code is making for someone, the more resistant it grows to change.
So, the thing is, don't put so much of your identity on it. It is not personal and it is not about you. In due time you will have chances to try your hand and build something from scratch, but until that day comes you need to work with other people's code in order to learn what approaches are feasible and where the pitfalls are.
In this particular case, only you can decide if the current code base is so much of a mess for you to bother. But you owe to yourself to give it another try, this time from the perspective that you want to make it not perfect, but as good as it can be. Once you have given an honest try to assess the potential of this system, and can explain in detail why that is negligible or non existent, you may look for the next job with a clean conscience.
A few tips:
1) Build demos and/or sneak in good code when you can. There will always be a subset of people that can appreciate good code once they see it working, increase your chances of finding those people and converting them to the cause.
2) Do not overwhelm them (them being veteran developers and management). You will scare them if you talk about starting from scatch or major overhauls. This is a battle of incremental changes, technically and culturally.
3) Don't burn yourself out. Its a daunting and draining task implementing the kinds of changes you're talking about in an organzation that doesn't want or appreciate it. If you can't find the right support, its ok to give up. If you do give up keep in mind that YOU didn't fail, you're just not in the right position or in the right organization to make it happen.
If you can align your interests (write better code) with their interests (make more money), you may have a chance to rewrite the code. :)
Prove them that refactoring the code can help you create more sites faster. With more sites in less time, they will make more money.
Learn? Of course. There's a huge difference in asking people here what to do in your situation, compared to living your situation with open eyes and mind. You'll be much better able to avoid creating situations like you're in now, after having lived it. Hint: it wasn't caused by poor coding practices.
It seems that in 3 weeks or something like that we will be moving in new offices.... And there is a promise for improvement... probably this is the best time to kill off the old home-made CMS and replace it with a framework! ...
PS: Thank you all for your responses... it's a good feeling that I am not the only one that was in this situation.
You have to realise that the company you are working for now might be suffering from industrial inertia - also, unless you are privvy to their financials, you probably have no idea of any pressure they are under to make ends meet. In general, selling something that you've already sunk costs into is more cost-effective than writing something new and then selling it, so if their primary objective at the moment is to make money and they have a system that is working, they will probably keep pushing it. It is rare, though not unheard of, for a company to be so developer focussed that they will undertake a wholesale refactor of their code base if it is currently bringing home the bacon. Also in something very old and convoluted (as you've said pull on one part and another part will collapse), refactoring is often less time-efficient than simply writing it from scratch. Ask yourself "if I manage to refactor this into a best-of-breed code, will I be enjoy putting the time and effort into it on a daily basis? or will it just be an albatross around my neck every morning until it is done?"
Consider your goals - you could consider staying at the company and learning about the business end (i.e. actually selling the software), maybe make some valuable contacts - that's a good goal but it might not be the right goal for you. If you currently just want to write beautiful code, then find somewhere else to work - perhaps a more greenfield development without legacy code.
edit to add a potentially relevant link on stackexchange that i've just come across:http://programmers.stackexchange.com/questions/155488/ive-in...
Privacy laws are pretty lax in the US but in Europe, Canada and much of Asia the story is different.
A service that can handle "privacy" would be very valuable and would probable get support from regulatory agencies around the world.
Most products are build for the web and then ported to mobile. I believe to be successful on mobile you have to think mobile first. Example: Web forms and surveys. There are services like Wufoo that build a great web product, but on mobile it sucks. We have a mobile app and close to all our users use this as their only computer. There is no service for great forms that we just can use.
This is the case for many other products.
These are my thought beside the typical mobile apps. Think of existing services, but think mobile first!
In the middle, people post comments on others' blogs or on meta sites such as HN.
I can't be sure, but from what you've said so far, I'm not seeing much unaddressed pain in the space.
AWS is an amazing tool and you have a few options here, but the downside is the more options you use to be highly-available (HA), the more expensive AWS gets (as you would imagine).
Your first option is to be HA across a SINGLE region; to do this you make use of the elastic load balancers (ELB) + auto-scaling. You setup auto-scale rules to launch more instances in different availability zones (AZ) either in response to demand or in response to failures (e.g. "always keep at least 3 instances running").
You compliment that with an ELB to load-balance incoming requests automatically across those instances in the different AZs. This is all fairly straight forward through the web console (except auto-scaling is still done via CLI for some reason)
If you want to be HA ACROSS regions you can't just use ELBs anymore, you have some added complexity and an additional AWS feature you will likely want to use: Route 53.
Route 53 is Amazon's DNS service which offers a lot of slick DNS-features like removing dead points from DNS rotation, latency-based routings, etc. There are also something like 29 deployments of Route53 (and CloudFront) around the globe so you'll hopefully never have Route53 become a point of failure for you even disaster strikes.
In this scenario you would setup the HA configuration for a single-region as mentioned above, but you would do it in multiple regions. Put another way, 2+ servers in multiple AZs in each AWS region. Then a Route53 DNS configuration setup to point to each ELB in each region representing those individual pockets of servers.
Ontop of that you would use Route53 to manage all routing of client requests into your entire domain; you can leverage the new "latency-based routing" (effectively why everyone was asking for GeoDNS for years, but even better) and monitor capability to ensure you aren't routing anyone to a dead region.
Here is what I would recommend given the size of your budget and need to stay up in the AWS cloud, in-order of expense:
1. Launch a single instance in a region with acceptable latency that has never had an outage before (e.g. Oregon has never completely gone down but Virginia has -- yes yes I know VA is older, but you understand my point). This solution will be cheaper than multiple instances in any region.
2. Launch multiple instances using the web console, in multiple AZs in US-EAST (cheapest option for multi-instances) and front them with an ELB. You skip any auto-scaling complexity here but you need to keep an eye on your servers. I think ELB fixed the issue where it would effectively route traffic into the void if all the instances in an AZ went down.
OPTIONAL: If you didn't mind spending a few $ more, you could do this strategy in the region that has never gone down for added piece of mind.
3. Launch single instances in multiple REGIONS and front them with Route53. This isn't really a recommended setup as entire regions will disappear if you lose a single instance, BUT I said I would list possibilities in order of price, so there you go. You could mitigate this by setting up auto-scaling policies to replace any dead instances quickly in the off chance you wanted to do exactly this but not babysit the web console all day.
4. Launch multiple instances in each region, across multiple AZs fronted by ELBs and then the entire collection fronted by Route53.
NOTE: The real cost comes from the additional instances and not from Route53 or the ELB; so if you can use smaller instances to help keep costs down (or reserved instances also) that might allow you to provide a larger HA setup.
What about my data?
Yes, yes... this is an issue that someone already touched on (data locality below).
You will have to decide on a single region to hold your data; in this case I would recommend using DB services that aren't based on EC2 and have never experienced outages (or rarely) -- this includes S3, SimpleDB and/or DynamoDB. AWS's MySQL offering (RDS) are just custom EC2 instances with MySQL running on them, so any time EC2 goes down, RDS goes down.
The other DB offerings are all custom and except for SimpleDB a long time ago, have never experienced outages that I am aware of.
Making this choice is all about latency and which DB store you are comfortable with (obviously don't choose SimpleDB if everything you do requires MySQL -- then use RDS); you'll want your data as close to your web tier as possible, so if you are spread across all regions you'll just want to pick a region with the smallest latency to MOST of your customers (typically West coast if you have a lot of Asia/Aus customers and East coast if you have a lot of European customers).
Want to Go to 11?
If you have the money and desperately want to go to 11 with this regional-scale (which I love to do, so I am sharing this) you can combine services like DynamoDB and SQS to effectively create a globally distributed NoSQL datastore with behavior along the lines of:
1. Write operation comes into a region, immediately write it to the local DynamoDB instance, asynchronously queue the write command in SQS and return to the caller.
2. In 1+ additional EC2 instances running daemons, pull messages from SQS in chunk sizes that make sense and re-play them out to the other regions DynamoDB stores; erase the messages when processed or if the processing fails the next dameon to spin up will replay it.
3. On reads, just hit the local DynamoDB in any region and reply; we trust our reconciliation threads to do the work to keep us all in sync eventually.
NOTE: If you prefer to do read-repairs here you can, but it will increase complexity and inter-region communication which all costs money.
The challenges with this approach is that you pull up a lot of DB concerns into your code like conflict resolution, resync'ing entire regions after failure, bringing new regions online and ensuring they are synchronized, diffs, etc.
There is a reason AWS doesn't offer a globally-distributed data store: it is a really hard problem to get right once you make it past the 80% use case.
Your data will determine if this is an option or not; some data allows for certain amounts of inconsistency in which case this strategy is awesome and works great; while other data (e.g. banking data) cannot allow a single wiggle of inconsistency in which case pulling all this DB logic up into the application is a bad idea. Your failure scenarios become catastrophic (e.g. your conflict-resolution logic is wrong and wipes out the balance from an account; or keeps re-filling the balance on an empty account... something bad basically)
It is all a trade-off though; if you managed your own Cassandra cluster though, Cassandra does all this and much more for you automatically; but then you just put your time into Cassandra administration instead of developing the logic around DynamoDB (or SimpleDB, or MySQL, or whatever); just pick which devil you feel more comfortable with.
I am not aware of a services company yet that offers cross-region AWS datastore deployments yet; Datastax and Iris Couch will setup things for that like you via a consulting/custom arrangement, but there isn't a dashboard for launching something like that automatically.
Hope that helped (and didn't bring you to tears of boredom)
I'm in one of the us-east zones and I haven't had a failure in at least a year. They retired one machine I was using and dealing with that was as simple as starting and stopping -- at a time I chose.
With five zones in U.S. East, the probability of a zone failure affecting a single zone systems is 1 in 5.
If you're a busybody who spreads your system across five zones, the probability of a failure affecting you becomes 1.
You're spending more money, and dealing with a lot more complexity, all to increase the probability that hardware failures will affect you.
Now, you're hoping that a zone-distributed system will be able to recover from failures, but that's tricky to do and it's quite unlikely that this will work if you haven't tested it. Add the fact that all the other "cool kids" will be trying to recover their systems at this time and make AMZN's control plane go down.
In the meantime, with probability 4/5 I'm sleeping through the disaster and the first time I hear about it is on hacker news.
Sorry to be fatalist, but it's a hard problem. This last outage was more than just an AZ failure. Region-wide API usage was affected, so operations like static IP reassignment and ELB changes were not taking effect. This means you are hanging out in the wind should there be something unusual that requires manual intervention (as was the case with us).
Route 53 is a good service but I don't know how its control plane works, and it could be that problems in a single region would disable the ability to update DNS records (I would guess that DNS reads are a lot more available than writes). And in any case DNS is not a very good failover mechanism due to upstream caching.
Unless your business model requires higher reliability than Instagram, Netflix, and Pinterest, I'd suggest going multi-AZ, crossing your fingers, and doing everything else right.
Anyhow, sorry I don't have a simple answer - I'm not sure a simple answer exists.
1. Use Rightscale. You can get away with the free edition, but for $500/month the basic paid edition will allow you access to arrays and all the excellent scripts available on the marketplace.
2. The front end. I would strongly suggest moving away from ELB. We are using it and are about to get rid of it. The main problem is what exactly happened last night. If a whole AZ goes down, the ELB for that zone can get screwed and the DNS was not updating the CNAME to remove the bad zone. Instead of ELB, we have our own LB solution we are going to roll out that will use Rightscale server arrays and will handle the updating of the DNS names itself. We also aren't going to use Route53, because we learned last night that the API for that can go down and you can get stuck with bad DNS records.
3. Application servers. Use at least 3 AZ and have them evenly spaced. This is easy to do in Rightscale with sever arrays. Make sure your voting ration for scaling isn't 50% because you might not scale correctly if you loose 2 AZ. Keep the vote to 30% and you will be happy (if one zone votes to grow, let it grow).
4. Database. This is the fun one. We have been using MongoDB with pretty good success. Our multi-shard DB has 3 servers per replica set and has them distributed equally between AZs. We use 4 EBS drive RAID-0 drives for storage which have had problems in the past due to the outages that EBS sometimes has. Our best bet has been a watcher process that will kill the mongod process if there's any problems writing to the drive array. By doing this, the replica set will automatically failover to the next server and we won't get stuck with a primary node that can't write back to disk. For backups, we just freeze the writes on the secondaries and do EBS snapshots even 15 minutes. Rightscale has some great EBS tools for managing this for you. If you loose a server, we can deploy a new server in a matter of minutes and it will rebuild the RAID array from the last backup so we have a warm spare.
5. Monitor, monitor, monitor. Rightscale has some great tools for monitoring everything. Use them, and use more monitoring on other infrastructure (ie Pingdom)
Doing something like this will cost a lot more that just sticking to a single AZ, but you should be able to survive one, if not two complete datacenter outages.
PHPFog are on AWS and I pay them to make sure they have the redundancy worked out. If they don't, I would yell at them until I got some money back.
I am considering configuring RDS for Multi A-Z, but need to research it a little more first. From what I can tell you just click a button to turn it on, but there were a lot of people complaining yesterday that the fail-over didn't work at all when it was supposed to.
I also have a bunch of EC2 VMs that do back-end processing and have a load of CRON jobs on them that need to run once every 24 hours. If these go down for a couple of hours then there is no noticeable impact to my customers, they can still log into my service and access their historical data.
I have considered spreading across multiple regions etc but at the end of the day it's just too expensive for the small increase in reliability.
I don't think they show how to do ELB across Regions, or diversity against single-ELB problems (although I haven't seen ELB fail yet). You'd probably have to build this yourself.
1) Has AutoScaling Groups & Elastic Load Balancers in two regions (and only two availability zones; let's keep front-end instances in the same AZ as your local/region-specific DB)
2) Has Databases in two regions and uses Master Master replication
3) Instances talk to their local DB. If they detect their local DB is down, they failover to the remote DB (ie, the far region). If they failover, they notify you.
4) DNS does geographic load balancing (pre-ELB). You'll need to use a provider like DynDNS or UltraDNS to give you Geo Load Balancing & Failover. Or, you could pair a monitoring service like CatchPoint with Route53
5) Application caching (Memcache, Redis, etc). Let's not put more load on the DB's than necessary.
That's a good start, at least.
- For the sim cards. Any monthly prepaid plan like the $50 T-mobile 4G with unlimited data, text &talk is fine.
- For internet try check verizon, att, t-mobile or timewarner. I currently have time warner. It's a bit expensive. FIOS with 20mb/s for $45 a month.
- For the event checkout http://www.eventbrite.com & http://www.meetup.com and search for the events.
- Fro the transportation, get a cliper card. You can take the bart and the bus with it. Unfortunately a car is a must have for the bay area if you are planning to driving around palo alto, ...
And, it's a good idea, we should setup an hacker news meetup !
Juggling: Theory and Practice.
I did freelance work online earlier this year of the sort you describe. It was inadequate to my needs and felt like a trap. I am currently getting public assistance and working on developing my websites as my only hope of a long term, real solution.
Feel free to email me.
An observation: given your description of recent history, it might be best for you if you ran your own business. In other words, create your own job. Maybe that's freelancing, maybe running a service, or maybe something not even internet or tech related.
Maybe iOS apps will get you that, if you're lucky or really good, but I think most people don't make much on apps. If you do, great, not trying to discourage you.