We also use tokudb storage engine for a large number of servers. This engine is designed for datasets that do not fit into memory, and we've had a great experience with it. We are able to load CSV's in 1/3 to 1/5th the time of innodb. Most queries aggregate queries run faster, there is online index creation and column changes. From a DBA prospective tokudb prints detailed statuses in the show processlist! It is definitely a niche storage engine, and not right for all cases but this has contributed to our continued use of MariaDB of PostgreSQL.
With PostgreSQL 10 things are looking interesting. We tested out 9.6 with it's parallel query and saw some significant improvements on aggregation queries. It was not enough by itself to warrant use moving off of MariaDB but has me very excited going forward.
MariaDB is not sitting still, there are many improvements coming to 10.2 which should be released soon. For most applications I'd use whatever database you are familiar with. If you have a specific case, terabytes of data, millions of incoming writes, requirements for live aggregation, that is where the PostgreSQL vs MariaDB really gets exciting.
MyISAM isn't used anymore. MySQL has ACID transactions and foreign keys. Get over it. The engine is still there for users who want to use it and that's not a reason to knock MySQL.
Almost every one of the correctness and SQL standard flaws in MySQL is handled by the sqlmode flag. It allows users to choose correct behavior, but doesn't suddenly break the millions of MySQL apps when they upgrade. To suddenly break millions of apps to be correct by default without sufficient time for apps to migrate makes no sense.
The fact is, MySQL is significantly faster for simple CRUD applications which is really common in today's REST and Microservice designs.
That being said, Postgres is an amazing RDBMS, and would probably become the #1 database server if administration tasks like SHOW DATABASES weren't accomplished through \l (or if \l where a shortcut).
Postgres users wonder why people use MySQL. They wonder why people use NoSQL when storing JSON in Postgres can be faster. The fact is, the learning curve of Postgres is needlessly higher than these other systems.
* Thread-per-connection (as opposed to process-per-connection with PG) means much lower memory overhead with many concurrent connections; with PG you might need to use something like pgbouncer, which doesn't come without its own issues.
* Logical replication; the MySQL format is compatible between minor release versions (e.g. between all 5.x releases), which enables upgrading individual replicas or master without complete shutdown. With PG this often a becomes way more complex.
* Better caching/buffer management. Essentially, InnoDB has its own implementation of LRU cache while PG relies on kernel page cache. This generally allows finer tuning on the database configuration level and you don't have dive into kernel page cache tuning (which will be always OS-specific).
EDIT: most of this stuff is probably even better described in the Uber article, linked in other comments here.
but switched back to mysql in 2016 which generated a lot of discussion:
There were a lot of followup posts afterwards arguing both sides if you search for them:
* MySQL is a really awful database. It's usable only for small hobby projects, like a simple Wordpress site or so. For serious datasets, it's a horror. Complex queries on large amounts of data are just broken. Many features don't work or work incorrectly. The design is fundamentally flawed.
* Use PostgreSQL instead.
* If you absolutely have to use MySQL, use MariaDB instead. It's a fork that fixed some of the issues.
* Still better use PostgreSQL. Or even Oracle or SQL Server. They're fine, MySQL is not.
- MySql is the choice for niche workloads and to be used by experts not only in DB but in MySql Quirks.
- PG is the sane choice for everyone. Experts benefit more, but inexperienced developers will be better served from a more solid foundation, and still PG is so good that will be along your side for years to come.
PG maybe will not shine in some niche workloads but we are talking about problems that most DO NOT HAVE.
Years before, I have read (and believed!) that "PHP, MySql, JS" was "better" for inexperienced (or new) developers, because are more "easy" to learn and use.
Then eventually is show how facebook and others used them.
However, after +18 years on the field, I become more and more convinced that is totally the reverse:
- PHP, JS, MySql, etc are tools for experts.
- More strict tools are better for everyone.
- Only expert will use correctly any tool, even if have quirks.
- This is even more notorious if the tool is more dynamic than static, or provide more "freedom"/less "safety", like C VS. Pascal.
- More well designed or strict tools provide equal benefits to experts, but sometimes a tool with not safety-protection allow to do hacky things easier (or possible).
Postgres shines if you need advanced functionality: recursive queries, lateral joins, window functions, foreign data wrappers, partial indexes, advanced types... The list of things MySQL doesn't do as well or at all, be it user facing or under the hood, is fairly long.
There are plenty of examples of both DBs scaling successfully in the wild.
IMO it's a coin toss decision for most CRUD apps. For more advanced apps, you might want to pick Postgres over MySQL for the same reason some companies prefer niche languages over Java or PHP: the pool of competent engineers is smaller but tends to be of higher quality.
The best advice I can offer is, "Use what you're more comfortable with." Both can be configured on a single machine to serve a ton of traffic. Both can be backed up to recover when they falling over. Both can be configured to keep your data consistent and safe. Don't worry about scaling, don't worry about sharding or how much better indexing method A is than B. Worry about your application.
Now that the obvious is out of the way, I personally prefer MySQL over PostgreSQL. It will be easier to scale when the need arises (the corollary to "avoid premature optimization" is "design to make optimizations easier"), it performs like a madman when configured properly, and there are simply more people out there who can help you configure it properly.
Tangent time: yes, it is comparatively feature poor. I can't store native JSON documents in it, I can't run Python stored procedures, I can't create nearly the same breadth of indexes. Then again, I have never needed to. JSON maps nicely to objects, and an ORM maps those objects (somewhat less nicely) to native tables and columns. I don't run stored procedures. I use purpose built tools for handling (for example) full text indexing when the built-in searches get too slow.
My biggest question, when everyone points out how many features PosgreSQL offers, is "why are you doing that in the database?" Your database is your biggest chokepoint. It's your biggest point of failure. If you're using it for anything more than "transactionally store and retrieve data", you're asking for even more trouble when it goes down.
Not if, when.
It also makes scaling harder; if you're doing most of your computations in the DB, you have to scale the DB when one machine can no longer handle it. Scaling DBs is inherently harder than scaling just about any other component in your stack; you can't just add another server to the pool.
In summary, if you use your database as a database, and only as a database, either option will stand you in good stead. The biggest advantage MySQL offers at that point is that there are simply more experts out there who can help you with it when you need that help (and will probably cost you less professionally).
If you spend actual time in the database using SQL, Postgres offers vastly more functionality, e.g. window functions, broader and more mature GIS support, more data modelling options. Postgres also certainly _can_ outperform MySQL in a variety of situations - it supports different indexing options (i.e. last I checked MySQL didn't have partial indexes), and is moving to more parallelism in its query plans. MySQL's partitioning support is a little more natural than Postgres's but that's changing in 10.0.
So, I would say, if you just want a back end for an app and have never _really_ cared about your RDMBS, use MySQL or whatever you're most familiar with. If you have complex query requirements, time series data, analytics workloads etc, try Postgres.
For my own startup which the primary use-case is analytics, I have moved back from Postgres to MemSQL.
The four reasons were :-
1) In-memory rowstore, thus eliminating my in-memory nosql dependency.
2) The ability to join in-memory tables with disk to provide up-to the second data is awesome.
3) Multiple read/writer group nodes.
4) The biggest for me. Columnar storage.
Some examples on how columnar storage is so awesome.
Note that I am comparing MariaDB vs MemSQL on a old desktop. 8 cores, 8 GB Ram, HD (non ssd) drive.
- A query with a count of 1 table with 50m rows.
MariaDB - 49.60 sec
MemSQL - 1.12 sec
- A query with a count, sum, where clause and group by, with 50m rows.
MariaDB - 17 min 44.84 sec
MemSQL - 8 seconds
Here are some things to be aware. Prior to migrating over to Citus. We were shopping for MySQL consultants to see whether they could improve on the above figures. They quoted $70-100k to work on the project, without guaranteeing results.
With Citus I was able to see far better results and this is why the company switched over. However fast forward to now, MemSQL beats Citus soundly. 1 desktop beats a cluster.
I wager, that with a MemSQL cluster with decent hardware on the cloud, I will be seeing sub 1 second queries across the board.
Oh and the amount of data? It's tens of billions of rows a month. That's the workload I'm talking about here.
This post isn't to bash postgres. I loved using it. If Postgres 11/12, included columnar storage. I would literally scream! :)
One also needs to consider the entire landscape. For example, the LAMP stack is mature community with out-of-the-box, ready-to-ship web sites. There exists lots of support and tools. Software stacks like Django for Python typically have community involvement for a particular database. If one is using an ORM like SQLAlchemy, what is SQLAlchemy best used with?
Here are just some of the things that should be qualified when asking database choice:.1. Transaction support.2. Scale out support. 3. NoSQL and other data store integration.4. Cloud support if deploying in a cloud. Really depends on your cloud vendor.5. Location, location, location. Is this an internal application on your company LAN, or a social networking application to be installed around the world? Replication in MySQL quite useful.6. SQLite? Why did you exclude SQLite? I'm assuming because the person posting this question already knows something about the workload that defeats SQLite? What is that workload? Testing against SQLite is trivial, just create an in memory database. 7. ETL support. Are bulk updates due to ETL processing of files and such common place? Bulk updates happen better when indexes are not attached, etc. 8. Reporting support. 9. Denomalized versus normalized support. It is common place in web applications with data distributed applications around the world to shard a database table, completely denormalized.
The point is that designing a database schema is heavily dependent on the expected workload. The choice of SQLite, PostesSQL , MySQL, SQL server, Oracle or what have you depends heavily on the workload. It is nonsensical to ask what the right tool for the job is without knowing the job.
Basically it gives me everything I need in a relational database and let's me worry about other things.
depending upon workload this can be significant.
-> Pg - Pg - Pg - Mo - Pg - Pg - Pg - Pg - My - My - My - My
Round14 - Preliminary data!
-> Pg - Pg - Pg - Pg - Pg - My - My - Pg - Pg - Pg - Ny ....
If there was one, I will switch in a heartbeat as I will love to have access to some more advanced SQL than what's available in MySQL (like window functions)
My team spent many months trying to mangle MySQL into doing a similar thing (which it will do) but eventually we hit a performance stop with MySQL.
We did many things to improve the performance; switched to using fastsockets in the kernel, changed memory allocators (tried two different ones) but at 40cores and 8 pci-e ssds it was spin locking some function in memory.
Not to mention it's history with throwing data away.
Most people use MySQL because they know it already- and I would say that's fair comment. But getting started with PostgreSQL is easier and it scales better, so it's hard for me to agree with using it in 2017.
To get started with a random test db you can do:
pg_ctl -D database initdb
pg_ctl -D database start
psql -h database
As a result of this design decision the resource management of each major part, like the buffer pool, the lock manager, etc is explicit and straightforward.
This, in turn, leads to indeed stable and predicable runtime behavior during high loads.
Basically, PostgreSQL is one of very few canonical examples of what sanity and attention to details could accomplish.
MySQL adhered to a philosophy of "make it fast, then if someone complains enough, consider making it correct".
The result of this is predictable: Postgres is now fast and correct in its implementations, while MySQL is fast and burdened with a large number of "gotchas" and legacy incorrectness.
MySQL's traditional "fast" table type doesn't support transactions. So you have to select a different table type if you want transactions, but then when you do, you discover that you need to usually go "up" one level from what you'd expect, because the REPEATABLE READ level only affects SELECT, not INSERT, UPDATE or DELETE queries. Which in turn means you can have an inconsistency between the set of rows you'd see in a SELECT with a set of conditions, and the set of rows that would actually be affected if you issued an UPDATE with that same set of conditions. You have to go up one level to READ COMMITTED in order to start solving this.
MySQL's Unicode collations have some bugs; when I was at Mozilla, one we ran into was that we had to set up our own custom collation to get, for example, "e" and "" to be considered distinct characters (if you ever used MDN, and once saw a strange bug where French versions of topic tags showed on an English article? That's why -- MySQL thought "CSS Reference" and "CSS Rfrence" were the same tag).
MySQL infamously accepted almost any kind of garbage values regardless of declared column type, accepted values larger than the declared column width (i.e., you could insert a 300-character string into a 256-character VARCHAR column), and would silently or near-silently mangle, truncate or just drop the data.
I cannot at this point imagine using MySQL for any kind of serious project.
It seems like Everytime on HNs people say postgre is so much better. Why is MySQL so much more popular?
it would be easier if you listed what you mean by "cons". Postgres is much much more powerful than MySQL if you are really into SQL and exploiting an RDBMS, MySQL administration is simpler.
This question is a bit too vague to warrant a specific answer.
MySQL is MySQL or MariaDB.
Not to be trite, but I think the biggest high-level Pro/Con comes down to this: Postgres has the more active OSS community, MySQL has Percona. Seriously, if you're in a situation where you have the option of spending money to solve problems, then having the option of hiring Percona to fix your problems alone makes MySQL a sound decision. If you don't have a large cash reserve, but are willing to spend time combing through message boards and blog posts, pick Postgres.
The biggest problem is linkrot. As a rough estimate 13% of links die every year, and it's quite possibly much higher than that. (https://www.gwern.net/Archiving%20URLs) Without the glorious web archive, bookmarks would be unusuable. And I wonder how many people know about web archive.. Youtube-dl may also be useful if you want to preserve music or videos (despite the name, it works on almost every site I've tried it on including audio sites.) Someday I intend to script something up to automatically scrape all my bookmarks and make a local copy, but it seems complex.
For instance, maybe I'm looking for a PostgresSQL tutorial. I start typing "postgres" and one of the bookmarks I forgot about from several months back appears. This approach has ended up saving me a lot of time over the years. Another cool thing is when a bookmark pops up when I'm searching that brings back memories. If the site is still up, I get a free trip down memory lane :)
My collection is at least 9 years old now. I've been maintaining the same Firefox database over the years by migrating it manually from version to version. Now it's seamless thanks to Firefox Sync. I get my bookmarks on my PC, laptop, and my phone. I have an Xmarks account as a backup, and for cases when I prefer to use Chrome.
With a paid feature called an archival account, Pinboard stores an actual copy of each bookmarked article, kind of like your own private Wayback Machine. It provides full text search over these articles.
I frequently save articles that I read so that I can refer to them later. It doesn't happen often, but once in a while I will desire to access an article that I read a few months or years later, and I find Pinboard well worth the value for making it possible for me to actually identify the article and retrieve its content regardless of whether the original link is still around.
I find this especially useful because it is my habit to collect citations for various facts. When I find myself making a claim in conversation, I really want to be able to access the original source where I learned about the fact, and provide the evidence to back it up. Or to review the source to confirm that my memory of it is accurate. Or sometimes I want to share a useful article explaining some topic with a colleague or friend.
I do occasionally use the browser bookmarks a sort of clipboard or working set, for 5-10 links at a time. I use Google Chrome and it syncs bookmarks between my devices.
Instead I (ab)use bookmarks as a way to increase the weight of URLs in chrome's navigation bar autocomplete/suggestion algorithm.
e.g. If you find that you're going to a site's homepage and clicking three times, instead once you get to the actual page you want, bookmark it. You can even give it a more memorable name, like "standup hangout" and then watch it autocomplete from the address bar next time you start to type the URL.
As others have said, most things can easily be searched as needed. But I was using bookmarks as placeholders, saying "I wanted to read x later", in most cases... sometimes other things.
So I started treating bookmarks as various categories of todos. I do have a reference folder, but it has less than a hundred items. I often use those only passively -- i.e. when typing into the address bar, the starred link will come up first.
All the other links are sorted into categories such as "files to download", "new articles", "new buyables" and so forth.
Now that I think of Bookmarks as deferred work, it has changed a lot of habits. My total number of bookmarks has slowly dropped, and I tend to handle more stuff as it comes, or not at all -- or at least to be more conscious of bookmarks as a cost.
An unexpected benefit has been a feeling of mental satisfaction, after closing a lot of semi-forgotten, open loops. I now think a big unorganized pile of bookmarks can represent a real liability, whereas if you actually go through all those links and delete the weaker ones you get a concentrated pile of goodness. You hit a point where you'd rather read your remaining bookmarks than most news feeds.
*Forgot to mention the best part: Backend pools these full-text documents, cleans and parses for NLP, then generates meaningful tags, and organizes documents in an auto generated folder hierarchy which is based on word2vec/doc2vec and content clusters. Whole thing runs on a dedicated server with two 1070 GTX video cards for the NLP work which is training and re-evaluating constantly as new content pours in.
Altogether it was 2-3 years of work.
sometimes I make use of the features "open all bookmarks in this folder".
other times I use the bookmark to (as somebody else mentioned already) weight consideration of sites I'm interested in getting results from.
aside: at hackreactor, I worked with some folks on the beginnings of a chrome extension to grab your bookmarks, analyze the content of each site, and suggest new bookmarks when you open a new tab. the suggestions part was working already by the time I came around. then I got a job and that pretty much fell out of priority... heh.
But these days it worries me to say that I just visit the same three websites over and over. Aggregation websites with links and content.
Sometimes I find myself staring at the url bar not being able to think of anything to do because I've visited my three websites already.
Of course besides those three aggregators there are sites like google and stackexchange that I visit indirectly. And any blogs, forum and such that I might find through google.
I help my parents and my kids work with bookmarks but I have none myself; and I was beginning to think that bookmarks were primarily used by non-technical people. I guess I was wrong!
Everything I need is a simple URL (like, my bank: usaa.com - why would I bookmark that?) or a quick Google search away. If I come across a deep link that's so important that I want to keep it, I email myself the link along with maybe a short description, and it will be searchable forever.
My lack of bookmarks fits with the rest of my "online personality". I have 14,183 threads in my work email inbox and I do not file emails into folders like most of my colleagues. I do not have the desire or the time to manage email folders or browsing bookmarks.
Also, the fact that I browse in a "clean" browser instance in SELinux that saves no history from instance to instance probably contributes to my lack of bookmark use.
Well I've marked a ton of urls and rarely revisit :(It's like having a camera, take photos and forget them forever. It's a tool to help you forget things, not to remember, sadly!
Very handy, and it also protects somewhat against linkrot.
I've tied it to a hotkey to copy any bit that is highlighted to the currently open scrapbook. (shift-ctrl-b) without further notifications or interaction other than the keystroke. Super quick and it doesn't get in the way of continued reading.
To take things a step further, I'm not entirely sure how I'd function without them.
(I'm sure I'd find a way, but it would be an adjustment and a loss)
Firefox's fuzzy searching in the URL bar makes bookmarks awesome. My "workflow":
1. Bookmark anything I might need later by clicking the bookmark button. It presents a little tooltip-like popup that lets me edit the title and tags if I want to.
2. Sometimes I edit the title/tags and sometimes I don't. I make this call based on a quick judgement call on whether the default will allow me to find the article later. Suppose the article title is "MySQL Adds Froitz-Based Blammo Filtering." Well, that should suffice. But if the title is merely "10 Awesome New MySQL Features" then I might want to edit the title/tags to mention something about "Froitz-Based Blammo Filtering" if that's what I'm interested in. 
3. Then I usually never use the bookmark ever again.
4. BUT, sometimes I do. And Firefox's fuzzy match implementation lets me type "mysql froitz" and get a match on this bookmark 100% of the time. Chrome's matching is stupider & I'm not sure about Safari. Safari makes adding bookmarks less convenient than FF or Chrome so I assume finding them is harder. (Maybe it's not, I don't know)
I don't know about Firefox's bookmarking performance characteristics. But, I know that I've been adding lots of bookmarks forever and it "just works" and it feels instance. The fact that I've never had to think about it beyond that point is a compliment of the highest order. That's one of the many reasons why I remain a dedicated Firefox supporter.
 This is just a theoretical example, of course. MySQL does not actually receive new features, awesome or otherwise.
Any time I want to update it, add a link, etc. I just use a text editor.
My data belong either offline or on serverspace I control myself. There's nothing especially secret about it, but like my email (going back more than twenty years), I wouldn't dream of storing data like that online outside my own control.
The bookmarking, by the way, used to take place in Firefox. The ongoing self-immolation of that once mighty browser has recently sent me to the Pale Moon camp. And it's like coming home. I couldn't be happier, running on various Linux'es on the household machinery. The Chrome/Chromium world hegemony is one of those sad, scary things I shall never understand.
As far as native bookmarks, I don't like that browsers have kind of black boxed their bookmarks and require individual proprietary cloud sync for these things (I realize Firefox has a self hosted option, but it's kind of outdated and last I checked the documentation was spotty. Even then it's only FF).
I know there's also the Netscape Bookmark Format which is kind of sketch, but at least it's something. I tried writing something that exported on close, I'd sync them myself, then imported on open, but it was pretty hacky (edit: also browsers exports are often very different so there was some normalization there that was fragile). There should be a way to setup an endpoint to natively sync this stuff with an open protocol and then all your bookmarks on all clients look the same. If you don't like that service, export someplace else and change your endpoint. Browsers should just be boxes for structured content.
I've got a fair degree of organization with folders and sub-folders but still spend way too much time trying to locate a specific bookmark. I've learned to edit the subject line because often you're bookmarking something called 'home' or a cryptic Github path.
1) I have a set of bookmarks specialized for search. Chrome can do this without bookmarks, but Firefox needs them. I'm talking about bookmarks like this:https://www.google.com/search?q=%s&tbm=isch
Note the %s in the middle, that's where queries go.When you save this as a bookmark and add a keyword to it ("gg" in my case), you can then search images on Google like this:
- Alt-d (jump to url bar)
- gg fluffy kittens
I have a few dozen of these: Google, G images, G translate, G maps, local maps, Wikipedia English, Wikipedia Czech, various dictionaries, whois, wolfram alpha, grammar check, YouTube, Maven search... You get the idea.
2) A huge curated collection of bookmarks to Java libraries. Something similar to all those awesome-java collections that are lately popping up, but more complete, in my browser, indexed for search and neatly grouped into like a hundred folders.
Firefox's bookmark manager is very primitive, though, and I've long been meaning to migrate my bookmarks over to org-mode in emacs, where I have much more powerful searching, metadata, editing, linking, commenting, restructuring, and navigating options.
Just like aliasing commands in the terminal, you can alias web pages in Chrome's address bar. So, when I type "je" in my omnibar it has an autocomplete option "Jenkins", and pressing enter will take me to the URL I set for the Jenkins home page.
This feature is poorly named "search engines", and yes, it is extensible to putting extra strings at the end of that URL (which could be registered as a search term within that site), but I've been using it for years, and 99% of my use is simply mapping arbitrary strings to arbitrary URLs. It works amazingly for that. No mouse movement digging into bookmark folders required.
Edit: Although, I realized that just full-text searching through bookmarks won't pop the most relevant links to the top.
I also still 'star' interesting links and categorize them into folders. Very handy to have Chrome sync the bookmarks across all of my machines.
A problem comes up when searching for bookmarks that don't have keywords in their titles. I use a WebExtension  to update my bookmarks with website descriptions, increasing the odds of finding them.
My girlfriend does make extensive use of it for all sorts of things.
I think my mom uses it as well. My brother and dad I'm not sure about. Not sure what that says for a confidence interval, but many people still do. Then again, I'm sure there must be clusters of people (when clustering by who knows who) that never learned it's there, or who choose not to use it.
For work, absolutely.I have a couple of top level directories on my bookmark bar:
CurrWork-<2-3 words describing topic of work>-<Date started>
Under KeyLinks I keep well...key links, like the links to the wiki entry on how to setup dev environments, link to the Holiday calendar, link to the Jira dashboard showing my team's sprints, link to the company roadmap, etc. Pretty much just links that I'll have to refer to periodically.
Under InterestingTech are articles or things of interest I stumble upon on my day to day, but that I don't have time right now to look into.... This one is honestly a bit of a bottomless pit at this point...
Under CurrWork-* I keep all the links related to the work I'm doing right now. That means wiki entries related to it, StackOverflow links I had to use to fix something, Jira tickets, Jenkin jobs, internal web-app links, code review link, etc. You name it. If it's in any way related to my current work, and it's a site, it's there.
And when I'm done with the current 'CurrWork-*', I remove the leading 'CurrWork-' and move it to the bottom of PrevWork.
I have an awful memory, but this in combination with an emacs org-mode file for each 'CurrWork' iteration I have, I manage to be able to refer to things I've worked on in the past when people ask. After they give me a minute or two to get my bearings of course.
I have hundreds of bookmarks, covering dozens of categories of research and reading. One of the largest subcategories includes hundreds of references that I may or may not need for future projects including software (stackoverflow questions, tutorials, bug solutions, framework and API references, optimization articles, in-depth guide articles, and so on), woodworking, economic/governmental/civic/legal research, fitness, electrical engineering and general circuity/wiring, real estate, recipes, piano repair, audio production, and so on. These are all intended to be kept until needed, most likely indefinitely.
Then there's a category for more temporary things that I need in the moment and am unlikely to need again, including news/blog articles I haven't gotten around to reading yet, solutions for bugs that I need to fix, torrents I haven't gotten around to downloading, and collections of references for small, specific projects that I won't need again afterwards.
So basically I use Chrome bookmarks as my personal address book for "things on the vast internet which I wish to return to eventually."
For major things I use daily, like youtube or gmail or facebook, I don't bother bookmarking them-- for those I just use the address bar's semi-intelligent autosuggest....Ctrl+L to go to address bar, then I just type g and hit enter, or y and hit enter, or f, etc. The only website I need to type out beyond 2 letters is twitter/twitch.
I guess this may sound odd, but Chrome has begun to feel like a natural part of my mind. The bookmarks and my gmail are an extension of my memory. My interactions with the net are an extension of my thought processes. I have seen other people make similar remarks about their phones.
Using it I hit Ctrl+M (the shortcut to open it) and then I have my top 20 sites key bound. So HN is Ctrl+M -> h. All my other bookmarks can be accessed via a search feature which you can tab or "/" to get to on opening the extension. I hate lists/folders so my bookmarks are all hidden away behind the search function. The extension is built for either the mouse or keyboard so I have a lot of flexibility in how I interact with it.
The site for it is: www.devmark.io
All of my bookmarks are resources, something for me to read or use at a later time. Some are for things I want to learn, some are for things I knew but have lost to time, and others are just.. out there. Like this: https://apps.fcc.gov/oetcf/eas/reports/GenericSearch.cfm I have no idea why I bookmarked that (or when).
Another example: This week I decided to learn Rust. I was listening to a podcast and the host mentioned rustbyexample.com and I visited the site and realized previous me had bookmarked it thinking I would decided to learn Rust at some point and it would be a nice resource.
Maybe something that would look at my recent history and say Hey X has been in your bookmarks for months and it's related to all the Y pages you've been visiting.
None of them are organized, I'd pay for something to automatically organize them.
Basically I just add the properties URL and BROWSER to any entry I want to be a bookmark. I have the numpad key 4 bound to a command that opens the url (works either in the org file buffer itself, or from an agenda view). I also have the numpad key 1 bound to an agenda search for the tag :bm: (searching for a property is too slow), so I can easily get a list of all bookmarks, which can then be filtered by tags, category, top-level headline and regexes.
Second is Favourites (like bookmark bar), I can access it from blank page. I'm saving webpages, that I visit often, news, important forums, etc. Also webpages, that I'm using currently in work (e.g. Postgres documentation, if I'm working with it right now.
Rest is just organized by topic list of webpages that I could use later. I'm not using it that often, but sometimes it might be handy.
I find them extremely useful for tutorials/learning new stuff which I know I need/want to learn but just don't have the time at the moment. Whether or not I end up coming back to them is a discussion for another day ;)
Most of my bookmarks are via HN.
I started doing this after accumulating a huge index of bookmarks spread across saved.io, Evernote, Google Bookmarks, iCloud, Firefox, Opera, txt files, Google Spaces and the other dozen or so bookmarking/collaborative knowledge sharing platforms showcased on Product Hunt.
I'm surprised there's no digital equivalent to the Hoarders TV show. I suppose thousands of bookmarks are less impressive than a garage full of old newspapers and rats.
For anything I want to remember for later, or keep for reference material, I clip it to Evernote instead. Find that much more useful, as normally when you're looking for a piece of reference material it's going to be able to remember some keywords from it than the title or where you filed the bookmark. Also means you can easily reference it even if the page is missing or changed in the future.
Also for the habit of reading/skimming articles and often hopping from one URL to another, I use One Tab. Super efficient to collect links in one page:https://chrome.google.com/webstore/detail/onetab/chphlpgkkbo...On the negative side, my work PC has over 800 URLs and home PC about 1500+
Why keep resources in a unique silo? You wouldn't keep all your PDFs/Word/rtf/&c in a "<ext> manager app", so why do URLs have to be kept in one?
Also, this way they all get backed up since I keep all work docs on my network drive.
I'm surprised no one else follows this pattern, but I've never seen anyone else use it, nor have I won over any converts via its sheer awesome factor <shrug>.
FYI, works in FF and Chrome, but not Opera. (Bummer, because I like Opera generally, and it's my default Android browser.)
I used to use Pocket a lot to do similar things, but categorising, and browsing the saved links was a little too cumbersome.
Plus I like that I can search just within my plethora of bookmarks if I want to reference something I know I saved a year ago. 
 - https://www.lifehacker.com.au/2015/02/quickly-search-just-ch...
I mean, Google is usually strong on that from with labels in Gmail, Keep but for some reason they never implemented that in their bookmarks. It would makes more sense than using folders IMO.
I do however use bookmarks on my laptop to point to locally installed document such as the full python library doc, in order to be able to access those when offline (eg: in a train).
Reason for this is that I love saving pages that I encounter. I use Evernote Web Clipper a lot, but it frequently fails to keep the styling perfect. Secondly, a lot of archivers can't save pages behind an authentication layer.
To be notified when it launches, let me know here!https://goo.gl/forms/X1IBqaA03kekR2Db2
I capture stuff to read in Pocket. If I eventually find the link to be valuable (news: almost never; how-tos: much more often), I move it into a Google Keep "PostIt".
The value-add is that I can add pics, notes, links to Dropbox docs, etc in the same PostIt, and organize them as I see fit.
I definitely would like some improvements. My "Topics" folder is huge and I don't really need it loaded each time the browser loads. Just save it in the topic and let me find it later. Also, if Chrome has my shortcuts, why doesn't Google highlight those in search results? And maybe auto-link the saved shortcuts to the terms I used when finding them in the first place. There's a lot of meta data in that action - search-search-search, save. Google knows quite a bit of my thought process (via keywords and sequence), so use that.
I stopped categorizing my bookmarks into folders a long time ago, however. They just end in a single folder. Though, I love to use 'tags', which I use for important stuff, that I want to distinguish from other important stuff.
For personal machines, I've got about 5 machine+OS combinations, with 2-3 browsers on each that I use for various things. I chose not to set up sync accounts in any of the browsers (I've already got too many damn accounts to manage, thank you!). So I sometimes save a bookmark if I'm in the middle of a long series of pages about something, as a sometimes-completely-literal "bookmark".
I use Bookmark Box to sync with other browsers by way of Dropbox. Its not perfect, but it works.
For the rest of the Bookmarks bar I have my most common links -- a few spreadsheets (in Drive), some web apps, and a folder for forums I frequent. I also have a bookmarket for Pepperplate, which I use on a regular basis.
- tabs I haven't read yet, but I need to restart my browser for some reason, and I want to be sure the tabs won't be lost ; in this case, those bookmarks are disposed of as soon as the browser restarted
- content I'm pretty sure I'll want to read back in a few time
I only use the bookmark bar, so I have to limit what I save. When it gets too big, I clean it up.
Organized in ~150 folders all with subfolders. Ditched the bookmark services when Chrome started syncing data across devices. There are three features I would love:
1. The search box in the manager does a full text search of the content on the bookmarked page instead of just the title (at the time it was marked, not updated). 2. The ability to search by URL with regex. 3. Show the date I bookmarked the page.
Just google everything when I need to find something. In the past I was using bookmarks to track blogs I follow but these days there is too much content. I just google/HN search stuff when I need to find something. I tried going back to bookmarking stuff/save for later but I just never got time to go back to the them.
I save my bookmarks in a text file, commit it and push it to a remote Git repo. I have this Git repo cloned on every system I use. Since the Vim editor is part of my daily workflow, visiting one of the URLs in the text file is a simple matter of pressing `gx` while the cursor is on a URL.
This is useful to me because I have this repo cloned on every system I use for various reasons, e.g. it contains my daily notes, productivity scripts, etc. So it makes sense to keep all my bookmarks also in this repo. Also having the bookmarks in a text file provides me the flexibility to add arbitrary notes/comments for each URL I save. The fact that I don't have to use the mouse and I can use Vim search or motion commands to find a bookmark is a bonus.
I also have folders for Work, Blogs and one for improving myself as a developer. I love browser bookmarks, but am not exactly a poweruser, but I would miss them very much if they were taken away.
My most desired feature in Chrome is being able to right-click a link and add it to my bookmarks. Presently, I have to open the link in a new tab/window (using right-click, then T or W) then go to the tab/window, click on the bookmark star, then close the page (i.e. before it finishes loading). If I want to avoid loading a page I don't want to look at right now, I'll right-click on the link, then E to copy the link to the clipboard, then go to a new tab, bookmark it, right-click on the new blank bookmark link, then E to open the editor dialog, type in some suitable title, tab to the address text box, paste in the URL, then click Close. Either way, it just isn't simple.
Apart from that, I also use browser bookmarks for links I want to (or have to) view in the near future. It acts as a constant reminder since it's always visually present.
I use Pocket for articles/links I can afford to view during my free time.
Structurally my bookmarks are an ever growing list, they all go into the bookmarks menu in firefox. I occasionally tag them too. Most bookmarks are part of my "online library", I keep them so that if I ever want to send a link to sth. I liked to someone, use them in an article, or maybe read again. I have a separate read-it-later list in an Org-mode file.
Some of the bookmarks are shortcuts, mostly to different dictionaries in WordReference, to Collins english dictionary, and various websites I browse often, like Reddit, HN, my school's, and my own website that I check every-so-often when I upload sth. new.
With the intelligent address bars of the browsers, you can search and find for most of the recent stuff that you used, and even sometimes very old stuff.
I don't use bookmarks anymore, and I feel like the bookmark bar is most of the time a useless distraction.
If there is things I really want to keep, I post it in a public Shaarli instance where I force myself to use tags, description and informative title.
Edit: removed markdown
I use Google Keep to store URLs, typically with some note, for example:* "Specialized Sirrus bike rear derailleur. Model number: DO20. URL: https://www.amazon.co.uk/dp/B0047D192E/ "* 2015-03-01: Visited doctor. They referred me to physio, and told me to read http://www.arthritisresearchuk.org/arthritis-information/con... for exercises/stretches to relieve pain."
Google Keep supports tagging and search, so I can usually find things. For things I want to read later, I either put it in Pocket or use Google Keeps' reminder functionality.
Chrome integration looks decent (save web pages as an image), but Firefox integration is lacking.
It's also useful to bookmark in browser because the address bar gives priority to your bookmarks over auto-complete and history.. So it's much easier to access those sites too.
P.S. I organize them by folder, so it's most likely design -> landing pages -> dark -> bookmark or personal -> finance -> bookmark, etc.
If any Chrome Developer is listening:
It will be amazing if there is some form of autocomplete to specify the folder for the bookmark. Right now on Mac, finding the folder in the drop-down is very hard. To find a folder, typing needs be fast. I almost never find the right folder, if the folder name contains a space. As after the space it starts to match from the first letter in the folder names if you take a brief pause to start typing the next letter.
I have so many Chrome bookmark folders that I don't know where anything is. The only way to find one is to just search in the Bookmark Manager. It sucks.
It also doesn't help that my preferred browser on different devices is a different browser.
I hope you are asking this question because you want to do something about this State of Affairs. I would gladly enjoy a good service that solves this problem in some innovative way that my brain cannot come up with.
My only other use is for groups of pages that I'm referring to or want to come back to as part of a project. I usually delete them after a few weeks.
For long term bookmarks I use pinboard.in
1. Autocompletion: for any website I use regularly I just write a substring of the url or Title (Firefox does this especially well). This covers probably 70% of my browsing.
2. Google. This might take slightly longer in case I want to find a specific article I had read some time ago, but it still seems less effort that having to bother with bookmarks, in my experience: either you have a very long list of unsorted bookmarks, in witch it's hard to search, or you have to spend time sorting them into sub-folders.
Now that I think of it, the following would be a very useful Google feature: +1 an url so that it becomes much more likely to bubble to the top in future searches.
I never have to worry about them going away and I can organize them into folders any way I like. Plus, they can be exported, imported, and shared. I use Firefox, so accessing the bookmarks is as simple as using a drop down menu. I actually use a bookmark tool bar for my ost frequently visited sites. This way, when I want to go to HN, for instance, I just click a single button and I'm there.
I've looked at other bookmarking software/services, and I still find that plain old browser bookmarks still fit every use case.
I also have bookmarks at the root level for things that I will Definitely See Tomorrow, which I never erase, because hey, They could be important.
Since it's the weekend, have this extremely educational video about languages https://www.destroyallsoftware.com/talks/wat
The Bookmarks Bar really has the only ones I regularly use though. Wish it was 2 rows.
I am currently running Bookmark Checker (chrome extension) and did set the parameters to "error connect" and at this very moment it is reporting : "Bookmark check status: Total bookmarks : 2238 of 3413 error connect: 2117"
so many dead links :(
I did not know about pinboard and I am really tempted to give it a try so I can do a full html archive without the fear of losing again 2000+ bookmarks in 5 years from now.
I bookmark most pages I view as unsorted bookmarks (especially helpful for news sites that have essentially no way to ever find old articles) and then ones I am more interested in I add to another folder that I occasionally divide into smaller folders (to avoid needing to scroll) and put all of these smaller folders ordered chronologically in a folder to the right of the tabs. I usually search the bookmarks first when looking for something, but I don't tag and too often neither the title nor url contain the right keywords for me to find it.
I would really love a more unified bookmark/history system along the lines of Vivaldi's calendar history, but being able to create icons that will flag the current page (to be able to look through just the more interesting history) and other icons that would cause the current page to be saved to a particular folder as a bookmark. Then at most one click would reproduce my current system other than occasional reorganization. Since I can't predict in advance most of what I want to refer to again, I want it to take as little time as possible to bookmark things. I liked the star in Firefox better when it didn't pop up the folder selection unless you clicked it twice.
For most sites I use keyboard shortcuts + the autocomplete in chrome. Aka to visit hackernews: Ctrl+L and then "news.y" and hit enter.
1 - Pages I want to autocomplete so I don't have to remember and type the full address or verify that I'm on the true site for my bank and not a phishing site
2 - Content to do something about in the future. Stuff to read later, stuff to download to my local machine, etc.
3 - Resources that I want to remember exist and be able to find. For example, I've got a page saved that produces blank graphics in whatever dimensions you want for use in stuff like web design. Forgetting what it is called, I could look it up in my bookmarks pretty quickly instead of having to open photoshop and create such graphics manually
1. Archival. If I like something and will need to refer to it/revisit it later (more than a month, say) I will bookmark it.
2. Frequently used pages sit neatly on my bookmarks bar so that I can get to the websites I want quickly just by glancing at their favicons.
I primarily organize in 3 levels.
Top level: This is where frequently used stuff goes. I have configured FF to only show favicons for these so they take little space. eg. HN, GitHub, Outlook, Reddit and Bugzilla.
Second level: This is where things go for archival. I have bookmark folders at the top level that represent a category. eg. Books, Movies, Tech, Coding. Each of those can be further categorised. An example is my Tech folder is broken up into Articles, Blogs, Podcasts, Material (projects, GH repos etc.).
The void: This is the final level or organisation and is just a catch-all folder called Sort-These-Out where all stuff I'm too lazy to organise (or which isn't well defined right now, or things I'll get back on another machine maybe (Linux vs Windows)) goes. It currently has 13 bookmarks. Not bad.
PS: Did you know you can send tabs across Firefox instances on different machines by right clicking and hitting "Send Tab to Device"? The best thing ever.
EDIT: Forgot these two features.
1. Keyword search. Kind of like the bang query syntax from DuckDuckGo you can set up a keyword to search a single website by creating a bookmark. So I can go 'gh mycoolrepo' for searching on GitHub.
2. Tags. Firefox allows you to tag bookmarks. It helps me a lot when, for example, I want to find all bookmarks related to vim (but don't necessarily have vim in the page title). I'm working on an autotagger that integrates into Firefox to save me from having to tag them myself.
: http://www.wikihow.com/Use-Firefox-Keywords (See method 2 for easier variant.
For longer term bookmarks I use pinboard.
I use a middle-ground for a few things: I may bookmark, say, news sites in pinboard under the "news" category. Every tag and combination of tags on pinboard has an RSS feed; I bookmark the "news" tag's RSS feed in Firefox, and everything tagged shows up.
The RSS is not for the content of the target sites, it's for what goes in and out of the news tag. So I might add another news site to my pinboard news tag, and vi-ola, it shows up in my Firefox RSS bookmark. Delete something from the pinboard tag and it's gone in Firefox.
For reference material, I built something sort of vaguely like pinboard.in into a home-brew app that I run for myself. It handles search, a modified form of tagging, and a timeline-like view, and I get to it with a JS bookmark (tada) that lives in-browser and sends selected text as a search.
(The app itself is a ridiculous mess, having grown as a sort of cancer in a different app I wrote for myself that now does several unrelated things. Maybe someday I'll pick that crap back apart into something releasable.)
(At the moment I have 519 bookmarks in 73 folders)
We maintain the bookmarks in yaml and generate the html to import into firefox/chrome/ie.Script: https://github.com/psiegman/bookmark-generator
I use Chrome. I like the fact that the bookmarks are synced to my Android phone and work computer. That way they are available whenever I want to use computer.
- pinboard. Been using it for many years
- DuckDuckGo's interrobang search to quickly access pin board bookmarks / maps / etc.
- the browser URL bars own autocomplete
- this may have happened also since until recently, I used different browsers on mobile (Firefox) vs desktop (safari)
I don't use the bookmark tabs, and I'm not regularly using what I have in my bookmarks, they're more for safekeeping, and to remind myself aobut things.
Plus it's fun to take a look through the bookmarks and rediscover things.
I have in my bookmark bar the most used sites. I have a few folders for topics and for work bookmarks.
Bookmarks help me to close a tab. It gives me the feeling that i still can read it but i don't have to do so now. Sometimes, depending on the content, i pocket it instead of using a bookmark for it.
I will definitely write a short review, but I need 10 people to view the link to help me access it first: https://stash.ai/landing?source=f520deef
Also admin stuff like webhost control panel, bugtracker, iTunes Connect etc.
Arranged in favourites bar in folders by category.
Saved articles go in pinboard.in however.
If I come across articles I like I save them to instapaper instead of bookmark.
For work...I've pretty much created my own wiki of bookmarks using OneNote. Employer uses SharePoint and some pages won't display or work correctly in Chrome, so I use IE for work intranet. So instead of bookmarks I have a notebook and put tags in the notebook for easy searching and I can put a good description of the site.
It has something called Links, where all URLs that you added are sorted by date. You can save a URL quickly with keyboard shortcut (CMD+D).
I also organize the links that I frequently visit by topics in the Lists section.
I never save anything for later, I either read it or forget it. I only regularly visit a few dozen sites, so usually the site is completed in the URL field after 1 character (such as "n" to load HN).
I make sure I use a descriptive sentence when I save them.
They're useful to me because the people creating the pages don't know about SEO and Google fucking sucks at giving me the pages I need unless I use weird contorted search phrases or remember the exact name of the document.
I have 12 icons in my bookmark toolbar that I use daily. I have a few that I don't use very often.
If it is an interesting web site with good articles, I subscribe to the RSS feed.
My bookmarks stay synced between my iPhone and Chrome on Windows using Apple's iCloud Chrome plug in. It stays synced between Chrome on different computers using my Chrome account.
Stuff I want archived for reference or I want to read later goes to Pinboard.in.
but my most usage of unity is just the left side bar. I don't use, for example, the search-and-find thing that much.
I like unity's color scheme too.
The only time I was pissed was when they enforced you to do amazon search in unity.
I actually don't think unity has lost to others technical-wise or design-wise. it loses, because it didn't follow the Principle of least astonishment https://en.wikipedia.org/wiki/Principle_of_least_astonishmen...
most people are reluctant to ui change. a small change would piss people off (ios5 to ios7, for example). unity just looks too different. the hatred is not necessarily rational I think.
if unity is going away, out of all the options, I like kde and xfce. I dislike gnome because of my early development experience with gdk.
That's really Unity's core failure; it was ambitiously designed to adapt to multiple formats (phone, tablet, laptop, desktop) and in the end, the required compromises led to a UX that wasn't _great_ in any form factor.
Microsoft tried the same thing with Windows 8 and had the EXACT same result. They alienated their core market of desktop/laptop users and completely failed to find traction in mobile/touch. Much like Ubuntu, Microsoft backed off that strategy with Windows 10 offering more substantially tailored experiences for mobile and desktop use.
Apple was smart and never tried a one size fits all strategy. Their mobile and desktop UX is entirely different.
Circling back to Unity, I also hate the window controls on the left. Not that controls on the right are intrinsically superior, just that I'm more accustomed to that location from other desktops. It seems contrarian and I just can't get used to it.
I try not to screw with working infrastructure, so it can become invisible to me. I've been using X11 for 24 years now. It's nice that it gained 3D capabilities, but I don't play games much, so the largest daily impact is that there are shadows under windows. Big whoop. Smooth video-playing is a much nicer feature, and we've had that for a decade.
I used fvwm, rxvt and netscape browser for years. Netscape mutated into Firefox. I added Chrome because it turns out that having two completely different browsers is an advantage for me. rxvt became rxvt-unicode. I might still be using fvwm, but XFCE has easier configuration when I move to a new machine, which happens often enough that I value it.
It will be nice when I can get the RX460 downstairs to drive the 4K monitor at 60Hz instead of 30Hz, but it's only going to be a minor improvement.
There are two kinds of idiots: the idiot who says "this is new, so it must be better" and the idiot who says "this is old, so it must be better". I try not to be either kind.
I believe fragmentation is a very big issue in the Free Software world. Since both Mir and Wayland wanted to work towards the same goal with minor differences in details, they should have banded together and have had proper discussions regarding both their ideas. It encourages better solutions and sometimes new things are born.
Mir was also never as performant as Wayland and didn't have any window manager apart from Unity to test with.
I personally don't have a thing against Unity but there were (16.04 days) some inconveniences like being unable to customize the dock. The separate toolbar and menu bar. GNOME does have thicker titlebars but combines toolbars, menubars and titlebars in their native apps.
GNOME also has a rich ecosystem of extensions. (Please stop breaking them through updates GNOME!!!). GNOME has a better universal search than Unity, is comparatively faster and snappier than Unity.
GNOME suffers from the problem of poor defaults which are easily fixed by distribution vendors. (Kali Linux ships with a dock, topbar and other IMO necessary extensions.)
My guess is that people think Canonical should have been focusing more on general distro reliability (e.g. the infamous "/boot fills up with kernels" bug) than reinventing parts of it. Hindsight is 20/20 though, we'd be having a different conversation if Mir had proven a big step forward over X.
If I remember correctly, to get a list of installed apps, you had to click "view more..." at least twice from inside the Unity menu. Compare that to Windows 8, which required right clicking the background of the Start Screen and then clicking "All Apps...", which is similarly nonintuitive.
Mir, don't know. What problem was it supposed to solve?
2) An SMS/MMS-enabled number is free from Google Voice.
3) Cell plans don't require credit (except possibly for the phone itself). You can pre-pay.
4) This may be discrimination, but so is requiring an email address. The question is whether it's illegal discrimination, and the answer is no. "Phone type" is not a protected class, like race or religion, in the United States. "Income level" is also not a protected class.
I've read an interview with ex-assistant on one of those shows and it was part of her job to routinely rile up contestants just before they were put in front of the camera. In this specific case, it was a "Top Chef" kind of show, and she was telling everyone that the celebrity chef running the show has just tried the contestant's dish and said that it's utter garbage etc (while in fact the dish was fine and they were not close to elimination) - everything to get them shaken and close to tears for the show. Very nasty business.
Statistically speaking, I'd bet someone entering American Idol is more likely to win than a startup is likely to become a unicorn.
(Also -- it's fine and normal to cry if your startup fails. For many or most, it will even result in clinical depression.)
Any generalization you get is going to be anecdotal far beyond the point of being meaningless.
There are amazing devs in every country. A better question is how to find out whether someone is a genius before hiring her and how to search for such people efficiently.
> African developers in general?
Don't be your racist hiring manager. Stop trying to generalize.
If your developer is from Nigeria or Ghana there is a high chance he may be speaking the truth. Churn rates of good developers in those countries are pretty high. Focus on mathematics and algorithms are used to weed out the weak developers.
South Africa and Kenya also produce very talented developers and my general experience with them has been mostly pleasant.
If you're looking for Native English speakers with manageable accents, look no further.
 - https://sailfishos.org/ - https://en.m.wikipedia.org/wiki/Sailfish_OS#Hardware_overvie...
Also: cost of living is a huge factor in salary, if you earn over 100k in the Netherlands you're considered loaded, but I keep hearing stories about software devs in the US starting out with 100k.
I will suggest you delete everything but the above. You will get better answers. Otherwise, you can probably expect a lot of snark and not much real information.
$84k US but living in a part of Canada much cheaper than Vancouver. 4 days a week. No crazy corporate or startup deadlines. All open source.
Trying living on 25k in Los Angeles.
- uMatrix (host-based filtering): https://github.com/gorhill/umatrix/
- HTTPS Everywhere: https://www.eff.org/https-everywhere
- Self Destructing Cookies (fine grained cookie control): https://addons.mozilla.org/en-US/firefox/addon/self-destruct..., the latest Chrome & FF have better settings for this built-in though.
Hopefully it won't be discontinued soon..
- Ghostery https://www.ghostery.com
- uBlock Origin https://www.ublock.org
- Stylish http://sobolev.us/stylish/ - https://userstyles.org
-Emoji Keyboard (2016) by EmojiOne
-ImprovedTube for a default x2 playback on yt
-Google translate, lets you select any text and get an instant translation
-Octotree, a directory tree for github
- Session buddy
- uBlock Origin
- Privacy Badger
- HTTPS Everywhere
The Great Suspender
https://lwn.net/Articles/718267/ / https://news.ycombinator.com/item?id=14002386
EDIT: This just got downvoted to 0, unsure why...
I'm operating under the assumption that Apple is more privacy friendly than Google under pretty much any circumstance.
I've done the Cyanogen + F-Droid thing and it's just miserably inconvenient, and none of the "proprietary" alternative app stores have any sort of catalog to write home about.
Is there a way currently to install baremetal Linux on a phone without emulating it? A small Linux, usb microphone and VoIP would get me 90% there.
My dream would be an android "distribution", that doesn't rely on some murky "update by getting a new image somewhere if you're lucky enough that someone built one for your device". WOuld work more like a linux distribution (packages and updating) and is generic over a variety of phones. Challenge is probably how to handle drivers.
Jolla -- company behind SailfishOS (ex-Nokia people) https://jolla.com/about/
As a replacement for the Play Store, check out https://f-droid.org/
However news that one researcher has found 40 0-days for it doesn't really sound good:
I remember reading about Plasma Mobile
But it looks like the latest phone they use as a Dev device is a Nexus 5x so it may be stalled/dead
Sony put everything you need to run your OS on some of their Xperia phones on GitHub as part of their Open Device Program. Jolla which is going to become open source sometime has been successfully (in feburary 2017) run on one of the devices.
So in the near future, there is hope in that direction.
I think the biggest issue is still device support.
Question is, are you willing to spend money (or testing/integration efforts) on a community O/S for your phone in exchange for knowing that you're not being spied on all the time?
I could imagine a model where you pay a reasonable price for a tested/supported third-party O/S, but maybe CM has shown this isn't economically feasible.
I've been running it for some time now and I like it a lot. Support for a lot of phones? No. Wide range of apps available via F-droid? Decent.
I'm not a very appy guy though. Most of my needs are browser based.
They truly care about the privacy and redesign the hardware according to this goal.
Now the thing with Android is that, it is just a framework, so saying Android is Open Source is missing big pointsThere are two major limits:1. What about apps?2. What about drivers?
Today, if you get an Android phone with Google Apps, especially Google Play Store, you have access to many tools that can be considered useful for everyday use, which you won't have with FOSS Android. Here are some examples:
With Play Store, you can have (mostly?) any IM, as long as you install the app going with it. The only possibility I know to do that with FOSS Android, is to use matrix or IRC, and have server-side libpuprle connectors.
With FOSS Android You don't have factorized push socket: Most apps pushing notifications on Android require GCM. But even if it doesn't require GCM, there is nothing fully open-source an app developer can use. All they can do, is open a socket to their own server, and deal with it, which is a huge battery-killer.
Many people are mentioning f-droid as an alternative to Play Store. I'm sorry, but I consider this a joke. I highly respect the work done on f-droid, but this is not a usable alternative.For instance, you want to save your SMS. We call that backup, but not every knows that.Well you search for "save sms" on f-droid. No result.You search for "save sms" on Google Play Store, the second result is SMS Backup+ which is open-source! You search for SMS, the result is on the first page.Same thing happen if you just search for "sms", QKSMS (an opensource SMS application) is much easier to find with Google Play Store than f-droid.Even to look for open-source apps, you're better off with Google Play Store!Again, I totally respect F-Droid devs, this is this way because of their choice of not tracking or saving any user information at all, which is legit.But then, some people might want something intermediary. Just counting the number of installations of an app can be really useful to better sort apps (SMS Backup+ and QKSMS really deserve to be among the top in the results for SMS).
Now, about the drivers. Yes Android is open-source, but good luck running a phone with a FOSS Android!At the moment, you have the choice with either replicant, which is old and missing gpu acceleration, or running a mainline Linux kernel with Mesa & stuff, but then you have no radio.
Though I have to mention that on the drivers side, Sailfish OS and Ubuntu phone have also those problems.
Reddit AMA: https://www.reddit.com/r/raspberry_pi/comments/5nwmfx/im_mak...
$50. Calling, SMS.Alarm clock, calendar, calculator, phonebook, file browser, web browser and music player. I expect these will be very simple given the screen.
 check out https://stats.lineageos.org/ for popular devices and also check the devices subforum on the XDA forums
If you don't care about the aesthetic, your best bet is Lineage. It is Android without Google and you can still get lots of stuff done.
Personally, I came back to a cheap WebOS phone with a cheap plan and after I installed WebOS 2, it has been really usable. LuneOS is supposed to be the continuation of WebOS, but it's just not that usable yet - I see no way to sync with my Google account (which syncs fine with the 10 year old WebOS instance via Exchange). So I would keep an eye on the LuneOS development. For such a niche and small dev team, they have been continuing making improvements. Totally impressed by that.
(For a little while, there was an unofficial Whatsapp API which I used to bridge chats to my Firefox OS ZTE)
* Tizen OShttps://developer.tizen.org/https://en.wikipedia.org/wiki/Tizen
Alternatively you could use Termux on Android to use the GNU/Linux ecosystem on your phone.
Project seems to be abandoned?
Inb4 nay-sayers, but if you have followed Fuchsia for a few months and have seen the speed of development and just how many people and new technologies are involved, you will see why this isn't even a question.
If you want to host site on S3 you can use the fork of Simply Static https://github.com/kennu/simply-static-s3
What is the purpose of this exercise? Blogging or learning WP->static page migration path? If your purpose is blogging and you have no interest in admin and revenue generation, just host on WordPress.com. If you want more flexibility, just self-host WordPress.
Why make it more complicated by adding another path of WP->Static Pages?
They also have very recently released new themes.
At a previous employer, though, we would get a raise for earning various certifications so I took a bunch of them that I wouldn't normally have taken. Most were worth at least a $1000 (USD) raise (added to my salary) and some (i.e. CCNP) were worth much more.
Bedridden homemaker needs good drugs: "Clearly, she's a hypochondriac and addicted to antibiotics. No, bitch."
Same woman, different scenario:
Newly diagnosed with CF, borrows $19k to go to a summer tech boot camp: "Sure, honey, what drugs do you usually prefer? Here, let me tack on a few more for good measure."
Still alive nearly 15 years later.
We started to put on premise machines that have GPUs in them to reduce our costs. We can get a desktop class Core i7 machine with a GeForce 1060 6GB for around $800 all in, where as on Amazon you have to have a G2 2.xlarge on demand it is $400/month, or $3478 one year term (that is for 1/4 of a G2 8.xlarge achine, thus you get one of its 4 Quadro K5000 4GB cards to yourself.) I could put 2 1060s in that machine for another $200 or so. This just beats AWS pricing in <4 months of use.
My expectation is that these 1060s will be good for two years, maybe even three.
Because our GPU tasks are low bandwidth and non-critical we can run them on our existing office internet. Our rent also includes power. So there are not too many additional costs over the bare machine costs.
To save around 80% (but slightly less convenient) there are two ways to use spot instances provided on http://forums.fast.ai that are largely automated and work well.
In part 2 of the course you'll learn how to set up your own server. As others have mentioned that's much better value if you're using deep learning somewhat regularly.
(I'm the founder/teacher of this course.)
With a spot request you can get a p2.xlarge for around $0.22/hr, which for me is sufficient for moocs and side projects, but for beefier instances the prices get high.
Unless you are able to get a decent deal (far better than reserved instances) then for raw CPU/GPU power hosting your own is better. Its the whole package that makes AWS attractive. (RDS, S3, instant spinup)
Spot prices is a good place to start if you are doing batches. just make sure you checkpoint little and often.
However if you are on your own, with limited funds, Get a second hand workstation (dell or hp z620/600) they tend to have decent PSUs, loads of ram and decent support, and a couple of Geforces with more Vram. (Quadros you are paying for the double float performance and ram. transfer Bandwidth and speed are the same or slightly better on geforce class cards.)
Best of luck!
Even my gtx 1080 is faster, albeit with slightly less ram than the K80 or M60s available in the cloud.
A titan X would fix the ram issue though.
The most economical would probably be to colocate some titan X boxes. Not only cheaper, but more percormant than anything you can get from IaaS right now.
i7 w/ dual 1060 GPUs?
So you need to work consistently 1-2 hours a day on your side project. It really doesn't matter what you do. If you manage to get those 1-2 hours in, you will muddle through and accomplish something. If your goal is to make a side project and bring in a non-zero amount of money, this is achievable. Learn whatever you learn on that project and then do it again.
Personally, I would spend exactly $0 on your task because, like I said, the thing that will kill you in the end is likely to be time commitment. If you spend money, you will be out the money and your time. So start with time and see where it takes you.
As others have said, no need to get fancy. Just build the simplest thing that will get you started, using the simplest tools you can find.
The idea is to look for apps that have low ratings, high downloads and lots of recent comments, then make them better. You can use synonyms and the same niche category to increase visibility on google play. This is where the money is.
So I made my own, and fixed the bug: http://bit-booster.com/best.html
And then I realized I needed a rebase button on the pull-request screen... and so it continues to evolve.
Here's the thing: I've always known I'm a good maintenance programmer. I've always preferred working on existing software instead of making new software from scratch. And writing add-ons for Bitbucket is basically just another form of maintenance programming: reading Bitbucket's code, noticing its flaws and shortcomings, and fixing them.
Also, I love git, and I love going very deep into git (e.g., https://github.com/gsylvie/git-reverse.sh). So this is my dream job.
I've only made $7,000 USD after 1 year on this side project. But $1 of those dollars feels better than $10,000 from my day job.
Since you don't have much knowledge of FE development, I would suggest you keep things Simple Stupid and try to do as much as possible with HTML and jQuery. I have created really complex websites using just PHP and jQuery (sites that have made me 6 figures over time), plus you will learn the real nitty-gritty like DOM manipulation, CSS tricks, etc - which you will need to use anyway at least a few times regardless of the shiny JS framework.
I would highly recommend at this time you don't get sucked into the React, Node, Vue, etc. You will only end up wasting months without nothing to show for it (but maybe I'm just too old school).
Whatever time you have left after that, use it to learn online marketing. Learn about list building, SEO, Copywriting, outreach and affiliate marketing. Because that's how you turn your technology into actual money.
You'd be surprised how little some of these businesses know. I have previously;- Built a travel database in MS Access for a Travel Agent (long time ago)- Ordered and setup ADSL connections and email for a water tank manufacturer and a furniture store- Capture requirements, researched, ordered and installed an office (6 people) worth of IT kit for a not-for-profit (didn't charge them for this work).- Designed and implemented a roster management system for an IT helpdesk for a university.
There are heaps of opportunities. Just have to know where to look.
You don't need to use the latest and greatest tech, in fact, I would urge you don't. For front end you can stick to simple JQuery interactions, bootstrap theme and you'll be fine, depending on the market sector you go for they may not even care about the design, so long as it's functional.
Summary of my project and where they came from:
http://www.oneqstn.com, before launching our company's product I put the question "Where would you expect to buy the Radfan?" on the shop page and then 5 options. I expanded this on its own dedicated domain and 5 years later it's still ticking along. Very popular in the middle east for some reason.
http://www.stockcontrollerapp.com, I manage in-house production of my company's hardware product and after moving from an Excel spreadsheet to a Python script I decided to make a stock management app for small factories. Has made my work life much easier and is more appropriate for me than Unleashed.
http://www.taptimerapp.com, I didn't like any of the timer apps that I had tried so made my own mainly for use in the gym. All the others had too small touch targets, hard to see at a distance/without glasses on, or stopped music playing when the timer finished so I made an app that addressed these.
Finding a real business problem and a real solution is what matters. The tech is just an implementation detail you work out later.
If you're looking for some inspiration from others who've built revenue-generating side projects and businesses, I'd start here.
IMO building a team (2 or 3) is the best way to go...
Hopefully you'll find the idea useful (ideally this should be a website but I'm testing the waters with a simple stylesheet...)
A lot of my focus was boiling down what approaches could plausibly work and what pitfalls to avoid.
Here's one case: the local/popular site to search for used cars sucks. It is slow, hard to see/compare all options, silly reloads the page on each added filter, filled with outdated listings, flooded with ads, and pic slides take forever (all of this on my slow phone over a slow 3g which is how most visitors must be using it). Furthermore, car dealers (who post most listings) complain about service and price. So I built the proverbial mvp and put it in the hands of my marketing partner (you won't sell a line of code if you do not partner with a person/company dedicated to push your stuff) who's already working on a deal with the used car dealers association, pitching a novel business plan, hopefully making some passive income for both of us.
Creating a course can get you the momentum. You can start there and branch out to other things.
I'm proud to say that very recently I did manage to complete a side project that I intend to launch in a week or so. It is a social site based on an idea I got from watching Japanese dramas.
What helped in my case was that my idea was really simple to build. I too have zero frontend / web design ability, so I just paid a guy from Craigslist to fix it up. Being able to bootstrap a finished product with a relatively small amount of time / money helps you get in that "closer" mentality instead of just playing around and never finishing.
I'd also suggest not worrying about making money at first. Just try to make a cool product or service. Money is a stressful and distracting motivator I find. Once you have something of value to offer and get some feedback from potential users, then you think more about pricing and marketing.
So in short, start small, don't be afraid to outsource and trade money for time, and don't worry about making a profit right away. That worked for me at least.
I build https://noty.im that way, an monitoring tool that call me when my site is down.
Then I realized more thing is needed and I started to add more features and plan to publish launch soon.
I will say not to worry about scale and technical first. I learned it the hardway, Just get it out. No one will really care if it's broken or something doesn't owrk when you doesn't have lots of customer.
So to answer your questions:
1. Can you provide some ideas on where to start?Pick a technology stack you familiar with. Apply to Microsft BizSpark to take advantage of $150/monthly credit. Learn FE, it isn't that hard.
2. What are some simple things I can build by myself? Any idea?
I build `https://kolor.ml` in a sunday. It's very simple but I need it. So you can try to build some simple/small utility that helps people with their daily live such as: a tool to call people up in morning.
A tool to check if my site has expose some particular header such as `nginx`, `php version` etc and if found an old one or vulnerable one, alert.
Of course, lots of people already build those, but the point is just get started, along the way you will realize what you really want to build
This is the approach I took with my book/videos at AngularOnRails.com, a "side project that makes money".
Another important thing is to surround yourself with people who have successfully done the thing you're trying to do.
I don't have much time right now but if you (or anybody) wants to talk about building side projects that make money, feel free to email me at email@example.com. I'm not an expert but I know a hell of a lot more than I did 9 years ago when I started.
The simplest way to start is to take a framework or system that has most of the basic parts ready for you to use.
Since you already know python, try to learn something like django and use Bootstrap with a CDN for your front end stuff.
I would recommend reading some of the posts on indiehackers.com to get an idea of how those people got started with an idea and how they got their first customers. Some do not even have any tech skills and just used wordpress or found someone to help them with the site.There is also a podcast for this that just got started that is excellent. The founder of indiehackers was a YC alumn named Courtland, he is a cool guy.
I chose to solve a problem that I personally encountered. If you cannot think of something, try picking something that you know requires lots of manual effort for some people. Then use some scripts from the book Running Lean to try to work out exactly what the problem is for those people.
Another great resource is OppsDaily which I love reading first thing in the morning. Cory sends out a problem someone has in a particular industry that needs to be solved. The criteria is that they must be willing to pay for it if someone responds. In many cases they will say how much they are willing to pay.
So, pick something _you_ will use and, if you enjoy building/using it, so will others. Obviously think about ways to monetize it, but money should be more of a side-effect than a motivation.
I'm currently working on https://insomnia.rest, which makes around $800/mo right now. I started it as a side-project a couple years ago with no intention of making money. However, traffic grew organically and I eventually left my job to pursue it full-time.
In summary, find something you love to work on and let it consume you. If you do this, making $100/mo should come in no time. Have fun hacking!
You're already successful at selling enterprise services to at least one company in management and backend data processing. Have you considered selling management and backend data processing advise, perhaps delivered in the form of a PDF or series of videos? This is stupendously valuable to tech companies if you meaningfully improve on what they have already and would allow you to sell to people who have expense accounts tied to, to steal a friend's phrase, the economic engine of the planet.
You don't need a commandingly high bar of programming sophistication to sell books. There exist services that can do all the heavy lifting for you. If you prefer knocking together a site to sell your own books, it is essentially an hour to get the minimum thing to charge money and ~2 days to get something which could plausibly be the kernel of an ongoing business.
import requestsfrom bs4 import BeautifulSoup
url = "http://www.python.org"response = requests.get(url)
page = str(BeautifulSoup(response.content))
def getURL(page): """
:param page: html of web page (here: Python home page) :return: urls in that page """ start_link = page.find("a href") if start_link == -1: return None, 0 start_quote = page.find('"', start_link) end_quote = page.find('"', start_quote + 1) url = page[start_quote + 1: end_quote] return url, end_quote
The summary, well, sums it up pretty well I think:
> Listen to your friends, coworkers, and clients. Find something painful they mentioned that you also have first hand experience with, or that youve needed at your job. Package it up so its easy to use. Build an MVP, get feedback, iterate. Charge more than you think you should. Listen to your customers. Launch on ProductHunt. Market the hell out of it! Use Content Marketing, reach out to communities, forums, friends, and businesses with cold calls/emails. If youve built something great, word of mouth will do its magic. You can do this in your spare time, and probably should.
Desktop and Windows might seem like a dying market but that's what people use at work and those are the people who can pay for things.
I had some frontend dev skills but didn't have the backend chops, so I hired someone on Upwork. I'm pretty busy at work so getting someone else involved is key (If I was by myself I'm not sure I would have stuck with it).
It's been a year and the app is doing about $3k/mo in revenue.
Building an audience + market validation before building a sideproject are the top starting priority for "a side project that makes money".
Start by building an audience (this can be as simple as interacting with professionals on Twitter and/or their own blogs, or even contributing here on HN!). I won't be able to determine what people want without asking people, and I will save a lot of wasted time by building something that I can guarantee people already want to pay for! For example: I intend to walk around my neighborhood with a survey to gauge interest in localized "technology disaster prevention" (aka initial setup of PC & phone backups with verification and increasingly annoying reminders) as a service. The first sideproject is the hardest because initially the audience is smallest but then can be re-used.
I hesitated to post this because most developers have an "if you build it, they will come" mentality (and a tendency to focus on technology/implementation details that they enjoy) that even I personally have a hard time overcoming myself. However, if the criteria is making money, building an audience is the right first step. Once the bare-minimum MVP functions, marketing makes all the difference on the "that makes money" part (see my list of random books to buy elsewhere in this thread)... and there is no point over-engineering something I can't convince anyone to use!
I realize I'm going out on a limb a bit to say that market validation before each sideproject comes second... I know of one example of someone who has built an audience while publicly initiating sideprojects without thorough market validation (focusing on technology instead -- note that this determination is 100% my own armchair quarterbacking with the benefit of hindsight); this person's projects appear to be faltering because of poor market fit. However, it hasn't stopped many from buying into this person's brand / other projects, and that audience is now following the next project even though initially it appears to be trending toward the same mistake!
PS. As mentioned elsewhere I could shortcut market validation by tracking down commercial products (already being paid for) that are getting a lot of visibility and addressing issues raised in bad reviews; however even when going this route I will still benefit greatly by having an audience to market the replacement.
Edit: switched to first person to preach to myself to get off my butt and start doing something!
I wonder if more programming is going to bring back those creative juices, perhaps you should think about doing something away from the keyboard instead?
For example, you can write about management and backend data processing (what you do at work). This way, you don't have to learn something new to start your side project (except maybe how to manage a blog). The blog can be monetized via Ad networks like Adsense and Amazon affiliate program etc. As you grow, you may take in direct advertisers, sponsored content etc.
I would at minimum leverage bootstrap or semantic ui as your ui. Otherwise, hire someone to do the web interface for you.
First thing is you should decide why you're doing it. Is it to make money, or have fun, or enhance your current mgmt skills or to learn to code in a whole new area?
Only once you've decided that, then questions like "I don't have any frontend dev skills. Where should I start?" and "Should I outsource the website development part?" are possible to answer.
Recently, I started on project for the sole purpose of learning more about nodejs and then in the second phase angular 2.
So a few months later I have a process that extracts data from amazon api and looks for price decreases in products. I learned quite a bit about nodejs and even about mysql database structures. So it was a good learning exercise.
Although I have accomplished my objective, I want to make money. This is the problem... Now I have learned what I needed, unfortunately I have learn more about seo, twitter api and facebook api to get users to visit my 200k webpages and make some money. So the side work winds up becoming a challenge and sometimes a burden, to continue to figure out how to reach your goal.
But when you reach your goal of $100 a month, then you will want more... So basically it never ends.
Regarding FE dev: I also have a technical Background (EE) and I hated any CSS (HTML isn't so bad). Though, flexbox is a life changer. It's actually enjoyable and I can get stuff done without spending hours on simple layout issues.
I recently had a quiet period in my freelance work so spent the time learning React Native amongst other things. I applied this to an idea I have had rattling around in my head for a point tracking app for people on Slimming World. I spent 4 weeks developing this and then shipped it to iOS. In under 2 weeks it has grown to almost 10,000 registered users, is number 4 in the UK Lifestyle Free Apps chart (ahead of Slimming World's own app) and has made enough ad revenue to cover the only costs I have had (App Store membership costs). It is never going to bring in big money but the lessons I have learnt are priceless.
I've done so many complex projects that at the end I couldn't sell, that's frustrating... please hear me: figure out first how to sell it (or at least get good traffic to a crappy wordpress site), then build a very crappy version and then improve it over time.
I read recently this, and I think is gold:https://www.blackhatworld.com/seo/making-money-online-it-all...
The satisfaction you get from these side projects will come from being able to finish them as opposed to try and make money from them. When you try to take on a side project with the goal of making money, you'll end up sinking way too much time in marketing and reaching out to possible customers as opposed to building something (which I find to be more fun and rewarding).
And the time you spend on trying to get people to sign up or even try your product doesn't have the same returns in satisfaction as building it. (my opinion of course)
The goal is to stick to something until completion.
Phsychologically this is easier if you are enjoying it.
So choose a project and a tech stack you can really get into.
Scratch your own itch so that even if no one else uses it or buys it are least you can.
Look at the non-financial upsides, so that if you make zero or little money you can still feel proud: For example - learning new skills that might help you get a raise, learning marketing so that your next project is more likely to be successful, etc.
A word of warning - once you have spent some time on a side project the shine will wear off and it will feel like a job - and you have to find ways to keep yourself motivated when you could literally just go an watch TV instead on your time off.
It is in alpha stage right now, but I'm curious to hear your thoughts about it: https://chrome.google.com/webstore/detail/mnmn/kepcdifhbfjep...
Screenshot of how it looks like: https://github.com/simonebrunozzi/MNMN/blob/master/screensho...
(note: the "+" button is only available for users that are "editors". If you try it out, your user doesn't have that enabled by default).
You could build some kind of email integration, or something delivered over, or using, email. The email processing itself would be mostly backend stuff, you could template the emails with Django or even Python triple quote strings.
Or you could build an API of some sort. The only front end really needed for an API company ( such as Stripe or whatever ), is documentation. You can write your docs on one of those doc hosting platforms ( readthedocs might be one? I don't know much about it now ).
For your side project, it's probably important to pick some things you like and start doing them, rather than trying to make certain up front which things are going to make money.
Find an idea that plays to your strengths and build something with a friend/coworker who is a better frontend developer. A good partner is invaluable, and with this you can already see how.
Also, charge on day one imo.
Very simple to say, very difficult to do (I've never tried, but I would probably fail). But I think there would be ways to systematically apply steps 1-3 until you find the right product to work on.
really got me thinking about best ways (ie most time efficient + value) to vet an idea before lifting a finger building.
How about an iOS/Android game with optional micro-transactions?
In game development you can indulge and improve as many different skills as you want; graphics, sound, music, mathematics, networking, AI, UX design, character design, writing, storytelling, difficulty balancing, teaching..
An indie game project will give the freedom to be as creative as you want, and you get to enjoy your own product, but of course you don't have to arrive at a finished, marketable product to have fun building it.
Firstly do you have any code, processes, methods or scripts you have created as part of your job that saves your team time or money? Could you re-package them for sale?
I don't think that tech is a good starting point because your solution might not need to be software.
Try see other people activities and have a look how much they spend for basic services you could improve. If you suck at UX/UI, give some money to someone else who could do it for you
what matters is how fast you can release something, do not speed time in courses or fancy optimisations
Also do not focus on a side project alone, let create 3 or 4 project a year
You'll be amazed by how many ideas you never have to waste time building, if you put up a paywall and nobody pays.
You dont need frontend skills
Find one similar in your region. If it does not exist, start one.
Coming up with an idea for a product that is useful and that people will pay for is more difficult than actually implementing one.
One approach is to decide which customers you can find the easiest and then ask them about their problems. Start there.
Regarding monetization you can use AdSense, donation button or charge monthly based on type of your product.
With respect to a project I'll let you in on a secret: if your service does something valuable, it doesn't have to look pretty.
Whatever you do get it up there in an embarrassing state and keep making it better.
I wasted lots of time by wanting to do things right and so on.
Although you ask great questions, this bit is what worries me. It doesn't feel good - it seems your job is taking too much energy from you. Have you thought about getting another, easier job?
1. If you know python, you can probably make a pretty natural jump to Flask. I didn't know Python, but I could program in a handful of other languages, so I figured I'd pick it up as a useful tool anyway. You may like this tutorial:
I'd say I really started learning when I got to the stage on authentication. The reason is that this tutorial implements OpenID, which isn't very common anymore, so I went off and implemented OAuth instead - heavily googling and scavenging, but ultimately having to piece together something that worked myself. I learned a lot that afternoon.
2. As others have said, time commitment is the biggest issue. Figure out what you can give this, and scope appropriately.
3. I haven't done this because I'm too much of a completionist to pull the trigger, but get your MVP out there and build off of it. For me, I've decided I'm pretty happy just spending time on the project, even if no one else has seen it.
4. Bootstrap is my friend, it can be yours too. I have never been a strong visual person. I like words on a page. I have no eye for what makes a good visual and what doesn't, which has been my biggest developmental item when I moved into an executive role last year. All that said, Bootstrap is awesome and makes it a lot easier to build good looking websites. I started off here and built out a static website for an idea I'd had, and am now circling back to build the things I want to be dynamic in Flask.
5. There are a lot of choices out there. Unless you're developing bleeding edge, and I may get flamed for this, most of the choices really don't matter that much. I chose Python+Flask+Bootstrap because I liked each individually, it seemed like something I could work with, and NOT because I decided they were objectively better than Node, Angular, Express, React, or anything else that I haven't touched. I also sort of like that there isn't a new Python web framework each day, so diving into Flask seems like a more stable investment of my time. I'm sure there are drawbacks.
6. When it all starts to come together, the real, revenue generating idea might be to address pain points in your day job. My sector is insurance. I know a lot about certain operational functions. Eventually, I could solve some of those and build a business around it, I tell myself. You probably have some specialized domain knowledge as well. Consider that.
Good luck, and have fun. Like I said, I'm happier just for having taken on the challenge. If I ever make a dollar, that'd be good too, but less important than I initially thought.
Be humble, trust win, trust lose, trust again. Adapt to the market, don't think it'll adapt to you!
Also, the real meaning of life is to find your creator, not make millions of dollars. When they bury your sad fat rear end in the ground, all you've made goes to someone else. The afterlife is a part of life. Read Acts 2:38.
* Don't necessarily trust, without thorough vetting, those who made money extremely quickly and claim to truly understand business. Being successful at business and understanding business can and often are very different achievements.
* You can entirely do it on your own, you don't need anyone else. 
* You cannot entirely do it on your own, you need a partner. 
* Maybe my greatest business insight: the longer it takes to find revenue to less likely you will be to find revenue or profitability.
 These are not mutually exclusive. For me, I needed to realize that I did not have to be trapped by a partner. And I also needed to realize I should not be trapped by doing things by myself. Picking the right partner is essential whether in business or just in life. I have been lucky to find them in both cases.
Do what you enjoy in the evenings.
And if you still want to do a side project then work on it and market it for maximum of an hour every day but for, at least, a year before starting a next project.
They aren't where they are because everything they do is brilliant. A lot of what they do can be improved upon. Admiring their accomplishments is fine. Trying to understand their success is fine. But assuming that they are unequivocally better than me in every way is problematic and simply not true.
I wish I had gotten the memo sooner that other people in positions of influence or whatever don't necessarily know more than me or have better answers than I have. It is a lot more complicated than that.
To expand on that, let's say that you want to become an uber-programmer, like John Carmack. In his own words (paraphrasing), "everyone can do it, all resources are available online, you just need sit and do it". While this is technically true, the reality is that he seems to be extremely gifted when it comes to sustaining focus when working on abstract/technical tasks (while your average developer starts checking out after 4-5 hours). If you're like that and read about people like him, the conclusion is that you're like a short person trying to compete in NBA - technically feasible, but the odds are against you.
-Don't get lost in the weeds. It's easy to let small things that are immediately interesting take precedence over more important things.
-You won't be what you are now in 10 or 20 years. You'll think differently and have different abilities. Age comes before you know it. Remember this always and don't think you have it all figured out because you never do.
-Work hard but take it easy. Don't let stress over little shit destroy your long term productivity (see lost in the weeds section). It's a marathon not a sprint.
-Things change fast. Plan for the future but do so loosely because you are probably going to have to re-asses shortly enough.
-A little a day for 10 years is worth more than a lot for 1 year in terms of what it gets you. Saving, learning, all those thing. Don't plan on big windfalls.
Stand up for your real values and stay true to them.
Don't just learn something, but publish, write, talk, demonstrate what you have learned. It helps build a public profile.
Who do you give energy and to what tasks? Make sure it's good.
Make a goal because it's a good exercise to start the journey. But just follow your heart after you start walking because if it was meaningful, you will find some way to incorporate it into your life.
Go slower. Enjoy the things you could only in university; there's no need to graduate earlier; you can work while going to school, but you cannot go back to thinking like a student after you've started working.
Invest as early as possible. Don't throw away money on flashy material crap.
Take care of your health.
If it's not working for you, walk away.
Have fun. If you're not having fun, change things.
Deal with fear and anger. They're useful, but only if you take positive action. Let them fester, and they will destroy you.
There are no guarantees.
It doesn't fit needs of @pmchorus company, but is useful for simpler cases when you need something more efficient than cron and more accurate than queue delayed jobs. (Example: we use it for sending triggered email deliveries, where batch processing of letters using cron is not possible, because of amount of letters and cron schedule limit)
The idea is that you schedule job on specified time and cronboy invokes it via http and execute it on your server.
- Seconds accuracy of execution
- Statistics on invoked jobs
- Secure execution of your code
- Retrying if job failed
- Laravel friendly facade api
What I really appreciate about Airflow is that it is self documenting. Anyone can logon and see the exact workflow, schedule of that workflow, the status of last few job runs and the python code that ran if they desire. Airflow documentation itself is good and getting better.
It can run a simple cron job but it also gives you room to build it out into much more. We use our Airflow scheduler to call simple API endpoints and then that same server does daily loads into our data warehouse.
Also 1.8 just dropped to stable and it is a huge improvement upon earlier versions.
2. Listen to what they say.
3. Try to understand why they say what they say; if necessary, ask questions.
4. If you ask questions, ask open-ended questions. ("Why do you say that?" is a great one to start with.)
5. Watch people whom you know from experience to be good conversationalists, emulate them.
6. Practice :-)
There's one solution.
- Talk to your coworkers
- Attempt to lead conversations
- Do not interrupt others
There's no alternative that will result in such a tight feedback loop. Try to do this as frequently as possible and your conversational skills will improve, guaranteed.
And always ask open-ended questions, the person asking the questions will control the direction of the conversation.
You could also try pursuing ride sharing arrangements. That can foster good conversation as well, while you are both stuck in the same vehicle, heading to or from work.
Two things he mentioned that stand out for your question: first, Google attempts to hire generalists, or at least "fungible specialists". Three of your five SE interviews will be general CS algorithms questions; the other two are likely to be specialized if you're interviewing for a specialized role. I don't think that it's likely to be different from the other "Software Engineer, Foo" interviews (e.g., "Software Engineer, Front End" or "Software Engineer, Mobile"), where at least one of the remaining two interviews will probably be specific to Foo.
Second, he specifically complained about people who show up and say "I want to do machine learning" and then say they have no machine learning experience / background. There are apparently a very small number of teams who will train a bright person how to do ML, but in general you're expected to have some background with it.
This seems like the sort of thing you should ask your recruiter (after getting in touch with one) and perhaps ask at a coaching session, if there's one in your city. I am a little genuinely confused that they seem to think their interviews need a coaching session, but hey, at least it's progress.
So pretty much no difference from a standard Google interview.
3 General White board coding rounds by SWEs(with some emphasis on data processing appropriate data structures, computations etc)
1 or 2 rounds specific to machine learning with some one working in that research area ( 1 or 2 based on the level you are coming in probably)
But as someone in the comments thread pointed out recruiter would be the best source
They give you an assignment to complete at your pace for a few days, and review your solutions and implementation with a group of your (future?) peers. This takes the form a small Q&A presentation and allows to assess your technical and communication skills.
Then, you spend another week as a paid 'freelancer' to do some actual work and interact with your team. At the end of the week you are assessed by your peers and are presented with an offer, in case all went 'well'.
The reality is certainly much grimmer (to me anyways).
That's really ok.
Now you've figured out what you really do care about (or at least a piece of it) - spending time with the people you love. In many ways that's a far healthier perspective to have on life.
So keep going to work and doing your job because, hey, you've got to pay the rent. If the only reason you're at this particular job is the company's mission, then start looking around for a place that pays more or has a better culture, or where you can learn more interesting things.
And in the meantime, keep doing a good job simply because you take pride in your work, and most hiring managers outside the weird "have to be passionate" startup scene will be glad to hire you.
Not to knock TV, but it is an escape. I know someone who used to get jealous of rocks(they get to just lay there). If you heal the burnout, you should start to feel your passions and motivations come back.
I've reached that stage at 23, after about a year into my first job. Since then, I've been working for the money.
Reflecting on why I first got into software development (SaaS for me). Am I still achieving those things? If not, I probably won't enjoy what I'm doing.
I have seen teams become more disengaged when everything comes top down, even if the top is a team lead. I have found success in empowering myself and others to look for things that they believe are important and work that into their schedule. I would never hold this against anyone as long as they don't go totally rouge.
Tldr;Reflect on yourself, your goals, and where you want to be. Are you there or not? If not, it doesn't mean your job is necessarily at fault. Sometimes the only person who will get you out of a rut is you.
You could look at how you viewed the startup when you joined. What excited you, how you felt that you were contributing to the vision. Then look at how and when that changed. Did the management change? Were a couple of unrealistic deadlines missed? After 4 years have you had a salary increase? Is it the still the same product or have there been pivots? Has the market not materialised? How about further investment? In 4 years things have probably changed and maybe you have ignored the signs that are now troubling you subconsciously.
You can either come to terms with working on problems that motivate you sufficiently or you can hit the road and find your happiness elsewhere.
I have felt burn out three times in my career. In each case, I had to disassociate myself from obligations for an indeterminate amount of time. It was essentially a responsibility "detox". When faced with the prospects of no obligations ... which ultimately lead to the prospect of utter boredom ... did new opportunities and enthusiasm to pursue them materialize.
The last phase of burn out allowed me to quit Oracle for which I was an exec for a (very) short tenure, begin foolishly day trading, and ultimately lead me to investing and market opportunities that lead to the creation of Codenvy in 2012.
Burn out caused Codenvy to exist.
Lately, I've been doing a lot more meditation. I'm trying to be as regular as possible with (at least 2x a day). It's helped quite a bit with feeling overwhelmed which is the biggest contributor to that sense of apathy.
The other thing I've been doing is working on a framework for creating a good work/life balance. As a side project, I'm trying to build a small desktop app that makes it easy to follow the framework. Ideally though, the goal is be presence when I'm working and focus wholly on that work. When I'm not "at work" to turn that off. Not having that off switch is something that seriously exhausts me.
If you don't eventually get inspired to do something else, take a long break, possibly quitting, and embrace the emptiness even more. Eventually, something should catch your heart.
Be open to totally new directions.
Also, 4 years is typically the time to start moving on from a company that you are not going to stay for a long, long time.
> But lately I just don't give a shit about bugs or features we're building.
Welcome to the world of adults.
We work to live, we don't live to work ;)
It's alright, you can have a job in the day and be with your girlfriend the rest of the time.
Do they allow open source? If they do you could always start something that benefits them but under your own git account. It benefits you in terms of marketing yourself and your skills.
If being fully engaged at work is something you need to be happy, then maybe a change is in order. Otherwise, as you said you're not doing insane amounts of overtime, simply having a job that you don't hate and pays the bills might be enough. Ultimately you have to ask yourself what you want from your job.
Feeling the same pain with you right now.
I think you should apply to other jobs, just to see what your options are.
Once that investment allows you to withdraw 1-2k per month, quit your job.
You will now be paid enough money to fulfill the costs of sitting at home and watching TV with your girlfriend indefinitely.
No need to worry about working, products, bugs, etc :)
If you still feel the same, then stop working at a startup and work at some established company.
The dirty secret is that most of us are building software that doesn't really need to exist. It's hard to stay super-passionate about it, especially as you get older and more experienced.
When I was younger, I couldn't really push hard and work really hard unless I believed in the mission. I had to have that passion fuel to do it. It's fun, it feels good when it's there, it's like being in love in a way. And I'd do 'heroic' stuff because of it. I've found, personally, that that passion fuel is more like adrenaline than actual fuel though. It tends to dry up after a while; I think this is do to life changes, if you're in a relationship or having a child or something like that, those things just dramatically outweigh "building some app" or learning some new tech. At this point in my life (married with a couple kids) I get buzzed about startup opportunities and stuff, but I don't think there is any code that will come close to what my family brings me in terms of that passion. You've made it 4 years so I'd guess you're not running on passion alone.
For a while, I tried to be really unemotional about work, I just wanted to be a professional and do that job. Honestly, I can do this, I know some people cannot. I enjoy making software, regardless of the project I can usually find aspects to enjoy and just enjoy the craft. You really don't need to give a shit to fix some bugs, at least I don't, they pay me and I do the work. That too tends to result in my relative unhappiness after a while; I feel like certain creative aspects aren't being honored if I'm just doing the job; also when the rest of my life is in disorder it's more difficult to find any satisfaction in just being a professional.
It's going to sound cliche but it's really all about balance, at least for me it is. I try to place the passion more directly in the craft and technology. I'm passionate about building great teams. I'm passionate about working with those teams as a team, I really love seeing others grow. I like my current company's mission and it's something I can get really passionate about but I try not to get too bent about the details and specifics. Something else which in ways feels bad to say and it took me a while to really admit it: I'm kind of passionate about financial successes, my company is making money and that cures a lot of the little things for me. I've worked on my priorities a lot and I know that I'd rather be part of something that we build in to success rather than having my product vision completely realized and fail. Boot strapping a start up can be remarkably emotional, you work close with people, you get close, product passions rather than craft passions can make it really difficult. You want some passion and you want to be a pro at the same time and you want to balance those things and it's probably a life long learning exercise to continually tweak them.
Take a real vacation, turn off your phones and devices, spend some time away from it all and then look at it again.
All that satellite imagery? Every single pixel, from every single satellite fly-over / observation, all interpreted / post-processed / calculated into actual meaningful data via Pythagorean theorem.
E.g. every data point processed for "incident angle" of the observation platform above parcel/pixel observed. Basics @
<< edit to format >>
I got into a debate with some fellow backpackers a couple years ago about how to measure total distance traveled when hiking in mountainous areas. They all said they could measure the total distance using the Pythagoras theorem but they were only measuring the distances of the angled sides of a triangle, not both the forward and up/down, which you have to traverse in the real world.
When I explained that they were only measuring the distance in a 2D world but since we live in a 3D world we need to measure both forward and up and down to get total distance traveled they were all in agreement that I couldn't be more wrong.
When I tried pointing out that this is an exercise in the physics of spatial dimensions (3D), and not a 2D flat plane, they couldn't (or more aptly put, refused) to grasp the difference.
I think this approach might be another good way to illustrate using it because while they all understood how it worked they didn't have a clue about when to apply it, which your example perfectly illustrates.
- Look at the mod queue, replying to any mod mail we've received and any posts that have been automatically removed by our AutoModerator script.
- Have a peak at the new posts, on any one day my sub-reddits only get a couple of hundred, so it takes a couple of minutes for a cursory glance.
- List all reported messages and see whether they're acceptable.
Then, because most of the sub-reddits I'm currently contributing to are about projects I'm working on, I generally answer any questions people have or problems they're facing.
However, this is just for me. It really truly depends on the type of sub-reddit you're looking to moderate as to what work you're required to do.
More generally, I think the moderators' main job is to set the mission/rules of the subreddit, and enforce them. Subreddits with weak enforcement rarely thrive as they devolve into the least common denominator.
CFS is interesting because:
a) Patients' symptoms appear to fluctuate "randomly" but are actually typically a complex function of genetics, blood markers, exercise, diet, medication and other factors.
b) There is considerable low-hanging fruit for pattern recognition, since despite the prevalence of the disease almost nobody has done serious ML work in this space.
c) Huge market opportunity - prevalence is comparable to HIV, and specialists often cite CFS as causing more disability  .
 Dr. Daniel Peterson (Introduction to Research and Clinical Conference, Fort Lauderdale, Florida, October 1994; published in JCFS 1995:1:3-4:123-125)
I foresee in less than ten years we will have a doctor in our pockets. No, it won't cure us and it won't replace a doctor, but it will give us all the information we need to have a 99% certainty of our condition.
Second batch for animals and their conditions.
Third batch, agriculture. Take a pic of a plant and tell me all the info, fertilizers, cultivation, etc, bonus for pest id and treatment.
Pocket computers should be able to diagnose every living creature.
- If the datasets are imaging, there should be enough per class for typical ML techniques. Otherwise you just get people over-fitting models on sets of 500 images and the illusion of progress.
- I'm quite happy with the Kaggle datasets generally, but why do others make consuming data so difficult. Heck, if we've already received the data, lets just take it the last mile and make it consumable with obvious labeling, standard formats, etc. This is such a pet peeve of mine that -- if you need help taking datasets to the last mile -- i'm volunteering, ask me to help make it presentable. Ideally it should be pull-able via curl/etc, unzippable and be able to get into a pipeline w/o manual effort.
Here is what you really want: Large amounts of curated/quality controlled data with ground truth that you can aggregate & share. Preferably with multiple studies and time points and/or followup. That is stated in rough order of difficulty to acquire.
Here is what you typically get fed into an learning pipeline: 1-2 orders of magnitude too small, with all kinds of noise, and no truth data(i.e. at best a bad proxy).
Hand-waving about unsupervised learning won't solve many of the really difficult problems (although it has uses, obviously). Neither will hand-waving about transfer learning. In some areas most retrospective data sets will never be really available because of consenting issues. QA is hard - the sheer variability of clinical systems in the field, not to mention protocol and practice differences, is often astonishing.
So where does that leave us? To make a real dent fast I suspect you need to focus on data availability, not problem. Ask the question:
What are the fastest path(s) to collecting large volumes of clinically representative data with some QA in place, consented for the ways we want to use it, and with real clinical truth or a decent proxy we can get at in an automated or semi-automated fashion? 1000 Bonus points if real outcome data will be available in future.
Note that most of this data is population level epidemiologic and administrative stuff, not the detailed biomedical measurements I see most people requesting - but I promise you there's some really interesting things that can be done with it nonetheless!
We will use NLP and AI to provide structured data from unstructured medical data (encounter notes, etc...) stored in the EHR for both analysis and integration. For example, one of our partners right now wants to integrate directly into our EHR in order to run computer vision algorithms on top of uploaded eye exam images in order to help diagnose eye diseases. We give them access to the eye image and other patient data, including the encounter, diagnoses, etc. After they have trained their algorithms, we then allow them to hook directly into the encounter workflow to send alerts live to the doctors during the appointment. We want to be a platform to help other startups and researchers connect with medical data both for analysis and also to help make a meaningful impact directly to doctors' workflows and patient care.
We would love to help out and/or learn about any use-cases that others might have requiring medical data. If you would like medical data or would want to integrate directly into a doctors' workflow in their EHRs based on NLP/AI hooks, we would love to hear from you. You can reach out to me directly at firstname.lastname@example.org
Right now we have no way of determining which interactions lead to which conditions, so we generalise based on the 3 inputs independently, when in reality it is perfectly normal to eat more when doing lots of exercise, or need doctor visits when doing exercise with inadequate nutrition.
Given you have surgeon [x] what are odds of a successful surgery with [x]. THIS is the guarded secret -- yet the most valuable.
If you have medical data (or want to be a cofounder) please email me :ransom1538 at gmail.com -- a prototype: https://www.opendoctor.io to find out data to this very question.
It could even offer suggestions like "Spend $200 more and recover on a island paradise!"
If globalism is good for low wage workers it certainly should be good for the medical profession.
In my opinion, large datasets testing wide spectrums of hormones in a large population, tagged with any diagnosed endocrinological condition would be extremely valuable. I bet with this information, we could learn a lot without conducting actual physical studies, by simply sectioning the data appropriately.
I'm not a doctor though, so I don't know exactly what would need to be recorded, but having dealt with bizarre endocrine disorders that doctors don't really have any answers to, my gut feeling is that such a data set would be incredibly useful.
Datasets are limited and expansion with AI would be huge.
One specific application - determining cost effectiveness of placing tourniquets in public places - much like the idea of having defibrillators at the mall. And funding community training, see the "Stop the Bleed" campaign.
And frequency by state, especially in the Western States where it is under-diagnosed.
What I would suggest to be maximally useful would be to focus on physiological data: EKG/ECG, EMG, glucose, SPO2, maybe various blood work counts.
All of those are data that are both well-understood and are thrown away regularly, and that if fed into a computer with modern ML methods we could maybe see some really cool stuff.
I'd suggest staying away from unstructured data and things that are primarily of interest to only the business side of healthcare--insurance figures, billing codes, EMR/EHR shit.
If you really wanted to get in there, putting up a minimal and standardized format for representing labs and medications would go a looooong way.
The problem in healthcare isn't the medical stuff--it's that people get bogged down in the inefficiencies of the system and zoom off solving problems that are removed from the immediate task of "what the fuck is wrong with this patient from the instruments I have at hand?"
While it isn't my research project, I've been trying to use computer vision and some naive AI to identify early breast cancer lesions in images from mouse tissue with mixed success, but it's something that can be very much accelerated with a large human dataset with outcomes.
(If you work in the field and what to help/hire me with/for something like this, kindly send a message to hn AT naj-p.com)
There are understandably some ethical guidelines that need to be worked for this sort of thing, but seeing as their are public repositories of not-so-dissimilar information (e.g. mammograms), it should be workable.
The data probably contains a number of symptoms or measurements (bloodwork), and a diagnosis by a doctor.
I can see how you can train a deep-learning model for that.
What if the patient is prescribed medication. Is the condition of the patient over time (after giving the medication) tracked by doctors?
Personally, I have found that once a doctor prescribes me some medication, he never asks me how things are going (except maybe once). So how accurate can the data be?
Patient history would help too. (I know there's HIPPA to comply with, but as much as we can get can help train better classifiers.)
Would be exciting/somewhat disruptive if YC democratized access to it.
Then those with Diseases or conditions.
Anything geospatial would be superb. Disease transmission for example.
It's the gold standard IMO for an intro to React
Besides all the tutorials and articles, my list also has a section on "Community Resources", which points to other useful lists, newsletters, and more: https://github.com/markerikson/react-redux-links/blob/master... .
I'll also specifically highlight the Reactiflux community on Discord. It's a great place to ask questions, chat, and learn about React and related technologies. The invite link is at https://www.reactiflux.com .
These also helped to understand specific React concepts:https://medium.com/@dan_abramov/smart-and-dumb-components-7c...https://medium.com/@dan_abramov/mixins-are-dead-long-live-hi...
Related tech but also helped me learn React:https://brigade.engineering/what-is-the-flux-application-arc...https://egghead.io/courses/getting-started-with-redux
-  https://www.robinwieruch.de/the-road-to-learn-react/
Also find some resources here: http://codecondo.com/reactjs-resources/
Just my 2c.
Go with Django. Python is a more productive and clear language. Django is excellent at ensuring an upgrade path, provides great docs.
If you have well defined services, it sounds like you do, then make them apis. You could have separate Djano apps or use Flask, a great micro framework.
Overall my experience with Python and the community for building web apps has been great.
* Dependency management is way easier with Python, so upgrading and deploying is less error prone
* Django scales really well thanks to excellent caching with e.g. Redis. Adding caching to views (pages) is child's play, and super easy to debug/purge.
* Speaking about scaling, writing a REST API or consuming a REST API is much easier in Django (django-rest-framework and requests, respectively)
* The Django ecosystem is great. If you have a problem you want to solve, someone has probably done it already and published a package
* The Drupal middleware is awful compared to Django middleware
* The Django admin interface is amazing , and at least as good as any off-the-shelf CMS. It does permissions and custom actions. Your sales team needs to upload a CSV file of suppliers? No problem, just add it right in there.
* Debugging. The Django shell and pdb/ipdb are great tools.
Finally, thanks to some non-obvious pitfalls in PHP, you have to take more care when writing PHP to make it secure. Apart from the flame-wars, the only practical issue is when hiring. You need to make sure you hire PHP candidates who know the language inside out. With Django/Python, you can be a bit more lax - there's far fewer non-obvious pitfalls, especially with the templating system and ORM of Django. It also takes devs less time to do a thorough code review. IMO, it's easier to grow a Django team.
My advice to you is to use Drupal because you can get far fast. Use lean methodology (lean canvas etc.) and get your MVP out as fast as possible and start making money. Don't try and build lots of functionality that you don't know works, and by works I mean makes a profit.
Watch GaryV, as he says - the market is always right. Focus on selling cars or vin checking or whatever it is you're doing, nobody apart from techies give a hoot what your site is built on. Some of the biggest sites in the world are built on PHP, e.g. Facebook. Some of the biggest sites are built on Drupal, e.g. Weather.com (the most personalised site as every visitor sees a different output, and delivers data to mobile apps).
Your code won't help your business succeed, selling will.
We found Drupal's hooking system a pain to work with and it made debugging incredibly difficult.
We were using Drupal 6 at the time, not sure what has improved since then.
* A framework works at a lower level compared to a CMS. So if you have lot of customizations to be done which is not limited to how it looks - a framework is better* If most of the work needed is to change how a site looks and content only, a CMS might be a good fit* From your point it looks like there is a need for a strong platform* Which language are you programmers comfortable with (if you already have a tech team)?
For the company I was working in earlier, Drupal 7 was being used.
For my own startup I am using Django.
Drupal or Django? A Guide for Decision Makershttp://blog.birdhouse.org/2009/11/11/drupal-or-django/
imho ... i worked with both of them :)
* drupal is a really nice, secure and extensible portal-system which can be enhanced with plugins written in php
* python-django is a web-framework based on common design-patterns
whats better for you!? it depends what you want to achive, but i think for a startup - and a maximum of flexibility + scalability - i would prefere python-django.
why? as a "general" webframework its more flexible than the portal/plugin approach of drupal + i would prefer python over php.
As time progressed, we added voting, rss aggregation, quizzes, and started extending views with URL arguments, relationships, fancy combinations of filters, login was via OAuth, and we were at 7.x by now. It was feeling unwieldy. Always aggressive in theming, what we wanted to do was taking an increasing amount of time, both in function and appearance.
A new project came along, and I thought, eh, Drupal can do that. And it surely could, seeing a nail, Drupal is a hammer that can do anything.
But it does it in a way where maintenance, or getting things exactly the way you want become incredibly time-consuming, and the code is a maze unless you're in it 24/7, and we had chosen Drupal not to be in code 24/7.
My background is not as a developer, but finance where using technology is an important skill. I can Bash, Ruby, JS, VBA/.Net, Python, R, and a few others to varying levels.
I shuddered a little when seriously considering Drupal as a solution to this new problem, mapping out the edge cases and exactly how many modules were needed (60+).
I ended up taking a week off learning NodeJS and using hapi as a framework. Just because I wanted a fresh start, not that I was enthralled with Node, but as so many others were using it, surely something was there.
This freshness gave some time to think about what I actually wanted, and to back-track from there, focusing on simplicity and well, simplicity. How to make everything as simple as possible, which is what a high level code framework offers.
You don't sound that technical, and I don't have experience with Django, however I recommend taking a code-approach, even if you don't want or need to be sitting in code, as you'll have access to and know how everything really works, not a pre-written GUI one where you're clutching at straws. Most web challenges are about the idea, the code implementation for what you sound like your doing, at an initial stage, is probably that not complex, and if you have success with the core idea, get technical talent onboard to optimise or do the things you find too hard.
A word of warning, however: Focus on making things simple. That will stop you going into a rabbit warren filled with code spaghetti.
Drupal is an excellent tool for publishing, and accommodates some complex publishing workflows that need multiple authors or layers of editing or approval. Core modules like comments or maps compliment this, but that's it. If your core need is greater than publishing (I suppose marketplace is publishing, and there are eCommerce modules), Drupal is a quick solution, but if your long-term roadmap means doing something remotely unique technically or in UX, roll your own using a framework to prevent re-inventing the wheel (and ending up with security nightmares). Bring on technical talent at an early stage when your idea seems proved. If you have funding, do this right away.
I've not lived in London, but grew up in Montreal and lived in Toronto. I've been fortunate enough to visit many US and European metropolitan areas. In saying all that, I really find Ottawa to be a nice-sized small city. It's a government town for sure, but there is also a lot of Aero & Defence (where I ended up) as well as high tech (Shopify, Apple, Amazon, QNX/Ford's new self driving car centre). It's big enough to have pro sports teams, but small enough that you can be in the country in 20 minutes if you want to. No 16 lanes of highway anywhere in sight, either.
At the end of the day though, try not to worry about it too much. As others have said in this thread, the first job will define a path in life, but so will the second job, or that next person you talk to on the bus, it's all a big random walk. You are fortunate to have two interesting opportunities in front of you, so have fun and good luck.
Having said that, the "modern" stack is definitely safer. The C++ market is great for really awesome, technical people, but there's not that many offers for people who are average. With modern tech it's the other way around - it is generally easier to use so there's less need for awesome people (which can make it boring if you are very talented), but it's very safe (in terms of job security) for average people. Also, in the C/C++ world you'll be competing against guys who've been building up their domain knowledge for 20-30 years...
My story is that I went with the safe side and my salary and working conditions are excellent now. The jobs themselves are completely uninspiring though. On the other hand, I'm not sure if I had the mental firepower to be a really great c++ programmer (say, senior engine programmer at a game company), as I get tired rather easily. So, even now (10+ years into my career), I'm not sure if I made the right call.
Re: London, it's huge, feels old/dirty (when I touch a building's wall, I wonder how many hundreds of people and dogs have pissed on that spot in the past), crowded and absurdly expensive. On the other hand, it is great for job hopping, so it might be wise to stay there in the formative years of your career.
London is huge, so is the market. So is the competition :-)
I have never been to Ottawa. I have moved to London 7 years ago, feel free to ask more questions if you have some.
I've been seeing highly-competent orgs move towards focusing more energy on making code "living documentation" by factoring it in ways that make it easier to understand, and where that is not sufficient, using a lightweight form of high-level documentation such as a wiki. I think this fits better into the lifecycle of most software projects these days.
For API documentation, if necessary (there are people actually integrating with your API), auto-generated documentation is the way to go, IMO. There are a few ways to do this, I can elaborate on a few if you're interested.
Going over interchange should save you some money, there are a bevy of ISOs out there to choose from, and virtually all of them will be able to replace Stripe.
Restricting what your customers can pay with, say only allowing Debit, or not accepting American Express, can also allow you to cut costs, albeit it reflects poorly on your business.
1 - http://fortune.com/2015/10/30/amazon-register-shutting-down/
You could also choose to only take debit cards(and exclude credit cards). I know a local grocery store that does this, and it can be far cheaper if you use the right payment processor.
The course is free and takes 6 weeks long, and is very interesting if you had never dwelled too deep into security or crypto. There's also a new cryptography class that will be available in September of 2017 - https://www.coursera.org/learn/crypto2.
For pretty much any crypto task that a "run-of-the-mill programmer" is likely to run into, they've got you covered.
Secret key encryption: https://download.libsodium.org/doc/secret-key_cryptography/a...
Password hashing: https://download.libsodium.org/doc/password_hashing/
Matthew Green's blog, A Few Thoughts on Cryptographic Engineering , has a wealth of interesting posts that are often aimed at explaining cryptography to a "technical but non-cryptographer" audience, and tend to be motivated by recent events in security/cryptography news.
: https://www.amazon.com/Cryptography-Engineering-Principles-P...: https://blog.cryptographyengineering.com/
Start at https://www.owasp.org/index.php/Password_Storage_Cheat_Sheet
The OWASP guidance is OK for a quick access to best practices, but insufficient for rigorous learning.
Cryptography takes time to digest the fundamentals and recognise how new concepts are both beneficial and, vitally, disadvantageous; sadly, there is no cheat sheet or quick fix.
Source: computer security PhD student.
The code examples are in Python, but I plan to add pages in other languages.
I should probably update it to use one of the more modern algos, but the availability of good bcrypt libraries makes it solid advice still.
* PyCon Crypto 101 - https://www.crypto101.io/ (and if you use Python, please use Cryptography library for encryption/decryption please, Python built-in provides sha and hmac already though, and please adopt your framework's security implementation whenever possible).
* Mozilla Web Security Guidelines - https://wiki.mozilla.org/Security/Guidelines/Web_Security
* Mozilla Secure Coding Guideline - https://wiki.mozilla.org/WebAppSec/Secure_Coding_Guidelines
* Mozilla Server Side TLS - https://wiki.mozilla.org/Security/Server_Side_TLS
* Mozilla Intro to Cryptography (slide: https://april.github.io/crypto-presentation video: https://www.youtube.com/watch?v=bg32spD2mB0)
* Mozilla Web wiki - https://developer.mozilla.org/en-US/docs/Web (understand CORS, Cookies, CSP, etc)
* Google's course on security - https://google-gruyere.appspot.com/ (original course page has been taken down by Google already)
* The Web Application Hacker's Handbook
* The Tangled Web: A Guide to Securing Modern Web Applications (written by the famous Micha Zalewski working at Google, and lately known for developing the American Fuzzy Loop AFL which has been used for uncovering many new CVE bugs).
* Hacking: The Next Generation
* Securing DevOps (to be released soon)
* USENIX - https://www.usenix.org/ (tons of free high quality conference talks, I like USENIX over ACM)
* Real World Crypto
* Go find bug bounty program out there, many well-written posts how one discovered bugs
* Follow a bunch of security engineers / security-minded folks on Twitter (e.g. @matthew_d_green would be a good start)
OWASP is a great reference, you read it as an index page. But like others have pointed out, the Wiki is often outdated, but concepts almost always remain the same. Use multiple resources before implementing a solution, and never just copy and paste solution posted by others on Stackoverflow. Sorry for so many Mozilla stuff definitely there's some bias from me but I trust folks running the sec team there.
I see a lot of "don't use" but I don't see any "do use" for that case.
Underlock is a small Ruby library that helps with Encrypting/Decrypting of files and other data.