hacker news with inline top comments    .. more ..    17 Feb 2017 Best
home   ask   best   5 months ago   
I started a one-man biz that's beating VC-backed startups distrokid.com
1123 points by pud  1 day ago   206 comments top 59
aresant 1 day ago 9 replies      
@Pud can undeniably build products that are laser focused and useful.

And his minimalist approach to company building is epic and I am a fan.

But it's sort of disingenuous to play David Vs. Goliath here like his line about launching:

"I figured only a small number of smart people would somehow find us among the masses."

He's just like the rest of us - just build a great product and be lean guys!

But oh wait he launched DistroKid on the back of the HUNDREDS OF THOUSANDS of registered musicians already using his other site - Fandalism - founded years before.(1)

EDIT - why I am a fan of pud is in the children to this comment he brings a data-gun to a knife fight and sorts out my impression w some facts.

(1) https://techcrunch.com/2012/04/06/philip-kaplan-fandalism/

brilliantcode 1 day ago 7 replies      
I love articles like this it's inspiring...but it really doesn't offer any new insights other than the fact that

Every one man success story on HN was largely possible due to existing audience from another entity that took years to grow

It's pretty disheartening once you read the article and it boils down to this very requirement: You Need An Audience First.

I'd like more insights and details in how lone entrepreneurs of HN were able to grow their audience.

Because when Step 1 is have a big enough audience the rest of the steps are pretty much a no brainer. It's this step that most of us are struggling with.

I've spent years in seclusion writing a SaaS that ultimately resulted less than the annual wage of a North Korean factory manager and it was rough both physically and mentally but it all boiled down to the fact that VC funded startups were able to do huge PR and buy ads, write blog posts, increase Adwords bid prices etc while I was not.

So yeah, you can succeed with a one-man biz outfit but not without an audience. It makes sense why Linkedin is valuable, why Pinterest, Whatsapp, Oculus, Twitch gets bought out even without revenues.

jtth 1 day ago 3 replies      
To be fair, any company making a net profit is doing better than most VC-backed startups.
inopinatus 1 day ago 1 reply      
Really resonated with me, also being a micro-startup with a minimalist customer-centric philosophy, albeit in a completely different market.

Steps to reproduce:

1. Identify a poorly served professional segment where the business processes are not the primary task,

2. Automate their processes with vision and passion and empathy for the customer,

3. Make fans for life. Maybe also profit.

Minimum requirements to execute: at least one seed customer; wifi; coffee; time.

Rock on pud.

econner 1 day ago 1 reply      
Met @pud once while he was CEO of Blippy. Very nice guy and great to talk to. I also absolutely love his post: "Why Must You Laugh at My Back End" (http://pud.com/post/9582597828/why-must-you-laugh-at-my-back...). He's built lots of crazy impressive stuff by himself and DistroKid is no different.
tom_pulo 1 day ago 0 replies      
I very much like reading this type of stories where founders candidly talk about their business, how well they are doing, what did it take to get where they are (remove a good chunk of the usual BS, add some numbers). Indie Hackers is a great source (https://www.indiehackers.com) for this type of stories.

I have a feeling that this is the dream of a lot of people on HN. Having an easy to run business with a couple of people, without crazy competition, not needing to go sell it to VCs, and making a good amount of money while running it. Not judging, just observing.

myfonj 17 hours ago 0 replies      
Minor complaint / remark: is there complete list of all stores and services that DistroKid supports (directly, or better even indirectly) somewhere?

All I was able to find were murky statements about 'dozen mayor services' and "150+ others" [0]. That page links "MediaNet Customers" page [0a] that displays 24 logos and links [0b] that displays list of 286 "MediaNet Content Partners". Is that it?

I was particularly interested about Bandcamp and all I've found that DistroKid mentions Bandcamp [1] and Bandcamp mentions DistrokKid [2] but no proof they really talk to each other.

[0] https://distrokid.desk.com/customer/portal/articles/1276117-... [0a] http://www.mndigital.com/about-us/customers.html[0b] http://www.mndigital.com/about-us/content-partners.html[1] https://distrokid.desk.com/customer/portal/articles/1601235[2] https://bandcamp.com/help/selling

nagvx 1 day ago 8 replies      
Please correct me if I'm wrong - isn't this a terrible business decision? It took a small team a year to make a product that is apparently taking the market by storm. Their secret sauce is apparently not some special algorithm, but common-sense automation. If the barrier to entry is actually this low, and they advertise the fact widely enough, they will soon be joined by competitors trying to do the same. They may be ahead of the pack, but they lose their selling point and their advantage faster than they would have done otherwise.

Why give away your hand like this?

b_emery 1 day ago 1 reply      
So artist pay $20 per year, and to quote the article "100,000+ artists", he's looking at $2,000,000 gross per year? Three employees. Amazing
startupdiscuss 10 hours ago 1 reply      

There are many people who want to know about your tech stack. I think the thinking goes: with one developer, it must be an insanely productive stack.


is old, do you still use the same set up?

georgespencer 1 day ago 1 reply      
I love what you're doing! But have to observe disparity between:

> beating VC-backed startups


> DistroKid intentionally has a small team and no investors. Were here to make the world a better place for musiciansnot to make billions from them. Wed make ludicrously more money if we charged what the other distributors do.

It sounds like you're coexisting quite nicely with different objectives.

urs2102 1 day ago 1 reply      
Ah yes! The CFML guy - @pud (that's how I always remember him from that one blog post he wrote)!

This is really cool, and love hearing about stuff like this. I love the idea of running a small team and scaling a product which doesn't have a huge overhead to make something which beats out the current market by just doing a few things better. Hats off to you man. Great stuff!

quadcore 18 hours ago 1 reply      
I started a one-man biz that's beating VC-backed startups

Click-bait title in my opinion. It's not because you don't need VCs to satisfy your customers that VCs are bad. They are good when you need them to satisfy your customers. Now distrokid-competitors' problem is not that they have VCs money, it's that they "hired" bad ones. Now, I grant you, good VCs for startups, those who gonna understand whats good and not for your customers may be rare... But look at google and facebook, they did right by taking VCs' money don't you think.

ajaimk 19 hours ago 0 replies      
Fandalism launch in January 2012. Distrokid in mid-2013. Considering it took @Pud near 4 years to reach these numbers with DistroKid, the year and a half of building a user base with Fandalism is not that big a deal.

This basically would be the same as all the freemium companies out there that offer everything they do for free for a while till they build their product enough to start charging for it later.

My favorite line is in the original DistroKid TC article where @Pud says the goal wasn't to make money with this but to get more people to the social network.

leroy_masochist 1 day ago 0 replies      
@Pud, would be great to hear your take on what it has been like dealing with the record companies. According to YC:

> Because the alumni network is so large and tightly knit, investors or companies who try to maltreat a YC-funded startup can usually be made to stop. [--> footnote] Except for the record labels, which are effectively a rogue state with nuclear weapons. There is nothing we or anyone else can do to protect you from them, except warn you not to start startups that touch label music. [0]

Your service does various tasks such as cover song licensing which are record label-facing. What has that been like?

And, congratulations on getting engaged.

[0]: http://www.ycombinator.com/atyc/#n3

erichocean 23 hours ago 2 replies      
https://distrokid.com/ is a near-perfect landing page for this kind of service. Well done!

Anyone know if there is an equivalent for independent films?

omarchowdhury 1 day ago 4 replies      
The entire thing is built on the ecosystems created by companies that were all venture backed at one point, though.
rmason 1 day ago 2 replies      
PUD has a well deserved reputation among solo entrepreneurs. I think Distro Kid is his fifth startup.

All but one of his companies (the one unsuccessful one) were powered by CFML.

bootload 20 hours ago 0 replies      
Having quickly looked at the audience for DistroKid, then see what happens with artists by major labels, no wonder the product is sought after by upcoming artists. [0] Bandcamp is another company that uses technology to squeeze out the middle man. [1] The first page is really well designed. You can easily see what the product does, "let artists make music, then distribute it for a fixed cost".


[0] "52 Ways to Screw an Artist, by Warner Bros. Records" https://news.ycombinator.com/item?id=13648245

[1] "Is Bandcamp the Holy Grail of Online Record Stores?"https://news.ycombinator.com/item?id=12324350

dyeje 12 hours ago 0 replies      
Eh, I go with CDBaby because the math doesn't work out for me as a small artist. I'd rather pay $50 and just have my album up forever. 9% doesn't matter much to me because I'm making peanuts anyway. Maybe if I get bigger I'll consider switching over.

It would be really slick if DistroKid had some kind of conversion process for CDBaby / Tunecore customers.

sk24iam 9 hours ago 0 replies      
This is very motivating. I am also running a site in the indie music industry as a one-man biz. (http://www.synthshare.com) It has been a challenge to build up the community, I'm sure leveraging Fandalism helped you kickstart your growth. Do you have any advice about user acquisition? Recently I've been getting feedback from music production communities which has all been positive and converted to users; however, it sometimes feels like the site isn't growing as fast as it should be considering all the positive feedback I've received.
mcguire 10 hours ago 0 replies      
Hey Pud, since you are answering comments here, I have a question: what failures have you had?
drenvuk 1 day ago 1 reply      
This is pretty darn cool and the first I've heard of it. I thought there'd be a cheaper way to do what your competitors have and here you've done it. What your success also confirms is that reducing your workload with bots and cronjobs are one of the keys to succeeding with a small team.

How do you handle support?

How are you able to find infringing music?

no_wizard 1 day ago 0 replies      
I think it's great that you have found great success as a pipeline service.

I also think it may be disingenuous to suggest this can or should work for every type of business. While I happen to agree that services that simply do a pipeline, aggregation, or intermediary service are A) not something I feel VCs should usually spend their money on in great quantities if at all and B) are often the most ripe for disruption, I disagree that finding scaled efficiency for all types of businessses in this manner is possible.

I do however think it's wonderful you are promoting the more traditional idea of a business with this product at least which is the more canonical bootstrapping or self funding/ side job till you make it/ type thing that doesn't need tons of employees to be a good value for those that are a part of that business.

That's just my opinions though I'm glad it's been a success

tomclancy 1 day ago 0 replies      
I've been watching DistroKid since it spawned out of Fandalism. Awesome to see how successful it's become.

Just curious, what % of artists gross over $19.99 a year?

technologyvault 1 day ago 1 reply      
This is another example that it's a misconception that you have to have a big team and lots of cash to get to the next level

Stories like this hearken to people like Mark Zuckerberg and Instagram's Kevin Systrom and Mike Krieger.

Futurebot 1 day ago 0 replies      
Interesting related statistics:

"The blockbuster effect has been even more striking on the digital platforms that were supposed to demonstrate the benefits of the long tail. On iTunes or Amazon, the marginal cost of stocking another item is essentially zero, so supply has grown. But the rewards of this model have become increasingly skewed towards the hits. Anita Elberse, of the Harvard Business School, working with data from Nielsen, notes that in 2007, 91% of the 3.9m different music tracks sold in America notched up fewer than 100 sales, and 24% only one each. Just 36 best-selling tracks accounted for 7% of all sales. By last year the tail had become yet longer but even thinner: of 8.7m different tracks that sold at least one copy, 96% sold fewer than 100 copies and 40%3.5m songswere purchased just once. And that does not include the many songs on offer that have never sold a single copy. Spotify said in 2013 that of its 20m-strong song catalogue at the time, 80% had been playedin other words, the remaining 4m songs had generated no interest at all."


haywirez 21 hours ago 0 replies      
Hey, does (or can) Distrokid offer an API? I've been dying to find a distributor that would let me programmatically submit/replace releases. It would only be used for my label, no bulk stuff, reasonable per-call fees or adjusting balances on a pre-paid account would be fine. Sale statistics API - even better.
stuffedBelly 1 day ago 1 reply      
As a part-time musician, I absolutely love DistroKid.
72deluxe 17 hours ago 0 replies      
I am surprised that the development time of 1 year was considered a long time by him compared with his 1-2 month initial estimate.

If only projects I / we had been working on for 2 years became so successful!

6stringmerc 10 hours ago 0 replies      
...and please count me in among the users for the past 4 years who have seen the value, enjoyed the service, had a lot of fun playing with metrics, and feel inspired to continue my 100% rights ownership in music pursuit for 2017 and beyond.

Great product. Great support. Great example.

tedmiston 1 day ago 0 replies      
> By contrast, our competitors largely have millions in funding ... And theyre owned by venture capitalists and/or private equity firms who are banking on a large exit.

Disincentives of the startup model at work!

These days it feels like you if your startup doesn't have polarized outcomes you're "doing it wrong".

spir 16 hours ago 1 reply      
3 employees, wow! Sounds like it would have taken 300 employees twenty years ago. Reads like an advertisement for basic income.
kriro 17 hours ago 0 replies      
Previous connections aside this is still very inspiring. Scaling this well with only three employees (and resisting the urge to add more people just to be bigger) is an amazing feat. I'd be very interested in more details. How focused were you on being profitable and reinvesting profits vs. taking venture money? My guess is profit was one of the key metrics?
aurizon 1 day ago 0 replies      
I see the comments where you started running with a sea of clients who trusted you to give them a fair shake.I see so many of the others where the music makers suck hind tit.Looking at your competitors, you have a clear advantage in your lean and efficient structure. In time your method will spread and and drive many others out.The problem I see is labels tied to the big companies may have restrictive agreements that exclude you. Couple this with the wish of many people to feel they are complete when they have bought into spotify/Apple/etc.

In any event, I wish you well, live long and prosper

stryk 1 day ago 0 replies      
Is this a digital-only endeavor? I don't see any mention of physical product services such as pressing CDs & DVDs, printing covers/liner notes/etc. And what about marketing materials like posters, flyers and so on. There's still a market for that stuff, and it could be another revenue stream. Merch is also pretty big for some artists, you could explore getting t-shirts and hats and stuff printed. (just an idea, i've no idea if it's profitable, or more trouble than it might be worth)
cyberferret 1 day ago 0 replies      
Great to see this. Stoked to hear that it is the same guy who started Fandalism. I used to use Fandalism years ago to post my clips but just kind of stopped. I never got any emails or communication from them and just assumed that they faded away. Spurred by this article, I checked, and am amazed that it is still going strong. Must remember to get back to posting on there again.
omouse 10 hours ago 0 replies      
What's interesting to me is how he focused on doing small things and building up on them.
ejanus 19 hours ago 0 replies      
Great! I would like to tap your brains. But now you have captured/conquered the music world wouldn't it be wise to disrupt movies/films distributions?
sidcool 21 hours ago 1 reply      
>thought it would take 12 months to build DistroKid. Its just plugging into some APIs and moving files around, right?I underestimated.It took a year.

Wow. I would like to understand more about this.

mshenfield 1 day ago 0 replies      
It's awesome to hear about a swarm of bots replacing millions of dollars worth of labor overhead. Are you worried about the maintenance and QA at all? Is there anything about the problem that lends itself especially to automation?
BorisMelnik 22 hours ago 0 replies      
Would have been so much better if it was titled "YC startups"
pareshverma91 1 day ago 1 reply      
Does DistroKid track the usage/consumption details of the songs for artists to look at or analyze? If so, what is the probability that they will expand to also have a recommendation system for indie music based on these aggregated stats?
rrtwo 15 hours ago 0 replies      
How did you get the music industry companies to agree for you to directly upload files to them?
sidcool 21 hours ago 0 replies      
A very well written and inspiring post. Congratulations and good luck. Do you mind shedding some light on the tech stack you used for distrokid?
Animats 1 day ago 2 replies      
It's Craigslist for music. Low overhead. This could do to the music industry what Craiglist did to newspaper classified advertising.
RainManDetroit 22 hours ago 0 replies      
Good, there are better backers out there who will buy you out whole and let you keep your soul.
sddfd 18 hours ago 0 replies      
Great work! I'd like to see more startups like this.
drewman 8 hours ago 0 replies      
*one-person ftfy
curiousDog 1 day ago 1 reply      
I probably missed it in the article, but how does Distrokid make money?
tudorw 1 day ago 1 reply      
FC, oh boy, happy days :)
sandGorgon 1 day ago 1 reply      
You have dozens of bots, written by one dude ?

Really interested to know about your stack and your deployment methodology. Did you build a bit/async infrastructure because of scale issues...Or is it inherently a mental model that is easy to grok easily?

yresnob 1 day ago 2 replies      
Is there something like this but for books and poetry books??
kyasui 1 day ago 0 replies      
I use and love Distrokid - thanks for the product!
ommunist 15 hours ago 0 replies      
This is fantastic success story! Thank you for sharing. A schoolbook example of disruption howto.
jasonwilk 22 hours ago 0 replies      
Congrats Pud.
tomelders 1 day ago 0 replies      
a distrokid API would be pretty useful.
bwhites 16 hours ago 0 replies      
blunte 1 day ago 1 reply      
Hey, nothing wrong with self-promotion - it can take you places! Look at POTUS.
Introducing Cloud Spanner, a Global Database Service googleblog.com
1055 points by wwilson  2 days ago   428 comments top 56
ChuckMcM 2 days ago 10 replies      
Congratulations to the Spanner team for becoming part of the Google public cloud!

And for those wondering, this is why Oracle wants billions of dollars from Google for "Java Copyright Infringement" because the only growth market for Oracle right now is their hosted database service, and whoops Google has a better one now.

It will be interesting if Amazon and Microsoft choose to compete with Google on this service. If we get to the point where you have databases, compute, storage, and connectivity services from those three at equal scale, well that would be a lot of choice for the developers!

suprgeek 2 days ago 2 replies      
Really a CP system but with the Availability being five 9s or better (less than one failure in 10^6)

How:1)Hardware - Gobs and Gobs of Hardware and SRE experience

"Spanner is not running over the public Internet in fact, every Spanner packet flows only over Google-controlled routers and links (excluding any edge links to remote clients). Furthermore, each data center typically has at least three independent fibers connecting it to the private global network, thus ensuring path diversity for every pair of data centers. Similarly, there is redundancy of equipment and paths within a datacenter. Thus normally catastrophic events, such as cut fiber lines, do not lead to partitions or to outages."

2) Ninja 2PC

"Spanner uses two-phase commit (2PC) and strict two-phase locking to ensure isolation and strong consistency. 2PC has been called the anti-availability protocol [Hel16] because all members must be up for it to work. Spanner mitigates this by having each member be a Paxos group, thus ensuring each 2PC member is highly available even if some of its Paxos participants are down."

tedd4u 2 days ago 8 replies      
The team here at Quizlet did a lot of performance testing on Spanner with one of our MySQL workloads to see if it's an option for us. Here are the test results: https://quizlet.com/blog/quizlet-cloud-spanner
jedberg 2 days ago 9 replies      
This release shows the different philosophies of Google vs Amazon in an interesting way.

Google prefers building advanced systems that let you do things "the old way" but making them horizontally scalable.

Amazon prefers to acknowledge that network partitions exist and try to get you to do things "the new way" that deals with that failure case in the software instead of trying to hide it.

I'm not saying either system is better than the other, but doing it Google's way is certainly easier for Enterprises that want to make the move, and why Amazon is starting to break with tradition and release products that let you do things "the old way" while hiding the details in an abstraction.

I've always said that Google is technically better than AWS, but no one will ever know because they don't have a strong sales team to go and show people.

This release only solidifies that point.

richdougherty 2 days ago 1 reply      
Some interesting stuff in https://cloud.google.com/spanner/docs/whitepapers/SpannerAnd... about the social aspects of high availability.

1. Defining high availability in terms of how a system is used: "In turn, the real litmus test is whether or not users (that want their own service to be highly available) write the code to handle outage exceptions: if they havent written that code, then they are assuming high availability. Based on a large number of internal users of Spanner, we know that they assume Spanner ishighly available."

2. Ensuring that people don't become too dependent on high availability: "Starting in 2009, due to excess availability, Chubbys Site Reliability Engineers (SREs) started forcing periodic outages to ensure we continue to understand dependencies and the impact of Chubby failures."

I think 2 is really interesting. Netflix has Chaos Monkey to help address this (https://github.com/Netflix/SimianArmy/wiki/Chaos-Monkey). There's also a book called Foolproof (https://www.theguardian.com/books/2015/oct/12/foolproof-greg...) which talks about how perceived safety can lead to bigger disasters in lots of different areas: finance, driving, natural disasters, etc.

karlding 2 days ago 7 replies      
I wonder how this will affect adoption of CockroachDB [1], which was inspired by Spanner and supposedly an open source equivalent. I'd imagine that Spanner is a rather compelling choice, since they don't have to host it themselves. As far as I know, CockroachDB currently does not support providing CockroachDB as a service (but it is on their roadmap) [2].

[1] https://www.cockroachlabs.com/docs/frequently-asked-question...

[2] https://www.cockroachlabs.com/docs/frequently-asked-question...

sudhirj 2 days ago 2 replies      
For those trying to compare this with AWS Aurora, Aurora is more a regular database (MySQL / Postgres) engine with a custom data storage plugin that's AWS/ELB/SSD/EFS-aware. Because of this the database engine can make AWS specific decisions and optimizations that greatly boost performance. It supports master-master replication in the same region, master-slave across regions.

Global Spanner looks like a different beast, though. It looks like Google has configured a database for master-master(-master?) replication, across regions and even continents. They seem to be pulling it off by running only their own fiber, each master being a paxos cluster itself, GPS, atomic clocks and lot of other whiz-bangery.

cipherzero 2 days ago 1 reply      
The white paper is available here: http://static.googleusercontent.com/media/research.google.co...

for anyone interested

runeks 2 days ago 9 replies      
I wonder why they charge a minimum of $0.90 per node-hour when they offer VMs for as little as $0.008/hr. This is hugely useful even for single-person startups, so why charge a minimum of ~$8,000 per year?
xapata 2 days ago 1 reply      
Amazon likes to respond to Google with it's own price drops and product launches. It's telling that their announcements are orthogonal instead of direct competition with Spanner.

When Google announced Spanner back in 2012, I'm sure Amazon and Microsoft started teams to reproduce their own versions.

Spanner is not just software. The private network reduces partitions. GPS and atomic clocks for every machine help synchronize time globally. There won't be a Hadoop equivalent for Spanner, unless it includes the hardware spec.

emersonrsantos 2 days ago 1 reply      
Thomas Watson in 1943 amd his famous quote: I think there is a world market for about five computers".

If he was alive, he could say these computers are Google, Apple, Microsoft, Amazon and Facebook.

abalone 2 days ago 1 reply      
How does this compare to AWS Aurora in terms of pricing and performance?

With Aurora the basic instance is $48/month and they recommend at least two in separate zones for availability, so it's about $96/month minimum. Storage is $.10/GB and IO is $.20 per million requests. Data transfer starts at $.09/GB and the first GB is free.[1]

Spanner is a minimum of $650/mo (6X the Aurora minimum), storage is $.30/GB (3X), and data transfer starts at $.12/GB (1.3X).

Of course with Aurora you have to pick your instance size and bigger faster instances will cost more. Also there's the matter of multi-region replication, although it appears that aspect of Spanner is not priced out yet. So maybe as you scale the gap narrows, but it's interesting to price out the entry point for startups.

[1] https://aws.amazon.com/rds/aurora/

tabeth 2 days ago 2 replies      
Forgive my ignorance, but could someone explain in layman's terms in which situation this would be helpful? E.g. if I have 1TB of data would I use this? If I have 1GB with a growth rate of 25GB/daily would I use this?
Ajedi32 2 days ago 0 replies      
> Today, were excited to announce the public beta for Cloud Spanner, a globally distributed relational database service that lets customers have their cake and eat it too: ACID transactions and SQL semantics, without giving up horizontal scaling and high availability.

This sounds too good to be true. But it's Google, so maybe not. Time to start reading whitepapers...

jdcarr 2 days ago 0 replies      
Link to the actual OSDI paper (not the simpler whitepaper) https://static.googleusercontent.com/media/research.google.c...
kennethmac2000 2 days ago 1 reply      
Looks cool, but the pricing seems a bit non-cloud-native (or at least non-GCP-native).

"You are charged each hour for the maximum number of nodes that exist during that hour."

We've been educated by Google to consider per-minute, per-instance/node billing normal - and presumably all the arguments about why this is the right, pro-customer way to price GCE apply equally to Cloud Spanner.

bowmessage 2 days ago 0 replies      
elvinyung 2 days ago 1 reply      
While everyone is puzzling over how Spanner seems to be claiming to be CA, I would like to take this opportunity to bring up PACELC[1].

The idea is that the A-or-C choice in CAP only applies during network partitions, so it's not sufficient to describe a distributed system as either CP or AP. When the network is fine, the choice is between low latency and consistency.

In the case of Spanner, it chooses consistency over availability during network partitions, and consistency over low latency in the absence of partitions.

1: http://cs-www.cs.yale.edu/homes/dna/papers/abadi-pacelc.pdf

andy_ppp 2 days ago 3 replies      
> clients can do globally consistent reads across the entire database without locking

How is this possible across data centres? Does it send data everywhere at once?

Seems too good to be true of course but if it works and scales it might be worthwhile just not having to worry about your database scaling? Still I don't believe it ;-)

EDIT: further info...

> Spanner mitigates this by having each member be a Paxos group, thus ensuring each 2PC member is highly available even if some of its Paxos participants are down. Data is divided into groups that form the basic unit of placement and replication.

So it's SQL with Paxos that presumably never get's confused but during a partition will presumably not be consistent.

sebringj 1 day ago 1 reply      
I was like YES!!! Then I read for a single node it is 90 cents per hours then I was like NO!!! so absolute minimal cost for me is $648/month? I was hoping there was like a dev version. Maybe I didn't read the fine print?
the_cap_theorem 1 day ago 0 replies      
>> Remarkably, Cloud Spanner achieves this combination of features without violating the CAP Theorem.

This is the best weasel PR language I have seen in a long time.

Note that the sentence does not actually proclaim that they solved (the previously "unsolvable") problem of achieving distributed consensus with unreliable communication while maintaining partition tolerance and availability.

The blog only says they don't "violate" the CAP theorem -- whatever that means. So the statement is technically correct. Still the intention is obviously to mislead the casual reader (why else would you start the sentence with "Remarkably"?).

A litmus test: The same statement is true for MySQL - or _any other_ database in fact:

 >> "Remarkably, MySQL achieves this combination of features without violating the CAP theorem"
It's a bit like saying

 >> "Remarkably, MySQL is not a perpetuum mobile"

tamalsaha001 2 days ago 0 replies      
One thing to note is Spanner's transactions are different compared to what you get with a traditional RDBMS. See https://cloud.google.com/spanner/docs/transactions#ro_transa...

An example is the rows you get back from a query like "select * from T where x=a" can't be part of a RW transaction. I believe because they don't have the time-stamp associated with them. So, you have to re-read those rows via primary key inside a RW transaction to update them. This can be a surprise if you are coming from a traditional RDBMS background. If you are think about porting your app from MySQL/PostgreSQL to Spanner, it will be more than just updating query syntax.

Disclaimer: I used F1 (built on top of Spanner, https://research.google.com/pubs/pub41344.html) few years ago.

dhd415 2 days ago 1 reply      
Given that CockrochDB is based on Spanner and F1, this DBaaS sounds like it will compete directly with them.
wcdolphin 2 days ago 1 reply      
Is JSON data type support in the works? Seems to be a very commonly requested feature these days.
gonyea 2 days ago 0 replies      
I can vouch for Spanner: it's a badass piece of Google's infrastructure.
mankash666 2 days ago 0 replies      
While the product is compelling (acid compliant, horizontally scanning DB), it does seem expensive.

If you use 2 nodes/hour, Cost = (20.9) 24 * 31 = $1400/month not anointing for storage and network chargers.

nodesocket 2 days ago 0 replies      
Related, I wrote a blog post on the network latency between Google Compute Engine zones and regions. I'm assuming Cloud Spanner will still have these latencies once multi-region is deployed. Cross-zone latency on GCE is very good though.


BinaryIdiot 2 days ago 1 reply      
Oh this looks really compelling! Though I'm guessing this is targeted to companies? I'd love to use this for some personal projects but the pricing seems really high. Am I reading it right that a single node being used at least a tiny bit every hour is about $670 a month?

Maybe I'm misunderstanding how the pricing works here. Any clarification would be highly welcomed :)

pagade 2 days ago 3 replies      
What is TrueTime really? Are their Distributed Systems 'sharing a global clock'?
mallipeddi 1 day ago 0 replies      
Few questions from reading the docs:

1) How big can all the colocated data for a single primary key get before they don't fit within a split? Can I implement a GMail-like product where all the data for a single user resides within one split?

2) Is there a way to turn off external consistency and fall back to serializability? In return you get better write latencies. This is similar to what CockroachDB provides?

Beldur 1 day ago 0 replies      
Here is a very interesting video from 2013 of Martin Schoenert explaining the Google Spanner White Paper (In german though): https://www.youtube.com/watch?v=2QKewyoOSL0

Now he works for Google as an Engineering Manager.

executive 2 days ago 1 reply      
Does it support spatial objects / can it replace PostGIS?
rdtsc 2 days ago 2 replies      
> This leads to three kinds of systems: CA, CP and AP,

What is a distributed system that is CA? Can you build a distributed system which will never have a partition.

tdrd 2 days ago 2 replies      
Doesn't seem possible to use this yet. No client libraries and no samples: https://cloud.google.com/spanner/docs/tutorials

Have they documented the wire protocol? I couldn't find it.

sandGorgon 2 days ago 2 replies      
> If you have a MySQL or PostgreSQL system that's bursting at the seams

Postgresql ? How does this work for people migrating from traditional SQL databases - typically people use ORM. How would this fit in with, say , Rails or SqlAlchemy ?

nodesocket 2 days ago 2 replies      
So does Cloud Spanner replace the existing Google Cloud SQL offering [1]? What are the pros/cons of each?

[1] https://cloud.google.com/sql/

rattray 2 days ago 2 replies      
Very interesting. How does this pricing compare to AWS Aurora? https://aws.amazon.com/rds/aurora/pricing/
etherealG 1 day ago 0 replies      
I'd love to see a jepsen test. Maybe the spanner team would be able to sponsor an independent test?
jallriddle 2 days ago 1 reply      
Is this similar to AWS Aurora or is this something else completely different?
danols 2 days ago 0 replies      
Interesting but without INSERT and UPDATE it just isn't worth it for me. When can we expect it to handle data manipulation language (DML) statements?
DivineTraube 2 days ago 2 replies      
"What if you could have a fully managed database service that's consistent, scales horizontally across data centers and speaks SQL?"

Looks like Google forgot to mention one central requirement: latency.

This is a hosted version of Spanner and F1. Since both systems are published, we know a lot about their trade-offs:

Spanner (see OSDI'12 and TODS'13 papers) evolved from the observation that Megastore guarantees - though useful - come at performance penalty that is prohibitive for some applications. Spanner is a multi-version database system that unlike Megastore (the system behind the Google Cloud Datastore) provides general-purpose transactions. The authors argue: We believe it is better to have application programmers deal with performance problems due to overuse of transactions as bottlenecks arise, rather than always coding around the lack of transactions. Spanner automatically groups data into partitions (tablets) that are synchronously replicated across sites via Paxos and stored in Colossus, the successor of the Google File System (GFS). Transactions in Spanner are based on two-phase locking (2PL) and two-phase commits (2PC) executed over the leaders for each partition involved in the transaction. In order for transactions to be serialized according to their global commit times, Spanner introduces TrueTime, an API for high precision timestamps with uncertainty bounds based on atomic clocks and GPS. Each transaction is assigned a commit timestamp from TrueTime and using the uncertainty bounds, the leader can wait until the transaction is guaranteed to be visible at all sites before releasing locks. This also enables efficient read-only transactions that can read a consistent snapshot for a certain timestamp across all data centers without any locking.

F1 (see VLDB'13 paper) builds on Spanner to support SQL-based access for Google's advertising business. To this end, F1 introduces a hierarchical schema based on Protobuf, a rich data encoding format similar to Avro and Thrift. To support both OLTP and OLAP queries, it uses Spanner's abstractions to provide consistent indexing. A lazy protocol for schema changes allows non-blocking schema evolution. Besides pessimistic Spanner transactions, F1 supports optimistic transactions. Each row bears a version timestamp that used at commit time to perform a short-lived pessimistic transaction to validate a transaction's read set. Optimistic transactions in F1 suffer from the abort rate problem of optimistic concurrency control, as the read phase is latency-bound and the commit requires slow, distributed Spanner transactions, increasing the vulnerability window for potential conflicts.

While Spanner and F1 are highly influential system designs, they do come at a cost Google does not tell in its marketing: high latency. Consistent geo-replication is expensive even for single operations. Both optimistic and pessimistic transactions even increase these latencies.

It will be very interesting to see first benchmarks. My guess is that operation latencies will be in the order of 80-120ms and therefore much slower than what can be achieved on database clusters distributed only over local replicas.

aladine 2 days ago 1 reply      
Great product from Google. I wonder what is the difference between Cloud Spanner and Google CloudSQL
tajen 2 days ago 0 replies      
> $.90 per node per hour

That makes $700 per month. Is this the minumum? or can we have 0 node when the lambda is idle ?

darkerside 2 days ago 3 replies      
> Unlike most wide-area networks, and especially the public internet, Google controls the entire network and thus can ensure redundancy of hardware and paths, and can also control upgrades and operations in general

I know this is a single system, but I'll still say it. This seems like another step in a scary trend for our internet.

gigatexal 2 days ago 1 reply      
The sql syntax reference looks similar to that of Postgres' syntax reference.
esseti 2 days ago 0 replies      
does anyone know if it works with django or a way to make it working? it should be a matter of a connector, no?
kozikow 2 days ago 0 replies      
SqlAlchemy engine please :) ?
crooked-v 2 days ago 0 replies      

How long until it gets shut down with a month's notice?

koolba 2 days ago 14 replies      
> Today, were excited to announce the public beta for Cloud Spanner, a globally distributed relational database service that lets customers have their cake and eat it too: ACID transactions and SQL semantics, without giving up horizontal scaling and high availability.

This is a bold claim. What do they know about the CAP theorem that I don't?

Separately, (emphasis mine):

> If you have a MySQL or PostgreSQL system that's bursting at the seams, or are struggling with hand-rolled transactions on top of an eventually-consistent database, Cloud Spanner could be the solution you're looking for. Visit the Cloud Spanner page to learn more and get started building applications on our next-generation database service.

From the rest of the article it seems like the wire protocol for accessing it is MySQL. I wonder if they mean to add a PostgreSQL compatibility layer at some point.

elcct 1 day ago 1 reply      
Avloss 2 days ago 0 replies      
Amazing! But why does this feel like such a de ja vue all over again.. (surely I'm missing something).. They've spent 5 years telling us that we just CAN'T scale SQL.. Now they'll tell us that actually.. they've figured it out! :)
api 2 days ago 1 reply      
Given the CAP theorem I wonder what trade-offs they make and how much visibility they give you into these trade-offs.

In any case this is much better than Amazon's offerings... when they actually ship it. :)

williamle8300 2 days ago 0 replies      
I see there's "data layer encryption" but the data is still readable by Google. Why would anyone want to keep feeding the Google beast with more data?

Software is about separating concerns, and decentralizing authority. Responsible engineers shouldn't be using this service.

kozak 2 days ago 0 replies      
I wonder how many people will get a seizure from that red-blue blinking rectangle in the video :(

Upd: Downvoting this warning will only increase that number.

theptip 2 days ago 1 reply      
> Does this mean that Spanner is a CA system as defined by CAP? The short answer is no technically, but yes in effect and its users can and do assume CA.

It's somewhat ironic that Brewer, the original author of the CAP theorem, is making this sort of marketing-led bending of the CAP theorem terminology. I think what he really should be saying is something in more nuanced language like this: https://martin.kleppmann.com/2015/05/11/please-stop-calling-...

But perhaps Google's marketing department needed something in the more popular "CP or AP?" terminology. I don't see what would be wrong with "CP with extremely high availability" though.

It's certainly wacky to be claiming that a system is "CA", since as the post admits it's technically false; to me this makes it clear that CP vs. AP (vs. CA now?) does not convey enough information. I'd prefer "a linearizably-consistent data store, with ACID semantics, with a 99.999% uptime SLA". Not as snappy as "CA" (I will never have a career in marketing I suppose), but it makes the technical claims more clear.

India has banned disposable plastic in Delhi globalcitizen.org
660 points by SimplyUseless  2 days ago   251 comments top 32
Maarten88 2 days ago 4 replies      
Rwanda also has this policy, since 2008, and they enforce it. The country is very clean, it looks different than other African countries (and countries like the US), just because there is no plastic rubbish everywhere. I think this is a very good policy, and would welcome it at home.


bendermon 2 days ago 3 replies      
Note: Thank you for down voting for pointing out a glaring 'fake statistics' and poor journalism, on a #1 trending post on HN.

The first and the second quote do not mean the same thing, not even close.

"A massive 60%t of the plastic waste in the oceans is said to have come from India, according to the Times of India."

The TOI reads - "Banning disposable plastic is a huge step for the capital and the country because India is among the top four biggest plastic polluters in the world, responsible for around 60% of the 8.8 million tons of plastic that is dumped into the worlds oceans every year."

As an Indian, I see a lot of journalists stuck in a colonial era. They go out of their way to tarnish and stereotype the great unwashed. They manage to turn even positive news to mock and heckle the less developed world.

But this article has taken it to great heights. The TOI isn't exactly known for journalistic integrity and often conveniently pulls statistics from their backside. But to misquote the devil, this article has certainly hit the lowest level.

wallace_f 2 days ago 4 replies      
The economist inside me says it would be better to tax disposable plastic.

The tax could cover the cost to clean up the litter. That would create jobs in three ways: 1) plastic clean-up jobs, 2) businesses and economic activity that desperately need disposable plastic can still possibly survive, and ) jobs making disposable plastic.

Anyways, it's a lot better than taxing things we all agree we want more of. Like jobs.

throwaway420 2 days ago 7 replies      
Not a huge fan of outright bans, and think this is probably the wrong priority for India to focus on. (I understand this is just Delhi)

Air pollution is huge right now. And sad to say, people pooping in streets and rivers is still a major problem.

To me, plastic remnants are a very minor issue in comparison.

geodel 2 days ago 3 replies      
I do not see any coverage in Indian media. Could it be one of those official notifications which public at large hardly follows but it breeds corruption by enforcement officials. Though looking at Indian papers I came across this rather frightening news:


hive_mind 2 days ago 3 replies      
This is India. Starving, thirsty, injured (hit by cars and trucks) cows and dogs roam the streets, hobbling along, limping along. There are people starving on the street-side. The poorest children are openly prostituted on the streets.

How are they going to enforce a rule regarding plastic bags?

The rich will continue to do whatever they want.

The middle-classes will continue to do whatever they can get away with.

The poor will continue to be shit on and abused.


About 20 years ago they banned smoking in public in Delhi (I was there when they did it).

All that this ordinance did was to give the police yet another angle to harass people. More corruption. More bribes.

bogomipz 2 days ago 4 replies      
This is great news! I have a question maybe someone from Delhi could answer, the article states:

"The ban took effect on the first day of 2017."

What are the vendors doing? Is water being sold in glass bottles with a deposit scheme for redemption now?

walrus01 2 days ago 0 replies      
I wish that they could find a way to do this in the Islamabad/Rawalpindi area. Due to a lack of budget and government coordination for large scale trash pick up, nearly every stream and ravine in the area is the designated trash dumping grounds. It's thoroughly littered with plastic shopping bags and plastic bottles. It won't solve the problem of people throwing trash on nearly any available piece of unclaimed or unusable land, but at least it'll be paper based or biodegradable.
rjurney 2 days ago 0 replies      
This is awesome. When I was in India ten years ago, everywhere we went in rural India the trees along creeks, rivers and streams were littered with plastic bags from where floods had deposited them. They were like leaves, and the trees were dead. They had already banned plastic bags locally in that province, and it is good to see a global ban on plastic in general.

In Himachal Pradesh, the plastic bag ban had resulted in a cottage industry forming where discarded newspapers were folded/glued into shopping bags. I'd like to see this same thing happen in the US. A friend imported a palette of these bags to Florida, and he was able to sell them to vendors and make a small profit. This tells me they might be viable here commercially.

As they say, reduce > re-use > recycle.

xyzzy4 2 days ago 2 replies      
Plastic is the least of Delhi's problems. First they should be banning vehicles that don't meet emission standards. I visited there twice and I'll be lucky if I don't come down with some lung problem.

Edit:They should ban the burning of plastic, not plastic itself. And enforce it.

SteveNuts 2 days ago 2 replies      
Banning it is one thing, we'll see if they can actually enforce it.
agentgt 1 day ago 1 reply      
If you are not using reusable bags when going to the grocery store (or any store) please attempt to try it.

It absolutely amazing how many more groceries you can put in a reusable bag over a plastic or paper bags. So at the bare minimum it is an optimization (less trips from car to kitchen).

When I bring this up with people I get unbelievable false rationalization like: I reuse the bags for trash or the reusable bags take 100x to make over plastic bags.

Plastic grocery bags have knack of flying into lakes, rivers and streams. I have saved many turtles and fish a long the Charles river that are caught in these bags. I have never seen a kitchen trash bag or a reusable bag in the river.

Many grocery stores even give a discount if you use reusable bags not to mention reusable bags are extremely cheap (I don't think I have ever paid more than $2).

Abishek_Muthian 1 day ago 1 reply      
When it comes to banning plastic bags, India takes the lead & not just Delhi; cities like Mumbai, Karwar, Tirumala, Vasco, Rajasthan all have a ban on the bag though enforcement is questionable. Where as rural villages such as 'Mawlynnong' has not only banned the plastic bags but also got the title of 'Asia's cleanest village', there are other such villages in India. It should be commended that, villages which hosts majority of population with lower economic backgrounds are doing their part to protect their environment even when it's much difficult for them to do so than their counterparts in the cities who usually don't need to worry about whether their children get to eat today.
sfifs 1 day ago 1 reply      
Various forms of plastic bans keep getting imposed year after year in different places in India.

Nothing practically happens because:

a. Police has better things to do than round up people and shops carrying plastic bags. They may probably take a bribe from the shop to turn a blind eye.

b. There's no really low cost and convenient alternative in many cases - in spite of a lot of shops in India using recycled newspapers for packaging.

Home delivery which is widespread among the richer classes is at least partially helpful since shops bring things in their own bags. However most of India is not rich

SoulMan 1 day ago 0 replies      
Passing a rule is very easy in India as opposed to the enforcement or people following that. However in this case majority of the source are commercial outlets which is relatively less difficult to enforce .
noahmbarr 2 days ago 0 replies      
All too often it seems US restaurants do the quick calculus of going 100% disposable cutelry/etc, making a decision we'll still be dealing with 50 years from now....

I am very supportive of these types programs, even if hard to enforce.

ra2 18 hours ago 0 replies      
It is also banned in Lucknow, a beautiful city in Indian state Uttar Pradesh.It is a good move.Although govt. should invest more in R&D in order to find environment friendly alternative.Street food sellers who deal in liquid edibles had to raise their prices in order to use current plastic alternatives which are currently clay and glass.
CaiGengYang 2 days ago 4 replies      
What are they going to replace plastic with ? Can you invent a kind of material that is as capable as plastic but has zero negative effects on the environment ?
random16 1 day ago 0 replies      
Three questions: 1) what will replace plastic for convenience products ( aka single serving consumables )? 2) are there organizations championing this effort elsewhere? 3) where can i read more facts about Delhi's efforts?
edblarney 2 days ago 1 reply      
It's great. But the 'trash problem' in India runs deeper.

They need to have anti-litter regulation, awareness campaigns, and enforcement.

smb06 1 day ago 0 replies      
I don't see this being implemented. At least none of the friends and family have reported this is being implemented. Folks from India - anyone have a different feedback and they heard this is really being implemented?
theparanoid 2 days ago 2 replies      
Here in California single use plastic bags are banned. Now it's a pain, at checkout you either have to tell them how many multiuse bags to charge or bring bags from home.

No so many years ago paper bags were common and what I used. Paper is great, it's biodegradable, renewable, and convenient.

l3satwik 1 day ago 0 replies      
Let's just hope that this is implemented in an ordered way. We all remember what happened to Delhi in the last Diwali celebration even after having so much of law against firecrackers.
dirkg 2 days ago 0 replies      
Good in principle, will cause hardships for many in practice. There are probably millions of street vendors who rely on plastic to pack food to go, as well as many other shops of course.

There are many other priorities to focus on which can have a far bigger impact.

gheijfnfd 1 day ago 0 replies      
Bangalore did this 3-4 years back. I had trouble getting disposable bags even in rural KA.
upofadown 2 days ago 1 reply      
>The ban includes bags, cups and cutlery.

What about all the plastic containers the food originally came in?

toephu2 2 days ago 2 replies      
I don't understand how we can build self-driving cars, send people to the moon, and create advanced facial recognition software, but can't build technology that can separate plastic bags?
awqrre 2 days ago 0 replies      
no more plastic trash bags... and of course no more using grocery bags as trash bags...
kumarski 2 days ago 0 replies      
Hopefully it is enforced.
vanattab 2 days ago 4 replies      
Can we get the title changed to make it clear that India banned plastic in Delhi not all of India.

Also the title says "literally all disposable plastic" then in the article it says applies to cups, bags, cutlery.

libso 2 days ago 2 replies      

"Delhi has banned disposable plastic"

Not all of India. Just Delhi.

X86BSD 2 days ago 1 reply      
Why only Delhi?! That country is drowning in filth not just Delhi.
GitHub commit search: remove password github.com
846 points by rsc-dev  1 day ago   251 comments top 65
jvehent 1 day ago 8 replies      
Too many comments here recommend to clean up the commit and just hide the mistake under the rug. This is wrong.

If you leak a password to any public location, there is only one reasonable course of action: CHANGE IT!

Don't even bother rewriting the commit. Focus on changing that password right away, and while you're at it, figure out a better way to manage your secrets outside of your source code in the future. Mistakes happen, but they shouldn't be repeated.

josscrowcroft 1 day ago 4 replies      
I liked this one:


"... It's not really removing any password, is it? But hey, why not use themomentum ... wheeeeeeeeeeeeeeeeee!"

dorianm 1 day ago 4 replies      
Right after my "remove secrets" post: https://news.ycombinator.com/item?id=13650614

There are just so many of those it's crazy:

Also replace "remove" with delete/rm/replace/etc.

And replace "YOURFAVORITEAPI" with CircleCI, Travis, Mailchimp, Trello, Stripe, etc, etc.

Also, companies I contacted consider it the customer fault and basically don't care.

therealasdf 1 day ago 3 replies      
For anyone wondering, if you want to remove a file or secret you've already committed, you can use BFG Repo-Cleaner to go through your commit history and completely remove any trace of it.


lucideer 1 day ago 3 replies      
There are so many of these.

It gets a little scary when it veers from professional security to individual personal privacy https://github.com/search?p=2&q=smtp.gmail.com+pass&ref=sear...

chiefalchemist 1 day ago 6 replies      
Computers are supposed to be good at what imperfect humans are not. This only proves how primitive the tool is.

That is, for example, if Gmail can ask "it looks like you forgot the attachment" why can't Git say "this is a public repo and you're about to commit and push passwords. Are you sure?"

It's going to be easier to fix the tool than it is to make humans be perfect.

djm_ 1 day ago 0 replies      
A good a time as ever to mention AWS's provider-agnostic secrets-aware git hook that attempts to prevent this at the repository level [1]

[1] https://github.com/awslabs/git-secrets

wybiral 1 day ago 1 reply      
A while ago I discovered similarly that there are several searches which lead you to active database logins.




joshfarrant 1 day ago 1 reply      
Throw the word 'oops' in for good measure.


raesene6 1 day ago 4 replies      
This is a good example of the increased risks from doing your development out in the open, any mistakes are exposed to a much wider group of potential adversaries.

On an internal VCS, this would still be a problem, but a bit less visible/exploitable...

sand500 1 day ago 1 reply      
To people like me who have done this many times in the past and want to add the file to gitignore


The other alternative I can think of is to hide sensitive values in environment variables

kh_hk 1 day ago 0 replies      
danarmak 1 day ago 0 replies      
Search for "update password" and you'll see almost as many results (268,000), many presumably with active passwords.

"Add password" finds 792,000 results, of which at least some (on the first page) are actual passwords.

Jonnax 1 day ago 0 replies      
I'd hope that all those people promptly changed their passwords after realising. But 200k commits, I'm sure a good percentage of them didn't.
karrotwaltz 1 day ago 1 reply      
Related topic: "Production AWS keys on GitHub" ~ 3 years ago


By just looking quickly, it seems that you can still find many recent live keys...

brightball 1 day ago 1 reply      

I get it...you like github but you don't want to pay for private repos. That's when you use Gitlab or BitBucket and then this problem goes away.

sirn 1 day ago 0 replies      
Searching filename:id_rsa also yield a rather interesting result. Alongside with "BEGIN OPENSSH PRIVATE KEY". I wonder how much of these also contains ssh_config.
adamvalve 1 day ago 0 replies      
ceautery 1 day ago 0 replies      
After this was posted, there has been some serious trolling in Github with "fake" commits matching this search.
teaearlgraycold 1 day ago 1 reply      
I set up a honeypot and made this commit: https://github.com/teaearlgraycold/honey/commit/7c4289717979...

Already had a couple of sassy individuals telling me my honeypot is shit via the tty logging.

_joel 1 day ago 0 replies      
Another for the list, JSON or YAML containing 'password' - https://github.com/search?utf8=%E2%9C%93&q=password+extensio...

Reminds me of the eye opening experience available at https://www.exploit-db.com/google-hacking-database/

mickael-kerjean 17 hours ago 0 replies      
GitHub has become the best place to find all sort of sensitive information.From root password, access to companies network, Api keys, everything is available from a search box, you don't need to be a genius to do serious damage, spying or doing all sort of black hat stuff.

Sure people should clean up their work, but as a fact not everybody does and it won't change tomorrow.You'll simply hear on the news: some Russian hacker are behind the attack or another bad excuse

sagivo 1 day ago 0 replies      
git is a great tool, unfortunately there's no standard of storing sensitive info (like passwords). some store it as ENV variables, some hard copy the files to servers using custom scripts etc.. would love to see an easy tool that developers can use to manage this info like people manage files on git.
bigtunacan 1 day ago 0 replies      
This made me realize an unexpected (to me) search behavior on Github. Basic/Default search will search commit history, but if I try to add advanced options I don't appear to get search history.


Here I was trying to search for "remove password" just on repos for nicksagona (just happened to be one of the first users to display when you go to this thread's search).

That comes up with zero results. This leaves me wondering how I would run similar searches on repos that I'm involved with as a way of auditing to make sure none of them have compromised passwords that would need changed.

I would love to hear suggestions on how to do this.

drinchev 1 day ago 7 replies      
How do you guys, handle this problem?

I use either `git-crypt` [1] or `ansible-vault` [2].

1: https://github.com/AGWA/git-crypt

2: http://docs.ansible.com/ansible/playbooks_vault.html

random_upvoter 1 day ago 0 replies      
Good thing all my commit messages are "xxx"
level 1 day ago 0 replies      
This is pretty much how I found a couple exposed Stripe API keys. You just need to look through code of people who use the example implementation, and then dig in the history/config a bit. If you send stripe a key or two, they'll give you a free shirt.


dpix 1 day ago 1 reply      
This is the main reason I use gitlab, because they have free private repos. When your trying to smash out code as fast as possible in a startup you don't want to have to worry about acccidently checking a secret in
LeonidBugaev 1 day ago 5 replies      
If you found a similar mistake in your repository, you can delete commit from history using: `git rebase --onto <commit-id>^ <commit-id>`. Or if you want actually rewrite it, see git rebase -i` documentation.
equalarrow 1 day ago 0 replies      
Not only passwords, but api keys as well. I can't tell you how many times I've come across public repos that have full api credentials in them. Boggles the mind..
franzwong 1 day ago 0 replies      
You'd better change the password instead of removing it.
nkkollaw 1 day ago 0 replies      
Looks like a lot of people read Hacker News... https://github.com/doutchnugget/awesomevim/commit/a7292962ce...
palerdot 1 day ago 3 replies      
Just using a random commit name like 'minor bug fix', 'updated version' for these kind of commits will save a lot of headaches like this.

One can do better by adding random lines/ logs in lot of files and sneakily remove password from one of them and then give a random commit name.

But then it all boils down to your mindset at that particular moment when you are commuting.

jakobov 1 day ago 0 replies      
Someone here should make a bot and leave comments on all those commits warning people to change their passwords.
ioquatix 1 day ago 2 replies      
Git-crypt is your friend!
WildUtah 1 day ago 0 replies      
When you put your dotfiles in a repository online, be sure to commit all the public keys and none of the private ones.

Github, like SSH, uses an asymmetric authentication scheme. They even publish everyone's public keys. It's much more secure than passwords.

martincmartin 1 day ago 0 replies      
Years ago, there were proggit posts of google searches for open myphpadmin consoles. I think people then went and deleted/messed with the databases. My search fu is failing me though, I can't find any of those threads.
oelmekki 1 day ago 1 reply      
The worst part in that is that it provides tons of passwords to analyze and detect recurring words or schemes. This probably also will hurt people that never commited their passwords in public repos. Github should probably filter out such searches.
starefossen 1 day ago 1 reply      
Or try "remove aws"
hharnisch 1 day ago 0 replies      
Thank you for highlighting this is a such a common mistake. I hope developers of all skill levels look at this and realize it how easy it is to make this mistake and learn from it.
mzzter 1 day ago 0 replies      
I would recommend using torus.sh or other secret manager instead of an env or text file. I've forgotten to include them in .gitignore too many times.
hatsunearu 1 day ago 0 replies      
some random guy spammed a ton of commits with a bitcoin address saying "please send me money"

jesus christ, how low can you stoop

cybergrime 1 day ago 0 replies      
There are tools that automatically rewrite your git history and can fix this.
ssebastianj 1 day ago 0 replies      
Maybe a confidential-linter or git pre-commit hook would be nice to prevent leaking confidential information.
westoque 1 day ago 0 replies      
Good find. Reminds me of the a query done for Google Search that exposes sites vulnerable to SQL injection.
burnbabyburn 1 day ago 0 replies      
leaks of this types are known since AGES, and still people are unable to keep private things private.
atti7 1 day ago 1 reply      
brockvond 1 day ago 0 replies      
can someone please give me an ELI5 breakdown of what I'm missing... did these people attempt to change their password via git commits?
jamesphillips 1 day ago 0 replies      
Once it's on the internet. Sites like github have become an amazing OSINT tool.
jgritty 1 day ago 0 replies      
And now I got goatse'd, thanks Hacker News, lol.
EJTH 1 day ago 0 replies      
Almost as many results on 'remove credentials' :-)
empath75 1 day ago 0 replies      
Rotate your keys and passwords in production, everyone.
tmsldd 1 day ago 0 replies      
The question is: how many are left in the wild?
joshuajeeson 1 day ago 0 replies      
Another one: Search for Root password :D
nico01f 1 day ago 0 replies      

This could be a big thing. It's time to write: How to write code without expose you

asc123 1 day ago 0 replies      
use quotes. only about 19k
thrillgore 1 day ago 0 replies      
One of the things I've done after making this mistake was creating an example config file for this information, committing that, and then dropping a gitignore on the real config file.
anacleto 1 day ago 0 replies      
Holy shit!
kxait 1 day ago 0 replies      
some of these seem legit
jandrusk 1 day ago 0 replies      
Encryption, do you speak it?
cultavix 1 day ago 0 replies      
imagine private repo's :)
androtheos 1 day ago 0 replies      
Not nice
Show HN: A guide to all HTML5 elements and attributes htmlreference.io
717 points by bbx  2 days ago   93 comments top 41
shpx 2 days ago 1 reply      

Same thing but guaranteed to be up to date and (more) complete.

For example, htmlreference.io's page for <input> doesn't mention the autocomplete attribute. MDN lists all its possible values.



callahad 2 days ago 5 replies      
I believe your kbd example is incorrect. You suggest

> To save, press <kbd>Ctrl + S</kbd>.

But the spec (both W3C and WHATWG) suggests that individual keys should be nested inside an outer <kbd> tag: "When the kbd element is nested inside another kbd element, it represents an actual key or other single unit of input as appropriate for the input mechanism."

Thus, the example should be:

> To save, press <kbd><kbd>Ctrl</kbd> + <kbd>S</kbd></kbd>.

Cite: https://w3c.github.io/html/textlevel-semantics.html#the-kbd-...

On the face of it, this seems ridiculous. It's too verbose, the tag name is misleading, and if you actually use the correct markup on GitHub or StackOverflow, it will render incorrectly because both sites assume the standalone <kbd> element represents physical keyboard buttons.

On the other hand, what's the value in semantic markup if we don't adhere to its semantics?

Practically speaking, I would be a happier person today if I hadn't read that part of the spec, and instead persisted on in blissful ignorance of the element's intended semantics. Thanks, specs.

blauditore 2 days ago 2 replies      
How is this updated (manually or automatically from official specs)?

Call me paranoid, but I see this diverging from actual specs, then people googling for "html reference" finding it and thinking it is something official. The result would be another W3Schools disaster[1].

In my opinion, the official W3 specification pages are not that bad, and alternatively there's the simpler MDN with strong community support (thus lower risk of deprecation).

[1]: http://www.w3fools.com/

dagw 2 days ago 1 reply      
Really cool. One feature request. when a description of a tag refers to another tag, then those tags should be links.

For example the description for <li> is:

 Defines a list item within an ordered list <ol> or unordered list <ul>.
Here I should be able to click on <ol> and <ul> to get information about those tags.

falsedan 2 days ago 2 replies      
It's missing the links to context & to the spec, which was the killer feature of the old WDG HTML Reference[0].

[0]: http://www.htmlhelp.com/reference/html40/alist.html

adontz 2 days ago 0 replies      
Links to http://caniuse.com/#search= will be useful.dt, li, option, td, th, tr are visible regardless of finder checkboxes
ahstro 2 days ago 2 replies      
Nice, but seems to be something wrong with the tagging. I unchecked all tags except `experimental` and I ended up with seven results, only one of which is actually experimental, (`picture`), the rest being _pretty_ cemented (`dt`, `li`, `option`, `td`, `th`, and `tr`).It also seems to leave out some other expermental tags, like `wbr` and `slot`, that are on the site.
edent 2 days ago 1 reply      
That's really well laid out. Less detail than MDN, but that site can be overkill.

Two small pieces of feedback. HTTPS support would be good. Also, when I scroll the list of element and then click on one, I'm taken back to the top of the list - I'd like to stay where I am.

darekkay 2 days ago 1 reply      
A more general/comprehensive API documentation tool (which can be also used offline) is DevDocs[1]

[1] http://devdocs.io

codingdave 2 days ago 0 replies      
The checkbox filters at the top are somewhat un-intuitive. For example, If I uncheck everything but experimental, I want to see all experimental... but what I get is all experimental that are not block, inline, etc. I mean, as a developer, I see how those filters work... but as an end user, that isn't how I want them to work.
etimberg 2 days ago 0 replies      
The canvas page [1] has incorrect defaults listed for the width. It states that the default is 100 but it is actually 300 [2].

1. http://htmlreference.io/element/canvas/2. https://developer.mozilla.org/en-US/docs/Web/HTML/Element/ca...

mouzogu 2 days ago 1 reply      
from a UX perspective i really like this. like the way it's laid out and presented - found it easy to scan.

I know a lot of people big up MDN vs W3Schools and all their arguments are basically correct but i find it (mdn) really ugly and hard to read. i often find myself going to w3schools to just copy a snippet or get a 1 line description of what i need.

mdn often feels more like a technical spec as opposed to a guide, which it kind of is i suppose.

great work on the layout!

alexgrcs 2 days ago 0 replies      
The <address> example is wrong. <address> defines contact information, but it's not appropriate for all postal and e-mail addresses.http://html5doctor.com/the-address-element/
xamuel 2 days ago 1 reply      
This is really great. It would be even greater if it could be used offline. Instead of hitting the server every time I choose one of the elements to view.
shdon 2 days ago 0 replies      
You've got the description for the "ins" element the same as for the (admittedly related) "del" element.
javajosh 2 days ago 0 replies      
Without compatibility information this is more dangerous than useful. For example, a naive HTML author would use the semantic tags, only to fail in most browsers. There is mention neither of compatibility nor polyfill. Better off with MDN and caniuse.com.
jean- 2 days ago 4 replies      
Good stuff. Would be even nicer if it showed information about optionally self-closing tags, which is one of the main reasons I occasionally need to look at the spec. For instance:

> A p element's end tag may be omitted if the p element is immediately followed by an address, article, aside, blockquote, div, dl, fieldset, footer, form, h1, h2, h3, h4, h5, h6, header, hgroup, hr, main, nav, ol, p, pre, section, table, or ul, element, or if there is no more content in the parent element and the parent element is not an a element.

Links to the relevant parts of the official spec would be nice too, e.g. https://www.w3.org/TR/html5/grouping-content.html#the-p-elem...

tenaciousDaniel 2 days ago 1 reply      
I wouldn't say hgroup is experimental. It was on the standards track and now it's being deprecated.
teh_klev 2 days ago 0 replies      
Nice...would be useful to have the summary of what a tag's purpose is on the main page list of tags. For three letter or less tags it's not always obvious what I'm looking to achieve a specific task.
macintux 1 day ago 0 replies      
Reminiscing about days spent digging through http://www.blooberry.com/indexdot/html/index.html
xophishox 2 days ago 0 replies      
Is there something similar to this for javascript?

I really find this format super easy to read, even if there may be slight inconsistencies and nuances to reading it.

tomkin 2 days ago 0 replies      
I love things like this, but I find I have about 30 of these sorts of things bookmarked, and I never think to pull them up when I'm in need I immediately go to Google and find something like the Mozilla ref @shpx mentioned. It's a real problem I wish there was a solution for. Other than of course having a better memory.
cabalamat 2 days ago 1 reply      
In general this is pretty good. One mistake that it doesn't make is to use low contrast for everything.

However, the code examples don't look very nice on my box due to the font which comes out as Nimbus Mono L. How about you link to an actual font? (I like Ubuntu Mono, though there are lots of other good programming fonts).

Also with respect to code samples, would black text on a white background be a possibuility?

HNaTTY 2 days ago 0 replies      
The filter checkboxes are a bit weird, for example elements (such as br and img) which are both self-closing and inline only show up if both self-closing and inline are selected, instead of one or the other. Also there are several elements such as li which are not categorized and show up even if no checkboxes are selected.
0x1d 2 days ago 0 replies      
This is the cleanest HTML5 reference site I've seen. I like that it has links back to MDN as well.

Is it design completely custom or did you use a template or theme? I'm really struggling with the design side of my side projects.

Also, if you don't mind me asking, how difficult was it to get approved for Carbon Ads?

darkhorn 2 days ago 0 replies      
Keygen is missing https://www.w3schools.com/TAgs/tryit.asp?filename=tryhtml5_k... which is available in Firefox 51.
karol 2 days ago 1 reply      
Can you help with a community driven website content as well?http://docs.webplatform.org/wiki/html
neogodless 2 days ago 1 reply      
For some reason, if I have self-closing and inline checked, and uncheck inline, meta items appear.
badthingfactory 2 days ago 0 replies      
Every time I look at lists of HTML5 elements, I about how I really need to get better at writing semantic HTML. Tomorrow I will go to work and create 500 more div tags.
projektir 2 days ago 0 replies      
The design is really cute!

I appreciate that the website works well even with all scripts blocked.

aiaf 2 days ago 0 replies      
I've only checked the anchor tag, and it is lacking.

It should link to the URI RFC to illustrate all possibly valid syntax for the href attribute. Also doesn't mention 'javascript:'

ulucs 2 days ago 0 replies      
quite cool! but even after deselecting all the options, I'm still being shown some tags (select, tr, th, td, dt, li). I think it's because they haven't been tagged with anything
criswell 2 days ago 0 replies      
If you like this, you'll lOoOoOoOve this: http://cssreference.io/
NKCSS 1 day ago 0 replies      
If I uncheck-all but experimental, it doesn't hide dt, li, option, td, th and tr.
KingMob 2 days ago 0 replies      
Nice. Can you add support for <svg>? I know it's not HTML...but it would be handy.
cyborgx7 2 days ago 1 reply      
Why does it want me to whitelist it in my adblocker when it claims to be free and to always be free?
jeshan25 2 days ago 0 replies      
really really cool. I've shared it in a facebook group for front end devs that I'm part of.

One minor issue: Your search bar doesn't look like one :)I didn't notice it until my second visit.

k__ 2 days ago 0 replies      
Is there a good list of the most used meta tags out there?
jkochis 2 days ago 0 replies      
Can you add the 'range' input type?
lisper 2 days ago 1 reply      
You left out <center> ;-)
SFJulie 2 days ago 0 replies      
Reading the comments:

- nice brogramming;- not up to date;- not refering the spec (sorry this is important when you are in trouble);- well read @shpx comment he is right

Verdict: more signal less noise policiy says, don't click to the bait.

normally I put a goatse.cx link here or a rickroll

TensorFlow 1.0 Released googleblog.com
629 points by plexicle  1 day ago   69 comments top 19
timanglade 1 day ago 3 replies      
Been using Tensorflow embedded in a mobile app for a few months and honestly, Im constantly surprised at how well thought-out the tooling is, and how quickly you can get results. Conversely, I think a few things are still unnecessarily dense (installing dependencies, optimizing hyper-parameters, and some of the embedded/XLA stuff is very raw). Kudos to the team though. It sounds like theyre on the right track with TF overall, and focusing on performance (including the XLA stuff) + ease of use (high-level, Keras API) is absolutely what I want as a user right now. Keep up the great work, yall.
plexicle 1 day ago 1 reply      
For anyone updating to 1.0--

There are quite a few breaking changes but there is a very helpful conversion script here: https://github.com/tensorflow/tensorflow/tree/r1.0/tensorflo....

You can find the breaking changes in the 1.0 release here: https://github.com/tensorflow/tensorflow/releases/tag/v1.0.0

andy_ppp 1 day ago 3 replies      
Amazing work; it makes using AI and Deep Learning accessible for everyone here really. If you haven't seen it check this out for an intro:


I wish AMD graphics cards were supported fully. I really think AMD should find a way to work with the Tensor Flow team on this...

taliesinb 1 day ago 0 replies      
> Plus, soon Google will open-source code that will multiply the speed of TensorFlow specifically version three of Googles Inception neural network model by 58.

Uh, nope, that was speedup on 64 GPUs (or CPU cores, can't remember). i.e. it scales linearly, something that TF hasn't always been good at v other frameworks. I'm amazed a journalist with (I assume) basic technical competence could make this mistake.

plexicle 1 day ago 0 replies      
TensorFlow 1.0 was just announced during the TensorFlow Dev Summit keynote.


You can follow the Summit live here: https://www.youtube.com/watch?v=LqLyrl-agOw

cstuder 1 day ago 6 replies      
How do I get started with machine learning?

I have a couple of applications in mind, mostly time series predictions. But the machine learning field seems to be vast and I don't know where to start.

jamesblonde 1 day ago 0 replies      
I am looking at the Martin Wicke talk. The Estimator API is very reminiscent of SparkML. Nice to see that the tensorflow crew are flexible enough to take good ideas from projects such as SparkML and Keras (now included natively in the TF stack).Other highlights include the hotspot compiler (I was not that impressed so far, but it's early days for them), and embedded visualizations (looked quite cool) for visually inspecting learnt manifolds.
drakonka 1 day ago 1 reply      
I stumbled across a three-chapter preview of the upcoming book Learning TensorFlow on Safari Books Online and went through them in a sitting. It was so accessible - both the book and TensorFlow itself - and inspired me to start learning math so that when the rest of the book comes out I will be better prepared to go deeper. I love learning in general, but haven't been this excited about learning something totally new (for me) in a long time.
jjuel 1 day ago 2 replies      
Not necessarily about the article so may get downvoted, but is there a good book for TensorFlow/Machine Learning?
fest 1 day ago 0 replies      
Does anyone use Tensorflow models in C++ applications? Is it possible to build Tensorflow as static or shared lib?
sixbrx 1 day ago 4 replies      
Kudos to the team. Anybody know if we can we train in languages other than Python yet (or do I have that wrong)?
mrcabada 23 hours ago 1 reply      
Could MacBook Pros (with Intel HD Graphics 3000 384 MB, to be more specific) train with GPU? I've always wanted to train algorithms but without using the GPU it is really slow.
120bits 10 hours ago 0 replies      
For a complete beginner. What kind of applications I can work on using TensorFlow?
jamesblonde 1 day ago 0 replies      
The tensorflow developer summit is being streamed live right now on youtube:https://www.youtube.com/watch?v=LqLyrl-agOw
ndesaulniers 22 hours ago 0 replies      
Even if you don't care about machine learning, TensorFlow's XLA is amazing for farming code out to the GPU. GPGPU has never been easier.
mark_l_watson 1 day ago 0 replies      
Great news. I have several TensorFlow examples in a new book I am writing. I need to read up on the new higher level APIs, and can hopefully shorten the book example prob-grams.
shmageggy 1 day ago 0 replies      
Ahh, so that's why the 1.0rc docs started to 404 an hour ago. Had me cursing under my breath :)
amelius 1 day ago 1 reply      
Is there a good birds-eye overview of what people are creating with TF or ML in general?
dang 1 day ago 3 replies      
We changed the URL from https://www.tensorflow.org/, which doesn't say anything about 1.0, to an article which gives a bit of background. If someone suggests a better URL we can change it again.
Googles not-so-secret new OS techspecs.blog
631 points by techenthusiast  1 day ago   531 comments top 50
michaelmrose 1 day ago 12 replies      
It was unfortunately obvious that the writer had insufficient tech chops when use the phrase

"a post-API programming model"

But pressing on how somehow manages to blame the lack of updates to android phones on the modularity of the Linux kernel. The joke of course being that linux is monolithic and googles new OS is a microkernel ergo more modular.

The quote is "...however. I also have to imagine the Android update problem (a symptom of Linuxs modularity) will at last be solved by Andromeda"

Its hilarious that he can somehow defying all sanity ascribe androids update issue to an imagined defect in Linux. Android phones don't get updated because for the manufacturers ensuring their pile of hacks works with a newer version of android would represent a non trivial amount of work for the oem whom already has your money. The only way they can get more of your money is to sell you a new phone which they hope to do between 1-2 years from now.

In short offering an update for your current hardware would simultaneously annoy some users who fear change, add little to those who plan to upgrade to a new model anyway, decrease the chance that a minority would upgrade, and cost them money to implement.

Its not merely not a flaw in the underlying linux kernel its not a technical issue at all.

MarkMc 1 day ago 16 replies      
It bothers me that Google does not seem particularly interested in doing the one thing that would make their Android platform absolutely dominant: Allow Chrome to run Android apps on Mac and Windows.

Google has already done 90% of the necessary work by adding Android apps to ChromeOS. Two and a half years ago it created "App Runtime for Chrome" which demonstrated that Android apps could run on Windows and Mac in a limited, buggy way [1]. If Google had put meaningful effort into developing such a strategy we would by now have a relatively simple way to develop software which runs on 99% of laptops and 85% of smartphones and tablets. Developers would now be targeting 'Android first' instead of 'web app first then iOS then maybe Android'.

Sundar, if you're reading this - do it!

[1] https://arstechnica.com/gadgets/2014/09/hack-runs-android-ap...

vii 1 day ago 5 replies      
It's awesome that Google is doing this and in public too https://fuchsia.googlesource.com/

Unfortunately, the hard part of an operating system isn't in a cool API and a rendering demo. It's in integrating the fickle whims of myriad hardware devices with amazingly high expectations of reliability and performance consistency under diverse workloads. People don't like dropped frames when they plug in USB :) Writing device drivers for demanding hardware is much harder than saving registers and switching process context. The Linux kernel has an incredible agglomeration of years of effort and experience behind it - and the social ability to scale to support diverse contributors with different agendas.

Microsoft, with its dominant position on the desktop, famously changed the 'preferred' APIs for UI development on a regular cadence. Only Microsoft applications kept up and looked up to date. Now Google has such a commanding share of the phone market - Android is over 80% and growing http://www.idc.com/promo/smartphone-market-share/os - they have a huge temptation to follow suit. Each time that Microsoft introduced a new technology (e.g. https://en.wikipedia.org/wiki/Windows_Presentation_Foundatio... WPF) they had to skirt a fine line between making it simple and making sure that it would be hard for competitors to produce emulation layers for. Otherwise, you could run those apps on your Mac :)

There are many things to improve (and simplify) in the Android APIs. It would be delightful to add first class support for C++ and Python, etc. A project this large will be a monster to ship so hopefully we'll soon (a few years) see the main bits integrated into more mainstream platforms like Android/Linux - hopefully without too much ecosystem churn

conradev 1 day ago 0 replies      
The drivers-wifi repository contains a stub for a Qualcomm QCA6174 driver[1] which is found in the Nexus 5X[2], OnePlus 2[3] and meant for smartphones[4]. The drivers-gpu-msd-intel-gen repository contains drivers for Intel 8th and 9th gen integrated graphics[5]. I think it's fair to propose that Google plans on running Fuchsia on both smartphones and laptops

[1] https://github.com/fuchsia-mirror/drivers-wifi/blob/master/q...

[2] https://www.ifixit.com/Teardown/Nexus+5X+Teardown/51318#s112...

[3] https://www.ifixit.com/Teardown/OnePlus+2+Teardown/45352#s10...

[4] http://www.anandtech.com/show/7921/qualcomm-announces-mumimo...

[5] https://github.com/fuchsia-mirror/drivers-gpu-msd-intel-gen/...

maheart 1 day ago 6 replies      
How credible is this source?

I don't understand half the decisions outlined in the article.

> I also have to imagine the Android update problem (a symptom of Linuxs modularity)

I seriously doubt the Linux kernel is anything but a minor contributor to Android's update problem. Handset developers make their money by selling physical phones. In two years, your average consumer probably doesn't care if their device is still receiving software updates. They'll jump onto a new phone plan, with a fresh, cool new mobile, with a better screen, newer software (features!), and a refreshed battery.

Maintaining existing software for customers costs handset manufacturers $$$, and disincentives consumers purchasing new phones (their cash cow). The money is probably better spent (from their POV) on new features, and a marketing budget.

resoluteteeth 1 day ago 8 replies      
Is this an actual plan of Google as a company, or is this some sort of Microsoft-style war between divisions where the Chrome team has just decided on its own that the future is based on Chrome and Dart?

Also, considering the way that the ARC runtime for Chromebooks was a failure and had to be replaced by a system that apparently essentially runs Android in a container, will it really be possible for a completely different OS to provide reasonable backward compatibility?

untog 1 day ago 4 replies      
If nothing else comes out of this, I hope we end up with an Android OS that works better than the current one.

I've been running Android since the Nexus One so I'm no newbie to the platform, but the ease with which iOS manages to get all UI interactions at ~unnoticable FPS and outstanding battery life is staggering when you're used to Android. It feels like some really fundamental choices were made badly on the platform that make it incredibly inconsistent and unreliable. A fresh start would be fantastic.

mncharity 1 day ago 2 replies      
> I dont see the average garbage-collected language using a virtual machine allowing for a target higher than 60fps realistically.

But... "average garbage-collected language using a virtual machine" doesn't describe any of C/C++, Dart, Go, Java, Python, or Rust. Nor Javascript.

I get greater than 60 fps with my existing Vive three.js WebVR-ish electron/chromium linux stack. Even on an old laptop with integrated graphics (for very simple scenes). Recent chromium claims 90 fps WebVR, and I've no reason to doubt it. So 60 fps "up to 120fps" seems completely plausible, even on mobile.

grizzles 1 day ago 2 replies      
Since Fuscia is a new kernel, that means it will probably only support Google hardware.

The status quo right now among android hardware vendors is to violate the GPL, and they have faced few if any repercussions for doing so. I wonder if Fuscia is sort of viewed as the way forward to addressing that.

Anyone care to speculate why there isn't a community version of chromium os? I'd donate to it for sure. It sounds like getting android apps working on it would be pretty easy: https://groups.google.com/a/chromium.org/forum/?hl=en#!topic...

bitL 1 day ago 4 replies      
Seems like Free Software that propelled early Internet pioneers served its purpose and those companies are turning their backs on it - first with Apple, GCC->LLVM, now with Google, Linux->Fuchsia :( I am getting afraid of another dark age on the horizon... I guess it's going to be inevitable as 90% of SW developers will find themselves redundant when inferring AI capable of composing code blocks and listening to/reading speech/specifications arrives in upcoming decade, making creation of typical web/mobile apps trivial.
techenthusiast 1 day ago 5 replies      
Article author here. Posted this in the notes, but possibly too buried:

For anyone interested, I intend to write quite often about consumer technology on this blog. Topics will include hardware, software, design, and more. You can follow via RSS or Twitter, and possibly through other platforms soon. Sorry for the self promotion!

Thanks for reading. Please do send any corrections or explanations.

thinkloop 1 day ago 5 replies      
Why can't the pure web replace apps and programs? All the pieces are almost there: hardware acceleration, service workers, notifications, responsive design...

I currently "add to home screen" for most things. I edit my images online, and develop code using cloud9 ide, etc. There are few things I need apps/programs for right now, and that's improving day by day.

iPhone is dropping heavily in world wide market share, but they still have a lot of the wealthy users. There is a non-zero chance they get niched out of prominence by Android (aka every other manufacturer in the world), at which point network effects start encouraging Android-first or Android-only development. There might be a point where Apple needs to double down on the web, and/or maybe kill off apps, like they did flash, to still have the latest "apps".

iainmerrick 1 day ago 1 reply      
"Fuchsia" and "magenta" are pretty gutsy names to choose, given how similar it sound to Apple's vaporware "Pink" OS from the 90s (AKA Taligent, AKA Copland). Somebody has a sense of humor!

It's really hard to tell if this is actually something that will ship, or yet another Google boondoggle to be swiftly discarded (like the first attempt at ChromeOS for tablets). Google under Larry Page built and discarded a lot of stuff; I wonder if it's the same under Sundar Pichai.


therealmarv 1 day ago 1 reply      
This blogpost has waaaay too much assumptions. When reading about this it seems easy to rip out Kernels, OS & Software and put it like a layer on a cake on top of a new OS. Even for Google this is crazy complicated. It will not be that easy. For sure not... and I also see no clear strategy WHY somebody should do that. It's like baking the cake with too much ingredients. ;)
Insanity 1 day ago 3 replies      
> IDEs written in Java are wildly slow

My favourite IDE to use today is IntelliJ, and I prefer it over my experience with Visual Studio (though to be fair, I did not use VS intensively in the past 3-4 years).

I don't experience IntelliJ as "slow". It launches faster than VS did when I used it, and once it is running I keep it open pretty much the entire work-week without any issues.

Nypro 1 day ago 2 replies      
So Google is going with a DartVM on this one. Dart is cool and all, but why DartVM? It's the same restrictive model we have with Android (dalvikVM) where you can only develop with languages that can compile down to Java bytecode. In this case, however, we will be using languages that can transpile to Dart source instead! Why not JavaScript engine? With the current movement with WebAssembly, I see a lot potential use cases. The biggest point being the ability to code in any language that compiles to wasm. The engine could be exposed to communicate with the OS directly or sth. If they are going to consider V8 alongside DartVM, then that would be cool. I truly hope they don't repeat old mistakes.
camdenlock 1 day ago 2 replies      
This could be the first time Apple needs to truly worry about Google. The one massive lead Apple still has over Google (and the other major players) is the incredible OS they inherited back in 1997 and continue to extend and maintain today.

Neither Android nor Windows nor Chrome OS nor your favorite Linux distro have ever been able to truly compete with the NeXT legacy as it lives on in Apple.

Google is smart enough as a whole to see this, and so it's not surprising that they're attempting to shore up their platform's competence in this particular area. What IS surprising is that it has taken them this long.

Perhaps what's truly surprising is just how much mileage Apple has gotten out of NeXT. It's astounding, and I know Apple realizes this, but I question whether or not they know how to take the next step, whatever that may be. And if Google manages to finally catch up...

endorphone 1 day ago 0 replies      
Conjecture is fun, but the linked piece takes some enormous liberties with crossing massive chasms effortlessly. Not only is Fushia not Andromeda (a project), the needs of IoT is massively different from the needs of Android. And the net investment in Android is absolutely colossal, and making some new APIs or a microkernel does not a replacement make.
wapz 1 day ago 2 replies      
I'm a minority I know but I don't like material design because it's terrible at "scaling." It looks great, it's beautiful, but you lose too much damn functionality. When I had to redo apps to material design we had to completely remove multiple buttons due to them not fitting material design standards. I really hope they have some way to alleviate this problem without using 50px icons for all the extra buttons.
sametmax 1 day ago 3 replies      
Google could have taken firefox and improve it to make it better, but they created something new.

Now instead of improving the linux stack and the gnu stack (the kernel, wayland, the buses, the drivers), they rewrite everything.

They put millions into this. Imagine what could have been done with it on existing software.

They say they are good citizen in the FOSS world, but eventually they just use the label to promote their product. They don't want free software, they want their software, that they control, and let you freely work on it.

mcguire 1 day ago 1 reply      
"The pitch will clearly be that developers can write a Flutter app once and have it run on Andromeda, Android, and iOS with minimal extra work, in theory."

How's that going to work? iOS, specifically? Is Dart a supported language?

dep_b 1 day ago 1 reply      
So in the near future billions of devices will no longer be running Linux anymore? That would be quite a blow to the OS in terms of chances of dominating operating systems that are being used by end users. Or will they simply fork it and strip it down until only the parts they really like will remain?
brianon99 1 day ago 5 replies      
Google is just afraid of GPL I think.
Flenser 1 day ago 0 replies      
yen223 1 day ago 1 reply      
Rust as a first-class language?
skybrian 1 day ago 1 reply      
re: "Flutter was [...]"

A bit weird to use the past tense here since it's not reached 1.0 yet. You can try it out today (tech preview) to create apps in Dart that run on Android and iOS:


(Googler, not on the Flutter team itself, but working on related developer tools.)

bsaul 1 day ago 0 replies      
Trying to see the other side of the coin : what economical reason is there for this project ?

A company the side of Google, with all its internal politics, doesn't work as a startup. Starting a third operating system project and hoping it to replace two major ones means convincing people inside the company to loose part of their influence. Now it may happen if chrome or android were failing, but they're clearly not.

techenthusiast 1 day ago 0 replies      
I updated the article with the following clarification at the top:

I use Andromeda equivalently with Fuchsia in this article. Andromeda could refer to combining Android and Chrome OS in general, but it's all the same OS for phones, laptops, etc. - Fuchsia. I make no claims about what the final marketing names will be. Andromeda could simply be the first version of Fuchsia, and conveniently starts with "A." Google could also market the PC OS with a different name than for the mobile OS, or any number of alternatives. I have no idea. We'll see.

shams93 1 day ago 0 replies      
As a sound/music app person the inclusion of ASIO for audio is exciting, Google's new is should be on par with iOS for sound with ASIO audio drivers at the core.
mankash666 1 day ago 1 reply      
I hope the userland is POSIX/Linux compliant. There's a TON of useful software reliant on this compliance that will go to waste if it isn't compliant out-of-the-box.
lenkite 1 day ago 0 replies      
AFAIK the flutter UI framework is a react-like framework written in Dart (with C++ as OS glue) including the UI -> graphics rendering layer. It builds upon Skia and Blink. I am not sure how that will allow compatibility with other languages. The only language for UI apps looks to be Dart. Which isn't bad - its a pretty well designed language, but I don't see how apps can be written in a wide variety of language as the author suggests.
akmittal 1 day ago 2 replies      
>the main UI API is based on, yes, Dart

Won't the Dart's single thread nature be bad to take advantage of Murli core processors? Or they are embracing web workers?

bitmapbrother 1 day ago 1 reply      
The author calls it Andromeda OS, but is this really the Andromeda OS we've been hearing about? I'm not so sure about that. What we do know right now is that the OS is currently code named Fuchsia.

Fuchsia repository:https://fuchsia.googlesource.com/?format=HTML

antoncohen 1 day ago 0 replies      
Link to the source code:


MichaelMoser123 1 day ago 0 replies      
The article says it's a microkernel, I wonder if it will be a more secure general purpose OS, well windows NT started as microkernel but they changed that wit NT 4,let's see if it will be different. I also wonder about driver support and battery consumption. Good luck to Google.
sjtgraham 1 day ago 0 replies      
> The pitch will clearly be that developers can write a Flutter app once and have it run on Andromeda, Android, and iOS with minimal extra work, in theory.

This is worrying for Apple. I can see the following playing out

- Apple continues releasing machines like the TB MBP, much to exasperated developer dismay.

- Other x86 laptop industrial design and build quality continue to improve.

- Fuchsia/Andromeda itself becomes a compelling development environment

- Developers begin switching away from Mac OS to Fuchsia, Linux and Windows

- Google delivers on the promise of a WORA runtime and the biggest objective reason not to abandon Mac OS, i.e. writing apps for iOS, disappears.

- Apps start to look the same on iOS and Android. iOS becomes less compelling.

- iOS devices sales begin to hurt.

Granted that the App Store submission requires Mac OS (Application Loader) and the license agreement requires you only use Apple software to submit apps to the App Store and not write your own, but it seems flimsy to rely on that.

conradev 1 day ago 0 replies      
Here is a link to the documentation:


avmich 1 day ago 0 replies      
I didn't see in the article explanations why those decisions were taken and not others. On the surface it feels like this is an OS insufficiently different from others to justify switching to.
pier25 1 day ago 12 replies      
We definitely need a universal OS for all our devices and I really believe Google is in a great position to get us there.

It would really surprise me if Apple got there first. Tim lacks vision and will keep on milking iOS even if the iPad Pro is a failure as a laptop replacement.

Windows is still king in the desktop space, at least as far a user base goes, but it's terrible on tablets and phones. MS has all the tech in place with UWP, but it's still pretty far in the race in terms of simplicity and usability.

Chrome OS ticks all the right boxes, and is experiencing a huge growth, but it's not universal. If Andromeda is real, and it's able to become a universal OS that merges Chrome OS and Android it might be the best thing since sliced bread.

ungzd 1 day ago 0 replies      
I think it's intended for entertainment and content consumption just like Android and Chrome OS. And Apple is trying the same with merging iOS and desktop. How long it will take until all computers will be set-top boxes where you can only netflix and chill and if you want, for example, to draw something, you have to buy Professional Grade Computer for $50000?
gillianlish 1 day ago 1 reply      
guys i hate to tell you this, but it's Feb 15 here in New Zealand, and Google has cancelled Andromeda.
kzrdude 1 day ago 0 replies      
Does anyone have a more detailed explanation of the component called modular?
am185 1 day ago 0 replies      
wow! it supports Golang, since it has glsl, this will have nice UI.


xtat 1 day ago 0 replies      
Honestly I'd rather keep the linux and ditch the JVM
HiroP55 1 day ago 0 replies      
The comment about Java based IDEs being slow is not entirely objective and fact based. I'd say it's more of an emotional argument.
BrailleHunting 1 day ago 0 replies      
NIH syndrome plus large organization people looking for job security, and rough, fat FLOSS full of maintenance and security vulns hell equals "Emperor's new clothes."
diebir 1 day ago 0 replies      
Quote: "I am not a programmer, so if anything stated above is incorrect, please, please send me corrections. I would also greatly appreciate any clarifying comments, which were one of my primary motivations for writing this piece." Essentially a bunch of nonsense, in other words.
mtgx 1 day ago 2 replies      
It would be a real shame if Google wasted this once in a decade or perhaps once in multiple decades opportunity to not have an OS written in a language other than C++.

Also, it would be mind-boggling if they didn't actually fix the update problem this time, and if it wasn't a top 3 priority for the new OS.

chatman 1 day ago 0 replies      
Microsoft is opening up more and more, and Google is closing down more and more.
Operating Systems: From 0 to 1 tuhdo.github.io
697 points by tuhdo  2 days ago   78 comments top 15
radisb 2 days ago 5 replies      
From the book:

"If a programmer writes software for a living, he should better be specialized in one or two problem domains outside of software if he does not want his job taken by domain experts who learn programming in their spare time."

Seems a bizarre sentiment, but after reading this sentence, I feel like I really wanna donate some money to the guy. If he gives a way I will surely do.

bogomipz 2 days ago 1 reply      
This looks great!

I would also like to recommend another free resource that might be a good complement(theory vs implementation) to this:

"Operating Systems: Three Easy Pieces"

available online at:


js8 2 days ago 0 replies      
Looks very nice so far. One minor nitpick - there should definitely be a chapter on inter-process communication, it's an important part of operating systems.
itsmemattchung 2 days ago 1 reply      
Every time an OS book (about weekly) is posted on HN, I immediately want to jump all over it but I gently remind myself to finish: CMU's Computer System's from a programmer's perspective and Elements of computing (nand2tetris).
koolba 2 days ago 0 replies      
For those of you that have read, or at least skimmed, this, how does it compare to the Minix book? (which was a joy to read!)
lorenzfx 2 days ago 0 replies      
How much time would it approximately take to work through this?
kriro 2 days ago 1 reply      
Seems to be for a x86 operating system. I'd have preferred some other architecture because so much of OSdev for x86 that I remember was working around quirks of the architecture (A20 gate etc.). I guess it's a valuable lesson but I'd really enjoy a fork of the book for the hardware you find in a Beagle Bone or Pi3 or something. Maybe this could be crowdfunded if the x86 version is popular?
emiliobumachar 2 days ago 2 replies      
It's a from-first-principles guide to building an OS.

From the title, I had mistakenly assumed it was about the first OS ever.

laxentasken 2 days ago 7 replies      
I have on my list to read something along the lines like this, but this seems incomplete. Does anyone have anything similar?
ejanus 1 day ago 0 replies      
I will read your book....and thanks for such a book
gigatexal 2 days ago 1 reply      
Well this has got to be one of the more ambitious things I've seen on HN. I wonder if the guy behind SkyOS is still doing operating systems. http://www.skyos.org/
jerianasmith 2 days ago 1 reply      
The hallmark of a Good book is that it should leave you wanting more. That's what the book is all about.
seewhat 2 days ago 1 reply      
On pg 38: Field Gate Programmable Array (FPGA)

Surely: Field Programmable Gate Array (FPGA) ?

lanbanger 2 days ago 2 replies      
The grammar makes that book fairly painful to read :-|
anhldbk 1 day ago 0 replies      
Nice job man!
Amazon Chime chime.aws
569 points by runesoerensen  2 days ago   324 comments top 65
niyazpk 2 days ago 7 replies      
The first (and personally the only) requirement I have with any chat system is that it should _not_ modify the text I enter in any way - especially if I am pasting something.

Sometimes I have to paste a line or two of code, or a few lines of a stack trace. Sometimes I have to paste a string which contains some particular set of characters. Microsoft Lync absolutely destroys the pasted text. It subtly converts the double quotes into some unicode nonsense. Then it converts some common character sets into smilies. When you copy text from Lync it is almost always guaranteed to be different from what what entered originally. God, I hate Lync with a passion.

ultimoo 2 days ago 8 replies      
I liked the part about Chime calling you at the scheduled start of meeting. So simple yet I had never thought of this since my org uses WebEx.

With a smart phone that would pretty well via a push notification or an actual call, but not sure how that would work when you want a join a meeting from a physical meeting room with its own AV system. I'm sure there is a way to get that set up.

krashidov 2 days ago 19 replies      
Enterprise conferencing software is so bad and so expensive I'm astonished it took this long for a decent competitor to come in. I'm really surprised Google didn't go all in with making Hangouts a decent competitor. I have a feeling this will make a lot of money.
greyskull 2 days ago 0 replies      
Amazon acquired Biba[0], this is that product with the backend swapped out. It's currently being beta'd internally and they haven't yet added anything over Biba the product. There are some great features planned, from what I've heard.

[0] http://www.biba.com/

therealmarv 2 days ago 4 replies      
I really do not like conference systems which do not work on Linux. Not everybody is using a Mac or Windows. Microsoft is also ignoring Skype and Skype for Business on Linux. This is all crap.
Taek 2 days ago 2 replies      
They tout security but I don't think it's open source and it looks like everything is stored on Amazon servers.

Minimally it is centralized, and you can't verify that there's no backdoor. In this day and age, that means we're both trusting their core intentions, and also trusting that some government won't step in and silently force their hand. I don't personally feel that is good enough to be considered secure anymore.

biot 2 days ago 1 reply      
"With Amazon Chime, you can feel confident youre communicating securely."

This wording has always struck me as being awful. People felt confident investing with Bernie Madoff as well. I'd rather have confidence from proven security instead of just feeling confident.

planetjones 2 days ago 1 reply      
Great product idea. At work we use Skype for business and it's a disaster - especially bad is it seemingly randomly says 'your device is causing poor audio quality' and mutes you. The only way to recover is to dial out and in. Before that it was the AT&T solution - such an ugly application with poor usability. If Amazon really polish this product and provide a great user experience and quality they could pick up a lot of business.

Edit: there's a problem here. Skype for business allows up to 250 participants. The AT&T solution (webex maybe) allows, I think, an unlimited number. Amazon Chime has a limit of 250 people. This wouldn't cut it for presentations in large companies e.g. announcement of annual results or divisional virtual 'town hall'

kupiakos 2 days ago 2 replies      
> Amazon Chime works seamlessly across your devices.

> No Linux support

lars_francke 2 days ago 3 replies      
I'm definitely going to try this (even though unusable for us because of missing Linux support). We have currently settled on Zoom and it's okay, they do have Linux support.

One problem I have with all video conferencing solutions we've tried (same for my colleagues, all Mac or Linux users, sadly no Windows users to compare) is high CPU usage. I have a 2015 MacBook Pro and when I share my screen CPU usage skyrockets to 150-200% basically pegging the whole CPU. Without sharing my screen CPU usage is at 80-100%.

I have similar problems with certain videos on the web (e.g. Ted.com and others).

Is this something everyone else here sees as well? I always assumed they must because we see it across devices and products.

Corrado 2 days ago 1 reply      
This is really cool but I wish they had more details on the Chat part of the solution. What does it look like? Can you theme it? Does it have any integrations (ala Slack)? Can you have inline pictures? Does it have a rich message API?
Narkov 2 days ago 3 replies      
Their claim of "a third of the cost of traditional solutions" is an apples and oranges comparison.

The basic and plus pricing options, while cheap, are practically useless with only 2 maximum attendees and the $15/user/month pro plan is hardly "a third of the cost...".

Looks like a great product with an average price point.

bobmagoo 2 days ago 0 replies      
Not sure what the long term play for Amazon is with a WebEx competitor, something likely to do with getting enterprise business, hope it works out.

In case you hadn't seen it, this is basically the anti-marketing video for how conference calls actually work: https://www.youtube.com/watch?v=DYu_bGbZiiQ

benevol 2 days ago 3 replies      
> Amazon Chime is a communications service that transforms online meetings with a secure, easy-to-use application that you can trust.

- Amazon, PRISM partner

cyberferret 2 days ago 2 replies      
Everything else aside, I am surprised/impressed to see that Amazon has the '.aws' top level domain! Does that mean they will be now branding all their AWS infrastructure under this domain?
hrayr 2 days ago 3 replies      
Are they competing with WebEx, Skype, or Slack? Looks like a compelling B2B offering from Amazon. I bet they'll have an accompanying hardware to go with this in the coming months.
jamiesonbecker 2 days ago 0 replies      
We use Zoom at Userify and love it. Fantastic Linux client, too. However, it automatically calling me (and saving me time auto-dialing auth codes) would be a pretty nice feature.
draw_down 2 days ago 0 replies      
"Meetings call you" is a good idea, as is the reconnecting stuff. Who knows if anyone will use this, but even with all the supply in this market, there is still space for something that actually works well. As someone who works remotely I can't wait until this gets figured out.
algesten 2 days ago 1 reply      
So, no video conferencing in basic/plus plan (1:1 doesn't count). It's funny how many attempts there are at making conferencing software that just have audio and some basic chat.

Entry level needs video, since you can get it for free elsewhere (i.e. hangouts).

krackers 2 days ago 0 replies      
Amazon's horizontal expansion is pretty fascinating. From an online stores and cloud service provider to consumer products and now b2b apps.
avip 2 days ago 0 replies      
I absolutely love the startupish way aws launches new services. They have the whole "landing page" and marketing pitch class A, but the product is alpha if being nice.

There's long, long way to go for this thing to compete with hangout, zoom, or anything else out there.

Source: I've just tried it out, chatting with myself on native app + 2 browsers.

manishsharan 2 days ago 1 reply      
...application available for Android, iOS, Mac, and Windows

Why not Linux?

legohead 2 days ago 0 replies      
No screenshots of the product?
dorfsmay 2 days ago 0 replies      
No Linux client? What's the advantage over WebEx then?

Both hangout and zoom can do Linux, but they aren't seen as corporate as WebEx.

zeta0134 2 days ago 2 replies      
I clicked through, and was accosted with a gigantic video. I wanted to close the tab right there, but I've seen this before, so I scrolled down to make the giant video go away. No dice, every single page element dances and animates and moves, and there don't seem to be any static images on the whole page. I can't scroll to a single position to read the actual text without some large part of my monitor animating in a suitably distracting fashion.

Why. Just... why? Why is this necessary?

JumpCrisscross 2 days ago 1 reply      
Does this feature end-to-end encryption?
nathancahill 2 days ago 2 replies      
Speaking of.. can Slack hurry up and buy Zoom? Aren't they pretty flush with cash? It seems crazy that they are letting this huge market (where they have a foothold) slip away to new competitors.
andy_ppp 2 days ago 1 reply      
This looks awesome; I'm regularly told that Amazon is a horrendous place to work yet they seem to be producing great software and interesting startup type concepts all the while. Not sure how they do it? AWS is a bit of a mess, maybe it's just there that is problematic...
pimlottc 2 days ago 1 reply      
How do they authenticate who they are when they call you? I've gotten bogus calls before from "the credit card company", so there needs to be a way to be certain you're talking to the right people.
dbg31415 2 days ago 0 replies      
Someone please do what you say and make one that's clearly better than all the other shitty dwarves out there today so the industry can standardize.

"Should we call, or Go to Meeting, or Google Hangout, or Skype, or Lifesize, or Slack, or Adobe Connect, or Zoom, or WebEx, or Chime, or..." It's getting ridiculous.

For new services: Please don't be based in the US or willing to cooperate with the US Government. Remember, "We don't snitch!" is an excellent marketing line -- I'll give you money for that. I don't trust Amazon or any of these at present.

vegabook 2 days ago 1 reply      
No Linux. Buh-Bye
webwanderings 2 days ago 0 replies      
I don't know how any of these could compete with Zoom, in terms of their offerings. Perhaps Zoom just doesn't have enough of a big name branding push, otherwise, it is hands down a product one should use over any other. I am a free-user of Zoom and I have explored many others out there; there's just no one who come close to Zoom's offerings.
po 2 days ago 0 replies      
This is the first time that I've heard Amazon really call out AWS as a name brand in a non aws-dashboard oriented product (maybe they have already in the past?) Are Chime user identities AWS IAM users under the hood?

AWS as a more consumer-facing platform probably has a long climb ahead of it but it could be quite helpful for Amazon to differentiate from their many product misses released under the Amazon name.

m_mueller 2 days ago 0 replies      
For those who tried screensharing, does it have a pointing-feature (i.e. viewer can point to something)? There's so many products out there like Skype and Hangout that don't support it and I don't understand why not, it seems pretty basic to me (just only show the arrow on platforms that support it, i.e. OSX, Windows and Linux).
vinay_ys 2 days ago 1 reply      
Super expensive dial-in rates. $0.214 per minute in India is basically twice the ISD calling rate.
Grimes23 2 days ago 0 replies      
Only amazon would reveal a product without including any screenshots or details.
dfrey 2 days ago 0 replies      
I'm so sick of proprietary walled garden messaging systems. So now I need slack, chime, skype, hangouts, imessage, allo, facebook, etc depending on who I want to talk to.
woodylondon 2 days ago 0 replies      
Wondering if you have a Plus account and setup a call if you then have remote access, group chat etc - a little unclear what happens between Plus and Basic user. if all users need to be Plus then can see this being a problem.
tea-flow 2 days ago 0 replies      
My Amazon login doesn't work. Is anyone else experiencing this issue? I just logged in on Amazon.com just fine using the same credentials (I use a PW manager). Thanks in advance.
codingdave 2 days ago 0 replies      
Screen sharing not being available for free is going to make us skip the free trial. We have plenty of options for voice and chat. And video just isn't that important to my teams. Screen sharing, however, is vital. And we are willing to pay, but as long as hangouts works for free, why pay?

I know everyone says hangouts is dead and Google isn't putting much work into it. But it does work. And unless they actually shut it down, it gives us what we need. Free. We don't use it for large webinars or anything, and it has its flaws, but... free. That is a really hard point to beat.

Roritharr 2 days ago 1 reply      
While we're on the topic of conferencing software, is there a List for Software to use when you want 4k/30p or 1080/60p?

Skype seems to not be up to the task. Our Gbit Connection is.

blintz 2 days ago 0 replies      
I'm really curious how this all-in-one concept will compare to the Slack approach of chat as the central functionality augmented by a bunch of integrations with external services.
slyall 2 days ago 1 reply      
Interesting that you pay per user per month. I wonder how it works for occasional and one-off users.

Eg if you want a vendor to join your team's chat or you use it to talk to clients.

cdnsteve 2 days ago 2 replies      
Plus plan: $2.50 per user, maximum 2 attendees - seriously, $125/mo for 50 users? I think they missed the mark.

Join.me: 50 meeting participants, $22/mo.

jcoffland 2 days ago 0 replies      
The Chime app for Android is very invasive. Instead of asking for permissions as they are needed it asks you to give up everything immediately.
alexandercrohde 2 days ago 0 replies      
One thing I'd like to hear the official policy on is message privacy (i.e. is management reading your stuff?).

That's a personal concern I have with slack.

avodonosov 2 days ago 0 replies      
No linux support?
xroche 2 days ago 1 reply      
So this is basically what lifesize.com has been providing for ages, sans the Linux support. Truly revolutionary I guess.
bikamonki 2 days ago 0 replies      
I see Amazon making a successful social network faster than Facebook making a successful market place.
sthomas1618 2 days ago 0 replies      
Zoom competitor?
malloryerik 2 days ago 3 replies      
Is Chime based on WebRTC in any way?
kr0 2 days ago 0 replies      
I hope they don't get in the habbit of releasing -ime products. Wow that's old already
fizixer 2 days ago 0 replies      
> ... transforms online meetings ...

(has no mention of collaborative white-boarding)

manuj10 1 day ago 0 replies      
UI Seems to be inspired from goto meeting.
chime 2 days ago 3 replies      
As the guy who has owned chime.tv for well over a decade, this is a bit concerning IP-wise.
hkmurakami 2 days ago 1 reply      
I noticed that this is from AWS. Is this the first SaaS application coming from AWS?
euyyn 2 days ago 1 reply      
A new player in the area! Is this the first Amazon enterprise service not for developers?
evantahler 2 days ago 1 reply      
Appear.in can't be beat.
jerianasmith 2 days ago 0 replies      
For simple and secure meetings, We should give Amazon chime a try.
thomasfl 2 days ago 0 replies      
No screenshots?
cobookman 2 days ago 1 reply      
Not sure if more impressed with the product or use of a TLD of .aws
_ao789 2 days ago 0 replies      
I didn't know there was a .aws tld..
nodesocket 2 days ago 1 reply      
How did they get that TLD .aws?
all_usernames 2 days ago 0 replies      
In soviet Russia...
happy-go-lucky 2 days ago 0 replies      
No wonder Amazon is the most innovative company of 2017.


Warren Buffetts Best Investment gatesnotes.com
649 points by jonbaer  2 days ago   194 comments top 20
hxta98596 2 days ago 5 replies      
It's hard not to be impressed by what the Gates Foundation is doing. I don't think they get the full recognition they deserve sometimes, maybe because much of their work is abroad in the poorest countries in the world.

Hard to fathom the number of people out there in the world thankful and who owe their lives to the foundation choosing to help their community or their cause.

With that said and not to diminish the good work. It is sad and eye-opening more progress has not been made on these issues.

I think when you really look at the numbers, besides seeing the positive trends one can also see how truly difficult, large and complex these problems are. The foundation has $50 Billion dollars! And Bill can get pretty much any world leader or other billionaire CEO to take his phone calls. Yet, sorry that is not enough not even close. The foundation has to focus on very specific issues and even then it hasn't "gotten to zero" where it wants to (though Polio is close, down to 37 cases). Private foundations can only do so much, the scale of these problems really requires the cooperation of governments. I'm not sure what can be done on that or what that means, just a bit breathtaking how governments can help people or really screw things up on a scale nothing else even comes close to.

OliverJones 1 day ago 1 reply      
You don't need a Gates- or Buffett- sized pot of money to make a difference. Sure, they can, and do, place massive orders at vaccine factories to get economies of scale. That's how a dose of the pentavalent vaccine comes to cost about the same as the CD that Windows used to arrive on. You probably can't place a US$50M order at a pharma company. But that's OK.

You can find a local NGO or charity that promotes, I dunno, literacy or first-generation college students or refugee settlement, or something. What matters is that they do a good job and you're interested in their mission. You can commit to supporting them with a regular cash gift. That kind of regular gift lets a charity plan their work. (One-off gifts are good too, but they don't have the planning benefit.)

But be careful: they may want you on their board of directors. :-)

cs702 2 days ago 1 reply      
To Bill and Melinda Gates:

It was really uplifting to read this post.

Because of it, the world now looks a little brighter to me.

We're all surrounded by alarmist news every day and often lose sight of the big picture.

PLEASE write a post like this one at least once a year :-)

woodandsteel 1 day ago 1 reply      
You know, we get a fair number of commenters here at HN who are cynical about the world, who say everyone with any power is corrupt, things are getting worse, and there is nothing you can do about it.

Whenever someone like that shows up, perhaps we can refer them to this letter.

wyc 2 days ago 1 reply      
I truly applaud the efforts of the funds. I don't think I could come up with any more noble endeavor, and I know there are some very capable people throughout their organizations who are dedicated to spending the funds wisely and impactfully.

However, is it fair to credit philanthropy as the sole cause of quality of life improvements across the most destitute populations?

 It's a story about the stunning gains the poorest people in the world have made over the last 25 years. This incredible progress has been made possible not only by the generosity of Warren and other philanthropists, the charitable giving of individuals across the world, and the efforts of the poor on their own behalf, but also by the huge contributions made by donor nations, which account for the vast majority of global health and development funding.
I think there is a fair amount of evidence that suggests development funding can have as much disastrous consequence as good. In his book The Great Escape[1], Angus Deaton (Nobel Prize economist) describes how non-interventionist economic development seems to have been the main driver of better outcomes across the world, and not the flush pockets of westerners, as convenient as that would be. Another book called The Road To Hell[2] provides many examples where large charitable efforts regularly produce even worse outcomes than no intervention at all. Is there some good evidence to suggest that the money spent by the Gates Foundation and other charities were solely responsible for the improved qualities of life across whole populations?

If you want to posit that their efforts have been invaluable in improving access to medical services across the globe, you'll get no argument from me. However, that's a small part of what makes up "stunning gains."

[1] https://www.amazon.com/Great-Escape-Health-Origins-Inequalit...

[2] https://www.amazon.com/Road-Hell-Michael-Maren/dp/0743227867

hans0l074 1 day ago 0 replies      
I was wondering today after reading this - what are the statistics on the next stages of the lives of these children? Saving their lives is, undoubtedly, the most important thing anyone can do. I think it's also important to see that they continue to have good lives well into adulthood (perhaps even learning about the frameworks that helped them survive as a baby?). In countries like India (where I'm originally from), child mortality rates have improved, but there seems to be this gravity well made of a mixture of superstition, religion, illiteracy, caste-based nonsense that is difficult to escape from. And this does not help translate this victory into winning the long-term war on poverty.
obilgic 2 days ago 14 replies      
I wish Bill or Warren would have run for presidency as an "outsider" instead of Trump. I guess our only hope will be their successor Mark (Zuckerberg)
wallace_f 1 day ago 3 replies      
It is an envious position to be in where you can positively impact such an incomprehensible number of human lives in the best of ways.

It's incredibly heart warming how many people have been saved by the Gates Foundation. Although, the realist in me worries about the possibility that in the long-run, civil liberties and human rights are also of importance to humanity in saving human lives, and human dignity.

Without going into an argument about the relative importance, shouldn't at least some significant portion be distributed towards solving the human and civil rights, liberties, and justice abuses here in America?

After all, the reason Bill Gates and Buffet are in a position to be able to help so many people is due the world they grew up in. The opportunities they were given. A large part of that is individual human liberty. Had Gates lived in another time, and been arbitrarily detained during one of his two arrests for refusing to unlock his electronic devices, would he have encountered a legal battle that derailed his entrepreneurship? If he was a minority, would the police had treated him the same way? If he was abused by the police the way Ian Murdock allegedly was, what would have happened to Gates? Would we have lost the opportunity to save 122 million lives to date through the Gates foundation?

erikpukinskis 1 day ago 1 reply      
I appreciate these efforts, but my mind is drawn to a somewhat orthogonal set of tactics.

When I see stats like $1 of vaccines releasing $44 of economic value, I think: there is a market opportunity here.

The question I keep asking is: what would a hedge fund look like that was making a bet that most people in poverty could produce the $44 if we invested the $1? What kind of corporate structure would be needed to distribute the resource AND reap part of the economic value?

Similar hedges can be made everywhere. If I believe black people are worth more than their social status (and commensurate credit access) would suggest.... how do I make that bet with cash?

You can answer "microfinance" but that's just phrasing the question in a different way. The real issue is: how, exactly?

It's not an easy question to answer, but I think unlocking it is a trillion dollar opportunity. You are essentially betting against the entire class of employers. Seems stupid, but so did Michael Burry's bet against the entire class of mortgage lenders.

WalterBright 2 days ago 0 replies      
Thank you, Bill, Melinda and Warren.
johnwheeler 2 days ago 0 replies      
One of the coolest things about this letter is it's presented in numbers--the format Warren appreciates the most.
gtallen1187 1 day ago 0 replies      
This is amazing. Thank you Bill, Melinda, and Warren.
justinph 2 days ago 3 replies      
Ghostery identifies 10 trackers and uBlock shows 7 advertising scripts, which seems like a lot for a page that you'd think shouldn't need advertising.

The page doesn't even display if you block some of these scripts. This is terrible.

perseusprime11 1 day ago 1 reply      
"All lives have equal value"

I hope one day all governments across the world operate with this principle.

Gates foundation is doing some amazing work. Looking forward to their contribution in the coming years.

giarc 2 days ago 2 replies      
For anyone interested in why polio hasn't been eradicated, it is likely related to the capture of Osama Bin Laden.


gadders 1 day ago 1 reply      
I see a lots of charts saying things are improving, which is great, but unless I missed it I can't see the piece that says "We spent $xxx on initiative YYY which caused this improvement."
sfblah 1 day ago 0 replies      
Rich people circle jerk. Yes I'm impressed but still.
chappi42 2 days ago 5 replies      
Nice <3

On the opposite number side, I hope they can help to find ways to reduce the population growth. It is unsustainable, especially in Africa and Arabic countries. (Pity one cannot just implement a China-like one child policy:))

toastednz 2 days ago 6 replies      
I find Gates' lack of focus on environmental issues disheartening. I know he does invest in clean energy and feels strongly about climate change, but this doesn't really do anything to prevent deforestation NOW. I know that bringing people out of poverty may have a side-effect of improving some environmental parameters, but it also makes whole swathes of people start demanding more meat, more palm oil, more cars, more disposable goods. The big international benefactors don't seem that interested in rainforest biodiversity.
Algorithms khanacademy.org
593 points by rsandhu  1 day ago   151 comments top 27
tomwphillips 1 day ago 2 replies      
This is an excellent course and helped me get my current job.

My background is chemistry/chemical engineering. I had applied for a data scientist position. Phone interview included a problem where I was asked about my solution's complexity. I admitted I didn't know about it.

Still got called back for an interview on site, but the weekend before I powered through this course. Unsurprisingly, it came up in the on-site and they were really pleased I had learnt about it. I got the job.

I also found it useful to implement all the algorithms in Python.

b3b0p 1 day ago 2 replies      
The Coursera Stanford [0] and Princeton [1] courses start again soon, February 20 to be exact. Not sure which one is better, but to refresh my atrophied CS skills of 10 years I've joined the Stanford course. Not sure how it compares to the Khan Algorithms course. Anyone have any feedback?

[0] https://www.coursera.org/learn/algorithm-design-analysis/

[1] https://www.coursera.org/learn/algorithms-part1/

SEJeff 1 day ago 2 replies      
For anyone wanting to learn algorithms from one of the other heavyweights, not being a C developer, I found this series extremely beneficial:


It also helps that Robert Sedgewick has been in compsci forever (got hit PHC in 1975) and is one of the subject matter experts in algorithms.

hal9000xp 1 day ago 4 replies      
It's strange they didn't cover dynamic programming at all. IMO every course should include at least one classical example of dynamic programming. For example:



imakecomments 1 day ago 0 replies      
As a follow up to this resource I recommend,

Algorithms unlocked: https://www.amazon.com/Algorithms-Unlocked-Press-Thomas-Corm...

CLRS: https://www.amazon.com/Introduction-Algorithms-3rd-MIT-Press...

Both include the same author as the one in this article (Thomas Cormen).

vga805 1 day ago 1 reply      
Another great resource I highly recommend: https://www.manning.com/books/grokking-algorithms
Apocryphon 18 hours ago 1 reply      
The Algorithm Design Manual by Skiena is pretty great.


It's nearly a third of the length of CLRS, and half of Sedgwick. Much more precise, yet offers more in that it talks about common problem solving uses cases with data structures and algorithms, rather than writing going through the theoretical proofs behind them.

godmodus 7 hours ago 0 replies      
I have to applaud the attempt.

And I'm kind of smirking right now, because again asymptotic got butchered.

I've spent the good part of this semester trying to get my head around a very formal, very dense script of my own algorithms course. And I finally cracked asymptotic. Maybe I'm just dense. But If that's the case, I'm sharing a classroom with others who are equally dense.

We dealt with all 5 classes, big oh, small oh, theta, big omega and little omega. We're required to always give the "most exact" classification for best/avg/worst. Including "does not get as fast as" or "does not get as slow as"

I'm willing to write a "freshman friendly" write up if someone's willing to post it or use it. I'm shit at self publishing.

bhu1st 1 day ago 0 replies      
Here is Princeton's Algorithm text I've found useful (The code is in JAVA though): http://algs4.cs.princeton.edu/home/

Pair this up with this excellent lecture by the authors Sedgewick and Wayne: https://www.coursera.org/learn/algorithms-part1/home/welcome

awesomepantsm 1 day ago 0 replies      
Talked to someone who wanted to work at one of the big megacorps as a software engineer, asked for advice to pass the interview. I asked them if they could implement quicksort. They said maybe, but they didn't really want to study algorithms. I guess they really didn't want the job after all.
ofek 1 day ago 0 replies      
Somewhat germane, https://github.com/patmorin/ods is a great resource for data structures.
malloreon 1 day ago 1 reply      
If you have the KA app installed this link opens in it!

Which is great, except it takes you to the main list of subject matters, and algorithms isn't in there.

So I'm not able to view this link on my iPad unless I uninstall KA?

fahimulhaq 1 day ago 0 replies      
We at educative.io re-published this course as a free course with implementations in Java, C++ and Python (in addition to Javascript).


One of the authors - Professor Balkcom is our advisor as well.

j2kun 1 day ago 0 replies      
I understand that the focus on sorting is to have a simple application that everyone can understand. But I can't seriously expect people to get that excited about sorting. I sure didn't. It's not like we don't have tons of other applications that demonstrate the same principles.
girzel 1 day ago 0 replies      
My high-school-age daughter is using Khan Academy to learn about logarithms for her math class. She was telling me about it, and I thought "hmm, maybe I should finally figure out what Big-O actually means". Now here we are! I guess we'll both be on KA tonight.
koolba 1 day ago 0 replies      
It's hard to pick one thing to tell budding developers they have to learn but Big-O notation is definitely up there.

The follow up to that is understanding what you're counting and why, i.e. branches v.s. statements v.s. dereferences v.s. logical I/Os v.s. physical I/Os ...

OJFord 1 day ago 0 replies      
Am I the only one to think that, for anyone capable of making it through the course, the introduction is incredibly patronising?

Not just the everyday examples of what constitutes an algorithm, but the voice, presentation, etc.

contravariant 1 day ago 1 reply      
Odd choice to start with the iterative factorial before moving on to the recursive one. Usually it's the other way around, since the iterative algorithm is faster and uses less memory.
uber1geek 23 hours ago 0 replies      
So what are some good resources for Data Structures out there, you can vouch for ?
bogomipz 1 day ago 0 replies      
I'm a big fan of Kahn and I like the addition of CS material to the site. I hope they continue to add CS material.
ankurdhama 21 hours ago 1 reply      
It is really sad that people still start the discussion about algorithms by telling it as a sequence of actions or operations to accomplish a task. How to go to airport is not an algorithm, how to cook food is not an algorithm.
evahop 1 day ago 1 reply      
Can anyone recommend an alternative introduction to asymptotic notation?
Kimalsi 1 day ago 12 replies      
as someone who doesn't know much about this and is trying to join the tech community, what will I achieve through this?
machiaweliczny 1 day ago 1 reply      
bigdataanswers 1 day ago 9 replies      
why python???? ... any language with functions will do. I mean just create a java class with all public static functions if you want it to work like python (global functions). Its really language agnostic. Your answer will be a number a string or a list of things. All languages can do that.

Im making an explicit opinion that python is no better than any other language for implementing algorithms. HN please prove me wrong in an objective way so we may all learn?

wopwopwop 1 day ago 0 replies      
Thanks for the link. However, it seems just a subset of Cormen et al.
oferzelig 21 hours ago 0 replies      
Unbelievable, a guy posts a link to some course and gets 458 HN upvotes (as of this writing).
Linus Torvalds: Successful projects are 99% perspiration and 1% innovation theregister.co.uk
615 points by oska  17 hours ago   166 comments top 31
tylercubell 15 hours ago 5 replies      
Chris Anderson: So you spoke to me last week about these two guys. Who are they and how do you relate to them?

Linus Torvalds: Well, so this is kind of clich in technology, the whole Tesla versus Edison, where Tesla is seen as the visionary scientist and crazy idea man. And people love Tesla. I mean, there are people who name their companies after him.

The other person there is Edison, who is actually often vilified for being kind of pedestrian and is I mean, his most famous quote is, "Genius is one percent inspiration and 99 percent perspiration." And I'm in the Edison camp, even if people don't always like him. Because if you actually compare the two, Tesla has kind of this mind grab these days, but who actually changed the world? Edison may not have been a nice person, he did a lot of things he was maybe not so intellectual, not so visionary. But I think I'm more of an Edison than a Tesla.


mwfunk 8 hours ago 1 reply      
I think we're all reading lots of different things into what Linus said here. My interpretation is that he's just saying that we tend to overvalue ideas and undervalue the less glamorous work involved in bringing those ideas to fruition.

The Tesla vs. Edison narrative is always couched in terms of the idea guy vs. the more pragmatic (perhaps more business-oriented) guy, not unlike the popular Woz vs. Jobs narrative, or the Jobs vs. Gates narrative in the '80s. These are popular narratives and archetypes that reflect the people involved, but can lead people to mythologizing history rather than understanding it.

In a different field, Lennon vs. McCartney.

So, among my peers at least, conventional wisdom is that Tesla was 100% an amazing visionary and got screwed over by unfair forces of history, and Edison was the villain whose contributions are overrated by historians. There's some truth there, but more than anything else it's a historical narrative where people are slotting Tesla and Edison into archetypes.

When a lot of people talk about Tesla vs. Edison, they're really just talking about those archetypes, and revealing to what degree they value inspiration vs. perspiration. I think that's all Linus is doing here, saying that in his mind perspiration is undervalued and inspiration is overvalued among his peers. I don't think he's really trying to make a historical argument, which is what a lot of the commenters here are assuming.

bananarepdev 15 hours ago 3 replies      
I deeply respect and admire Torvalds, but in the corporate world, the hypey/trendy thing helps tech people getting important changes approved by upper management, at least in my experience.I work in a tech company whose culture is primarily driven to productivity (read: getting business features delivered using the established stack and tools, as quick as possible). There is certainly an upside to it, but it produced a mindset in wich, in the face of a new problem, developers didn't even think of the possibility of bringing another tool to the toolbox. I mean seriously, to the extent of building their own xml parsers.Only recently we have been able to assemble an architecture team (i know, i hate the concept) to actually find adequate solutions to the bigger issues, and the hype is sometimes a powerful enabler to push things forward.
frederikvs 16 hours ago 7 replies      
It's funny how he says "Don't do this big 'think different'... screw that", apparently without realising that Apple did exactly what he's talking about.Almost all of Apple's big innovations were stolen from other places, and then they shut up, and got to work : making it work properly, making it user friendly, making it sexy, and then selling it.

Apple's slogan may have been "think different", and they have the image of being radical innovators, but hardly any of their innovations actually originated with them.Apple is 99% perspiration and 1% stealing good ideas :-)

There's even an infographic : http://mashable.com/2012/10/27/apple-stolen-ideas/#Fs4Q5gSS....

soneca 17 hours ago 1 reply      
The strong words of the title are not the relevant or interesting part of the article (presumably of the talk also). The success in managing the network of kernel collaborators seems to be the real story here.

"It's a social project," said Torvalds. "It's about technology and the technology is what makes people able to agree on issues, because ... there's usually a fairly clear right and wrong."

EDIT: Just for context, HN thankfully edited the title. When I wrote this the post was using the article's title: "Talk of tech innovation is bullsh*t. Shut up and get the work done says Linus Torvalds"

whsheet 17 hours ago 6 replies      
I like the header and experimented with current trends:

- Talk of AI is bullshit. Shut up and get the work done.

- Talk of Machine Learning is bullshit. Shut up and get the work done.

- Talk of VR is bullshit. Shut up and get the work done.

- Talk of Smart Contracts is bullshit. Shut up and get the work done.

- Talk of IoT is bullshit. Shut up and get the work done.

Not sure if I entirely agree with him but there's some truth.

chiefalchemist 2 hours ago 0 replies      
In the context of consumers, no one wakes up and thinks, "I gotta buy me some innovation today." NO. ONE. People just want their problems solved. That is the target. Sure, innovation might be part of the means, but innovation is not an ends (as it's often framed to be).

p.s. As a side-snark...Enough already about all these various dev technologies. So they enable still-shitty user experiences? So what. No one says, "Oh. I love they use _____."

Users. Don't. Care.

So please, for the love of God & country, stop stroking yourself with your shiny new (dev technology) object. No one cares. The technology is a means. The experience is the ends. Stop focusing on the wrong problem. Please?

altitudinous 16 hours ago 3 replies      
Upvote 1000x - I'm sick of hearing people who've done nothing and "thought leaders" ranting on. I'm pleased I put my head down, did work and did something that worked for me that I can encourage others with and give evidence. I wish those who have done nothing would go away - and stop filling my inbox, and targeted advertising spots, and meetups with their crap. I wish I could help others see over the shouting and point to what does work instead of seeing them fall for the hype every time.
anjc 16 hours ago 12 replies      
Someone who should be a multi billionaire says to ignore marketing and sales and just work.

Edit: I'm not saying that he WANTS to be a multi billionaire, but the fact is that he has attained disproportionately less value than he's created. By rights he should be one of the wealthiest people in tech. He might have 150m but that's peanuts given what he's done. The wealth of the guy who made Instagram dwarfs that. The guy who made Whatsapp has a net worth of 8b.

Creation might be 90% perspiration as he says, but perspiration doesn't equal success, and success doesn't equal a career. Obviously everything isn't about money, and Torvald's legacy will be timeless. But if you want to ensure earnings, at some point it's a good idea to sell.

aiNohY6g 16 hours ago 0 replies      
> Torvalds said he subscribes to the view that successful projects are 99 per cent perspiration, and one per cent innovation

Certainly true. IMHO, innovation is about orientation, while perspiration is about walking. They live in different timescales: GTD takes time while innovation is a spark. However, both are equally important: it would be useless to go forward in a wrong direction, it would be useless to identify a meaningful direction without going forward, and it would be of course useless to walk backward.

An acceptable - and subjective ! - balance is hard to find, these days.

theprotocol 17 hours ago 1 reply      
It takes all kinds. I say this seriously and without intent to offend: it's good to have a grouchy curmudgeon in the industry to keep us grounded.
davexunit 12 hours ago 3 replies      
Invention, my dear friends, is 93% perspiration, 6% electricity, 4% evaporation, and 2% butterscotch ripple.
mrlyc 6 hours ago 0 replies      
Good grief, not that again. One of Edison's contemporaries said that Edison would have found a working lightbulb faster if he had put more thought into his investigations.

What works for me is a series of plan -> do -> review sequences with about 10 to 15% planning, 80 to 85% doing and 5 to 10% reviewing.

gbrown_ 16 hours ago 0 replies      
The OpenBSD folk have a similar (and surprisingly polite) motto of "Put up or hack up" which I've always been fond of.
simo7 14 hours ago 0 replies      
This is one of those statements that end up being understood in slightly (but significant) different ways based on one's own experience.
happy-go-lucky 12 hours ago 0 replies      
> "All that hype is not where the real work is," said Torvalds. "The real work is in the details."

That's where the real work is, in the details. I respect those who walk the talk and he's one among them.

jest7325 13 hours ago 2 replies      
Genius is one percent inspiration, ninety-nine percent perspiration. - Thomas Edison

It's not so original and it sounds like it was copied/pasted from an old quote 100 years ago

douche 17 hours ago 1 reply      
Hear hear! Linus really is the patron saint of grumpy, cynical engineers. There's nothing more frustrating than listening to the bullshit artist spin castles made out of air, knowing that you're the poor son-of-a-gun that's going to have to do the hard work to make it actually happen, and get blamed when reality intrudes on the grand vision.

Also closes with a great quote. Code is easy, it's either right or it's wrong. People are the sticky wicket

> It's almost boring how well our process works," Torvalds said. "All the really stressful times for me have been about process. They haven't been about code. When code doesn't work, that can actually be exciting ... Process problems are a pain in the ass. You never, ever want to have process problems ... That's when people start getting really angry at each other.

jondubois 6 hours ago 0 replies      
It's 99% luck, 0.99% perspiration and 0.01% innovation.
hacker_9 14 hours ago 2 replies      
Is it just me, or is this a predictable answer from someone who works exclusively on an operating system? There is no innovation needed in his job, everything has already been made by MS, Apple etc, he just has to take existing ideas and fit them into Linux. Or even write the code that glues pieces together.

Now ask someone in the VR/AR department. Everyday they have to think up is new 'innovative' ideas because they are on the bleeding edge. We know innovation is needed because so far not everything is working.

What about Neural Nets, where there have been a lot of innovations to get from one one 'neuron' to what we now call deep learning. And the list goes on.

AceyMan 10 hours ago 0 replies      
wrt the rewritten HN headline, I've begun to prefer a different quip also attributed to Edison

"Opportunity is missed by most people because it is dressed in overalls and looks like work."

+1 for that take on innovation.

bingeboy 10 hours ago 0 replies      
Yoga is 99% practice and 1% theory. K. Pattabhi Jois
Animats 5 hours ago 0 replies      
Of course he says that. His project was a clone of something that already existed, Unix.
bingeboy 10 hours ago 0 replies      
Yoga is 99% practice and 1% theory.K. Pattabhi Jois
waynecolvin 13 hours ago 0 replies      
Trust is earned, Linus is right.
singaraja 10 hours ago 0 replies      
can 99% perspiration be automated with 1% innovation?
throw2016 13 hours ago 0 replies      
This is a social political problem especially with 'exceptionalism'. People adapt to the environment they are in, that's one thing human beings are good at.

In the US there is intense pressure right from school to colleges to work to be 'exceptional', and to be recognized and celebrated for it.

There is nothing necessarily wrong, excellence is worth pursuing and to have individuals believe they can achieve it. But there is a huge difference between motivation by passion and interest and motivation by social recognition and celebration.

There are pitfalls and side effects in a society from a toxic focus on 'winners' and 'losers', constant judgement, politics and one upmanship, the ability of people to work together without the need for self congratulation and diminishing the collective. It takes a village and all.

Excellence always comes through, you don't need to do anything special, individuals who are brilliant will always shine in a self evident way without labels or self congratulation via their work, throughout history and now and in the future.

But you can't progress alone, progress comes from a generational interlinked collective, and there is huge risk of diminishing the collective and brushing every other factor under the carpet by an extreme focus on individuals.

arc_of_descent 14 hours ago 0 replies      
I would look at it as the Zeno's arrow paradox. 99% perspiration could be again broken down into 1% inspiration/innovation, and 99% perspiration, and so on...

We need innovation simply because its fun.

Gibbon1 15 hours ago 1 reply      
Ja cause if you tried 2% innovation you'd need 198% perspiration to go with it.
thomasmarriott 15 hours ago 0 replies      
Fuck yea.
zomg 14 hours ago 0 replies      
linus troll status = epic
Ask HN: What are some books where the reader learns by building one project?
594 points by kyoob  12 hours ago   190 comments top 96
rickr 11 hours ago 7 replies      
Michael Hartl's Rails tutorial (https://www.railstutorial.org/) is a great example of this.

It'll run you through building a twitter clone and introduce you to git, heroku, a bit of CSS/HTML, and even goes into AJAX a bit.

I can't recommend it enough to people looking to get into rails.

zeptomu 9 hours ago 2 replies      
TECS: "The Elements Of Computing Systems"https://mitpress.mit.edu/books/elements-computing-systems

In this book you build a (virtualized) computer. It is one of the best books I ever read.

munificent 9 hours ago 2 replies      
What a delightful coincidence! I just posted the next chapter on my in-progress book "Crafting Interpreters" which walks you through implementing an interpreter (well, two actually) from scratch, a chapter at a time:


I really liked "Build Your Own Lisp" too. Fun book. :)

superquest 10 hours ago 3 replies      
Miguel Grinberg's "Flask Web Development" [1] is an excellent introduction Python-based web development. You build a Twitter-clone. The book is an adaptation of the authors 18-part tutorial on the same topic [2].

[1] http://shop.oreilly.com/product/0636920031116.do

[2] https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial...

skykooler 8 hours ago 2 replies      
Linux From Scratch: http://www.linuxfromscratch.org/lfs/

Goes through building a Linux system from the ground up, and gives a pretty thorough overview of why everything is working the way that it is.

yannickt 9 hours ago 1 reply      
"Physically Based Rendering" walks you through the theory behind, and the implementation of, a photo-realistic rendering system (not real-time).

An extremely thorough guide to ray tracing.


shrikrishna 11 hours ago 2 replies      
Orchestrating Docker.

I don't know if it's one of the best, but it teaches Docker concepts with a single project, and as you progress through chapters, you will find different ways you can deploy applications using Docker containers.


dzuc 11 hours ago 2 replies      
joncalhoun 3 hours ago 0 replies      
Web Development with Go (see https://www.usegolang.com) is based on starting with pretty minimal Go and web development experience and walking you through the process of building a complete web application.

I find this approach works well because you don't have to ask "why did he make that design decision" but instead I intentionally make common mistakes a beginner would make, wait until they become an issue, and then I demonstrate how we can fix that issue. As a result you really get to understand not only how to create a web app in Go but also why developers tend to follow different design patterns.

I said this in another comment, but it is based on Michael Hartl's Rails Tutorial. I think showing someone how to go from nothing to a full app is a great way to help them get into web development without the frustration that comes from piecing together blog posts/docs/trial&error.

If you are interested in Go I'd love to get your feedback :) and if it isn't obvious, I am the author of the book.

e19293001 3 hours ago 0 replies      
> Each chapter builds on the last by adding something new to the same project

I bet this is what you are looking for:


This book taught me how to write a compiler.

Here is its description from its website:

* Comprehensive treatment of compiler construction.

* JavaCC and Yacc coverage optional.

* Entire book is Java oriented.

* Powerful software package available to students that tests and evaluates their compilers.

* Fully defines many projects so students can learn how to put the theory into practice.

* Includes supplements on theory so that the book can be used in a course that combines compiler construction with formal languages, automata theory, and computability theory.

If you already know C or C++ or Java then this book is for you. In my opinion, you can learn many computer science concepts and be able to apply to your field. The book will teach you how to write a grammar then write a parser from it then eventually be able to improve it as you go on reading and doing the exercises. It was a great moment when I feel comfortable writing recursive functions since grammars are composed of recursive functions. You'll also learn a nice way on how you can get your compiler to generate assembly code. Another feature of the book is the chapter on Finite Automata wherein you'll learn how to convert between regular expressions, regular grammars and finite automata and eventually write your own 'grep' which was for me is a mind-blowing experience. There are lots of other stuffs in this book that you could learn. Thank you Anthony J. Dos Reis for writing great books for people like me.

DrPhish 10 hours ago 2 replies      
If you are interested in assembly language concepts and don't mind something slightly archaic, I thoroughly enjoyed this book: https://www.amazon.com/Peter-Nortons-Assembly-Language-Book/...

It goes through lessons that build up to a pretty good interactive disk editor (DSKPATCH) written entirely in x86 assembly.

Its the book that got 12 year old me out of the BASIC ghetto.

saeranv 4 hours ago 0 replies      
This is a slightly different response from others, but I think fits the intent of the question: The Better Explained Guide to Calculus: https://betterexplained.com/calculus/

Kalid basically iterates the series around the concept of deriving the formula for the area/perimeter of a circle, and then builds up to deriving the surface area/volume of a sphere. The focus throughout is the building up of an intuition of calculus before leaping into formulas. Even with uni-level calculus, I did strengthen my intuition of what's going on by reading through his book.

It's pretty fun, and I actually spent some time visualizing the calculus of geometric solids afterward i.e http://www.trinco.io/blog/derivative-of-x3

chromaton 10 hours ago 4 replies      
The Build Your Own Metal Working Shop from Scrap series by David J. Gingery.
iainmerrick 10 hours ago 1 reply      
Aaron Reed has a terrific book, "Creating Interactive Fiction with Inform 7" (http://inform7.textories.com) that walks you through building a complete (and fairly sophisticated) text adventure.

Inform 7 is very much a niche programming language, but it's really interesting and unusual, well worth investigating if you want to broaden your horizons. Vaguely Prolog-like, but written in natural language.

toss1941 9 hours ago 1 reply      
I remember seeing a C++ (Borland) book that was entirely about building a flight simulator. Google is betraying my attempts to find it though, and i'm not sure who the actual publisher was. That was during the 90's when I was 14 or something so it was a bit above my head, but I remember almost buying it anyway.

Edit: It may have been this: https://www.amazon.com/Build-Your-Own-Flight-Sim/dp/15716902...

plinkplonk 10 hours ago 1 reply      
A Retargetable C Compiler: Design and Implementation


A most excellent grimoire.

nkassis 10 hours ago 0 replies      
Not a book but Casey Muratori's Handmade Hero series is really interesting. It's a from scratch tutorial on building game in C on windows but delves into many interesting programming topic that would be useful outside game development.


thewhitetulip 11 hours ago 1 reply      

You can learn how to build a todo list manager in Go.

This book is about building a webapp from scratch without using a framework.

laser 34 minutes ago 0 replies      
This was on HN recently, but I really enjoyed Robin Wieruch's react tutorial in which you build a HN like app: https://github.com/rwieruch/the-road-to-learn-react
dbcurtis 2 hours ago 0 replies      
Make: Analog Synthesizers

Build an old-school analog music synth. Very DIY friendly, and great for electronics n00bs. The book is build around a project call the Noise Toaster, but you learn all the analog synth basics along the way. Fun stuff. Old school, and it isn't a wall-sized rack of Moog modules, but hey, good humor.

PaulKeeble 7 hours ago 0 replies      
The classical Design Patterns book has a first chapter which takes you through the design of a text editor using the patterns provided in the book. If what you do is read the chapter and then the patterns referenced as you go and build the text editor based on their design you get exactly the sort of thing you are looking for. Its a different way of doing it than the entire book but arguably just in a different format for what is otherwise a reference book.


parisidau 3 hours ago 0 replies      
My book "Learning Swift" does this.

Swift 2 version: http://shop.oreilly.com/product/0636920045946.do

Swift 3 version: http://shop.oreilly.com/product/0636920053989.do

We build a note-taking app for iOS, macOS, and watchOS.

r_singh 10 hours ago 1 reply      
Eloquent Javascript - http://eloquentjavascript.net/

You build 5 projects through the book - a programming language, paint program, a dom game and a skill sharing website using node js.

jimmies 10 hours ago 0 replies      
Hacking the Xbox by Andrew 'bunnie' Huang. (http://hackingthexbox.com/) The book is free and can be downloaded online.

It goes from adding the LED to the XBox to tapping the security mechanism. Plus, the original Xbox is cheap nowadays too, so you won't have to shell out a lot of money doing it. Local craigslist should have plenty of them.

jdc0589 10 hours ago 0 replies      
so, this isn't exactly what you are after; but, I've always had a bunch of little exercises and projects that I just kind of go through in a new language/platform as a learning exercise.


1. given a large file, or set of files, write a program/routine to count the number of times an arbitrary sequence of characters appears. No regular expressions or other pattern matching helpers from a library/sdk, you have to do it all yourself. This one is pretty small, but there's lots of opportunity for optimization.

2. build a link shortener service with some analytics/tracking.

3. write a simple tokenizer for whatever syntax/language you feel like. JSON is a super easy one.

4. write a little website crawler. multithread it. implement rate limiting (something more advanced than random sleeps; e.g. token bucket, etc...).

5. make a couple easy data structures yourself. If the language/platform you are working in has the same structure in an SDK (or there's a good open source one), write yours to the same interface and then run it through their test suite. e.g. linked lists, queues, etc...

jetti 12 hours ago 1 reply      
Programming Phoenix Productive |> Reliable |> Fast. It goes through building a website that starts from the beginning and goes up to Phoenix Channels (websockets) for a real-time video streaming application that lets users comment on the video at specific times and it is broadcasted to all other users.
corysama 9 hours ago 0 replies      
Ray Tracing in One Weekend

Ray Tracing: The Next Week

Ray Tracing: The Rest of Your Life


detaro 12 hours ago 0 replies      
Physically Based Rendering From Theory to Implementation doesn't necessarily have the reader build a project, but it explains its reference implementation very well, and you could go ahead and write your own in parallel based on it. http://www.pbrt.org/
larc 9 hours ago 1 reply      
Test Driven Development with Python (http://www.obeythetestinggoat.com/) teaches you to use Django and TDD practices to create a basic list web app.
khedoros1 9 hours ago 0 replies      
I bought this a year or two ago: https://www.amazon.com/Tricks-Programming-Gurus-Advanced-Gra...

It's basically a set of tutorials that lead you through the steps of building a software 3D graphics rasterizer. It covers rasterizing, lighting, shading, shadows, textures, etc, and the math behind each set of concepts. It's built on late-90s C and DirectX, but the capabilities used are covered by just about any game programming library. The author builds kind of an abstraction library on top of the DirectX code, and that's pretty easy to rewrite in whichever language and toolset you're comfortable with.

crasm 10 hours ago 2 replies      
I haven't bought or started this yet, but this book on interpreters is on my todo list: https://interpreterbook.com/
lanekelly 8 hours ago 0 replies      
I used the older React+Webpack version of https://survivejs.com/ to learn React recently. You build a small Kanban/Trello webapp.
Hernanpm 11 hours ago 4 replies      
arenaninja 11 hours ago 1 reply      
Not exactly a book but gitlet.js: http://gitlet.maryrosecook.com/docs/gitlet.html

About how to build a subset of git's functionality in NodeJS

nickhalfasleep 6 hours ago 0 replies      

Radiosity: A Programmer's Perspective by Ian Ashdown is a full numerically accurate hemicube radiative transfer engine from start to finish. Now a free pdf.

wazoox 10 hours ago 1 reply      
Bill Hails implements Scheme in Perl, maybe interesting to compare the two books:http://www.billhails.net/Book/
quantumhobbit 11 hours ago 0 replies      

There is an associated book. Great intro to the fundamentals of computer engineering.

sobinator 9 hours ago 0 replies      
'The Art of Monitoring' by James Turnbull (https://www.artofmonitoring.com/)
deletia 10 hours ago 1 reply      
lugus35 6 hours ago 0 replies      
Practical Common Lisphttp://www.gigamonkeys.com/book/

Filter spam, Parse binary files, catalog MP3s, stream MP3s over a network, and provide a Web interface for the MP3 catalog and server.

blairbeckwith 11 hours ago 0 replies      
Michael Hartl's Rails Tutorial (https://www.railstutorial.org/) leads you through Ruby, Rails, Git, and deployment to Heroku through two very small projects and one related larger project that takes up the bulk of the book.
cpcat 9 hours ago 0 replies      
Ray Wenderlich's online tutorials on How to Make a Game Like Candy Crush https://www.raywenderlich.com/66877/how-to-make-a-game-like-...
vmware513 8 hours ago 0 replies      
For learning JavaScript and Ember.js: http://yoember.com
wyc 7 hours ago 0 replies      
I remember The Unix Programming Environment by Kernighan and Pike as an excellent introduction to Unix shells and general Unix programming. IIRC, throughout the chapters, it has you build some kind of interactive command line music organizer, which really demonstrates how much you can get done with a few simple shell scripts.

Amazon link:


jenkstom 7 hours ago 0 replies      
Programming from the Ground Up teaches x86 assembler from first principles. All on a free OS (Linux using asm). There is a free version available online, but if you want to support the author you can buy it from amazon. https://www.amazon.com/Programming-Ground-Up-Jonathan-Bartle...
mavelikara 8 hours ago 0 replies      
Jekor's Haskell tutorial videos: Redo implementation in Haskell.


rokob 8 hours ago 0 replies      
Erlang and OTP in Actionhttps://www.manning.com/books/erlang-and-otp-in-actionYou build an application with increasing complexity. There are some other applications that are built along the way to demonstrate some ideas in a smaller scope, but the overall structure of the book is designed around making one application better as you learn more about OTP.
dyukqu 9 hours ago 0 replies      
For HTML & CSS: http://learn.shayhowe.com/html-css/ [0]This is the project you build as you learn: http://learn.shayhowe.com/practice/organizing-data-with-tabl...

[0] There's an e-book version too. Scroll the page down just a little bit.

ChrisKingWebDev 7 hours ago 0 replies      
A bit late to the party, but I've been making by way though "Developing Games With Ruby" (https://leanpub.com/developing-games-with-ruby/read)

It's really good and the game is actually pretty fun.

yisheng 11 hours ago 0 replies      
I really liked The Elements of Computing Systems (nand2tetris). The reader gets to build a computer from nand gates up.
nether 9 hours ago 1 reply      
Accelerated C++ involves working on a program to analyze student grades. The book itself is excellent, but the overarching project is very dry.
matthiaswh 3 hours ago 0 replies      
Coincidentally I'm about halfway through writing a long Vue.js tutorial series where I build one project in this manner. Would love to hear more ideas about what people look for with this type of book.
pjungwir 10 hours ago 0 replies      
Code by Charles Petzold takes this approach to explaining how a computer works. By the end he has a working system with CPU, RAM, inputs, and outputs.
dragthor 5 hours ago 0 replies      
Big Nerd Ranch books - at least the Android one


distantsounds 8 hours ago 0 replies      
django's tutorial has you creating a polling website: https://docs.djangoproject.com/en/1.10/intro/tutorial01/
peterarmstrong 4 hours ago 0 replies      
I wrote Flexible Rails back in 2007 that did this:https://www.manning.com/books/flexible-rails

I wouldn't read the book now though, since Flex, but the approach worked well.

kevinclancy 11 hours ago 1 reply      
Modern Compiler Implementation in ML by Andrew Appel
coleifer 9 hours ago 0 replies      
I co-wrote a book on Flask that walks through the building and progressive refinement of a blog: https://www.packtpub.com/web-development/learning-flask-fram...
nedwin 10 hours ago 1 reply      
KLF's The Manual is a guide to producing a number one hit.
drharby 8 hours ago 0 replies      
C++ design Patterns and derivatives pricing by Mark Joshi

He kinda takes a "Defense of Duffers Drift" approach towards designing iterative versions of the same project, slowly introducing concepts such as factories and singleton.

bcoughlan 6 hours ago 1 reply      
Not a book, but I loved this series about building a Pascal interpreter (currently 12 parts): https://ruslanspivak.com/lsbasi-part1/

It was always on my bucket list to learn to write programming languages, but it's very daunting - this made it easy to learn in bite-sized chunks.

vivekd 10 hours ago 0 replies      
Automate the boring stuff - Al Sweigart.

This book teaches you python through a series of example projects. You can get it online or order a physical copy and help support the author here:


contingencies 5 hours ago 0 replies      
More hardware oriented I would suggest some of the new Raspberry Pi or Arduino based electronics kits, which are similar to books in that they provide a prescriptive path in to comprehending new areas through the satisfying and hand-on exploration of a defined project. Programming is involved too.
chris_st 1 hour ago 0 replies      
waterfowl 10 hours ago 0 replies      
Discover Meteor is like this, involves building a reddit clone type thing called Microscope(a reduced version of an actual product Telescope).


dugmartin 7 hours ago 0 replies      
I always thought "Building X in Y" would be a great brand for a series of books. There was a dBase III book in the mid 80's that I loved that took you though building an invoicing system for a fictional company. It wasn't just about learning the language/runtime but rather took you though the author's thought process and outlined the pros and cons of different approaches and db schemas.
akaru 5 hours ago 0 replies      
A mostly dead technology, but does teach some general programming in the way you're asking: https://www.amazon.com/d/0596805632
algebraicgeo 8 hours ago 0 replies      
Are there such books for algorithms?
sigmundritz 7 hours ago 0 replies      
Richard Stevens' UNIX Network Programming books: http://amzn.to/2lmH8hy The 1st volume at least is going through a simple TCP/IP telnet server and enhancing it with features as you go (single connection, multiple connection, forking, multi-threaded server), etc.
dom96 8 hours ago 1 reply      
My Nim book[1] is sort of like this. Instead of working on a single project, most chapters have a little project (for example a chat CLI app or a Twitter-like website) which teaches you different things about Nim.

1 - https://manning.com/books/nim-in-action?a_aid=niminaction&a_...

myth_drannon 8 hours ago 0 replies      
Implementing Lisp in Python - http://fogus.me/fun/lithp/
vvoyer 6 hours ago 0 replies      
PHP and MySQL by example: you build an IMDB like website in a very progressive and beginner manner.


grimoald 8 hours ago 0 replies      
OK, you don't build a complete project, but it's a similar approach:

Think Stats: Probability and Statistics for Programmershttp://greenteapress.com/thinkstats/

Here, you learn statistics by implementing statistics functions in Python along the way and use them to solve the questions in the book.

qz_ 5 hours ago 0 replies      
TeMPOraL 8 hours ago 0 replies      
Maybe not best in the "build a project" genre, but about (Common) Lisp, you have Land of Lisp - in which you build several small games and sort of reuse some code between chapters, and Let Over Lambda, which does build its further code on top of earlier code, although it's more of an exercise in bending Lisp to your will than a well-defined project. I recommend both books; I found them very good.
paullth 7 hours ago 0 replies      
Ben Rady's Serverless Single Page Apps is one I've enjoyed recently, maybe not something I'll use at work anytime soon though.


jonaldomo 10 hours ago 1 reply      
Anything by Manning that ends with "In Action". I volunteer to be a technical reviewer if there is a new technology I want to learn (and get a free physical book!)
rb808 11 hours ago 0 replies      
Professional Java for Web Applications. If Java is your thing.
sfifs 9 hours ago 1 reply      
I learned C++ back in the 90s using a book called "Type and Learn C++" - it involved building a text editor.
evilc0 8 hours ago 0 replies      
Wondering why no one mentioned, LFS :)It's not directly about building something in a concrete programming language but it's about build your own linux system.http://www.linuxfromscratch.org
__strisk 9 hours ago 1 reply      
I remember liking http://www.tangowithdjango.com/ .

It goes over building a link aggregating service using Django.

However, I also remember it being free. Doesn't seem to be an option with the most recent release.

msangi 7 hours ago 0 replies      
Crafting interpreters. It's still a work in progress, but the chapters published so far are very good.


dump121 10 hours ago 1 reply      
Any such book for distributed systems?
cestith 8 hours ago 1 reply      
"The Art of the Metaobject Protocol" leads you from a very thin layer on top of CLOS to a very rich layer of objects-oriented object implementations. It's a classic.
douche 10 hours ago 1 reply      
Two decent ones for game development:


Walks through building a JRPG-style game with Lua. Pretty impressive for the price, especially with all the royalty-free assets that are included.


A little dated now, (although DirectX 9 isn't quite dead yet), but this one has some pretty interesting topics. Good chapter on procedural terrain generation, some basic pathfinding, minimaps and fog-of-war.

diminoten 10 hours ago 3 replies      
I dunno why, but I was kind of hoping there would be non-CS answers for this too, like how to woodwork, or like tying flies.
hariis 6 hours ago 0 replies      
Anything for mobile app development?
kornakiewicz 7 hours ago 2 replies      
Missing Clojure example here. "Clojure for the brave and true" was a little disappointing.
drproktor 8 hours ago 0 replies      
MEAN Web Development - great book, really nice to kick start with the MEAN stack.


innocentoldguy 9 hours ago 0 replies      
I liked Take Off With Elixir, by Rob Conery. It walks you through writing a program for a fictional aerospace start-up. It also comes with a 3-hour companion video, which is good too.


tmaly 11 hours ago 0 replies      
I had an interface electronics course back in the day that was like this.

Bandit Algorithms book is sort of like this. starts out simple and touches different methods

ryan-allen 4 hours ago 0 replies      

Build your own legit Analog Synthesizer!!! Not software but it is an end to end project. Good little book.

shinta42 9 hours ago 0 replies      
i love this topic, i hope to learn much soon
40% of foreign students in the US have no close American friends on campus qz.com
408 points by pmcpinto  1 day ago   281 comments top 63
habosa 1 day ago 18 replies      
It seems obvious that a huge (and hard to solve) reason for this is simply language! There are many levels of language proficiency:

 Level 1: Can read and write it at your own pace Level 2: Can comfortably converse in a professional setting (little slang or cultural knowledge needed) Level 3: Can comfortably converse in a social setting (slang, faster speech, less clarification) Level 4: Can do all of the above passively, being able to pick up valuable information just by overhearing conversation without focused metal effort Level 5: Ability to do all of the above in a noisy and hectic situation (like a party, sporting event, etc)
It's really, really hard to make good friends without getting to Level 4 or 5. I work with many people for whom English is a second language. At work they don't miss a beat and are great teammates. But it can fall apart in a social setting. Once there are 5 overlapping streams of conversation stuffed full of cultural references these coworkers of mine very frequently lose track of the conversation and become quieter and quieter over time.

I bet you'd find that this 40% number is much different for students studying abroad in a country where their native language is common. My US friends who moved to the UK or Australia had no problem making friends.

Bartweiss 1 day ago 5 replies      
People should know the Quartz headline is wrong.

Follow the source link and you find that 40% have no close American friends, which is different from "friends on campus". I checked the source looking for a baseline (how many domestic students have no close friends) and discovered that it was specifically about international students making friends with Americans.

> "Nearly 40 percent of the survey respondents had no close American friends and would have liked more meaningful interaction with people born here"

This is a very different result - still important, but the corrected stat and the free-response listed in the source make clear that we're looking at a different question than simple loneliness.

edit: The HN headline has been updated, which is great news. Now if only Quartz could meet the same standards...

beeftime 1 day ago 2 replies      
Before I got into software development I worked for a South Korean company that helped high-level students get into American and Canadian universities. The single biggest source of stress for myself and the other westerners working there is that the administration's (and the parents'!) only goal was to get these kids into a school, and absolutely no thought was given to acclimating them to a different culture or teaching them the skills they would need to thrive in a very different educational system and social climate.We (the foreign teachers) would try to sneak cultural lessons in and hold extra classes about how to join campus organizations and social clubs, and we'd always get in trouble with the management because "it wasn't important".There's a very real sense in East Asia that once you get into an American university that you'll become successful, but some of the brightest kids are coming back to their home countries with a 2.7GPA, a memory of crushing loneliness, and many tens of thousands of dollars wasted. It's a solvable problem that no one is really interested in even examining.
jusob 1 day ago 1 reply      
I spent a year in US as a foreign student. From my previous experiences (internships in UK and Germany), I knew it would be difficult to make friends with "locals". In UK, I was in a dorm during the summer with a private room, shared bathroom and kitchen. Most people were eating in their room. I finally made friend with one british guy as he was cooking. He told me later he didn't think he would make friends during the short summer session, even less with a foreigner. His girlfriend owned a pub, I had a great summer!

In the US, I didn't realize it would be even more difficult because 90% of the students in MS are foreigners in the college I attended. Anyway, I decided that the best way to make American friends was to be with them all day long. I joined a fraternity on campus.

This was not easy. Most fraternities never had a foreign student, except the one I was accepted in. Because my english was not great, I focused all my time on 1 fraternity to increase my chances of being accepted. I was the only foreigner in all fraternities this week.

It was not always easy, but it was worth it. I joined while doing my MS, all my brothers where freshmen. We had very different work load and about 4-year difference. But I did make friends with all of them. I spent Christmas with one of my friend's family (I didn't leave the US for winter break). It was a great experience.

I'm rather an introvert. But when you travel in a foreign country, you have to talk to strangers all the time. Expectations are also lower when you don't master the language fully. You have to be very direct and explicit in your communications.

wizardforhire 1 day ago 1 reply      
Fwiw 10 years ago I was organizing weekly dance parties in a college town. 80% of our attendance was foreign students. Nightly I would have kids from all over the world profusiosly thanking me. The bulk of the complements were in one way another "thank you so much for putting this on I feel like I'm home. I don't get American culture but I get this." 10 years later I still have people coming up to me randomly thanking me saying they've stuck around because of friends they made.

In my limited experience I feel there's a real cultural mis-match with kids from other countries coming here to study. Most of the kids I met were from huge cities and the shock of being in a small college town was in many was too much for them. That and the greek system was overtly hostile to them.

On side a note it's humorous what many kids take back with them from their time in America. It's worth doing an image search of "American party" to see what I mean.

pm90 1 day ago 8 replies      
As a former international student, the first year was incredibly lonely as I used to be a deep introvert. The article mentions some of the obstacles but perhaps one of biggest ones is being able to communicate well with Americans. I used to be incredibly anxious about buying groceries simply because I talked very fast English with a heavy accent. I could talk but not communicate; which means that I could convey facts, but I couldn't strike up a conversation with a stranger, or make a joke. I didn't understand American sarcasm and would be alarmed by some of the things Americans said. And this is despite me having pretty good knowledge of English; I can only imagine how hard it must be for Chinese/Korean etc. students.

The best thing that happened to me was to get an internship in a company where my team was composed mostly of Americans. I learned how to talk slower and more importantly, slowly understood sarcasm as well. Perhaps most Americans don't realize just how much of a shared culture is needed for immigrants to understand before they can communicate effectively.

JoeAltmaier 1 day ago 1 reply      
I sometimes work with foreign engineers, often educated in America and working their first or second job. My wife and I often invite them and their spouse home for a dinner.

In every case, they mention sometime during the evening that its the first American home they have ever been inside. After years of school and job. Every, every case.

Americans, we can do something about this! Invite a newcomer coworker to join you for dinner! Its so simple.

ChicagoBoy11 1 day ago 0 replies      
I moved to the US when I was 12 and my middle school principal tried finding someone who spoke my language that I could shadow. She couldn't. Little did I know then, but I think that it might have been the thing that had the biggest outcome in my success in the US.

There were other kids who joined the school from other countries right around the same time as me (we all did ESL together), but they all spoke languages that were highly represented in the school. Over the years, it was incredibly noticeable to me how insular they ended up -- hanging out mostly with expat friends, speaking their native language on breaks, etc.

Meanwhile, I had to try and make American friends any way possible -- which for me was through our school's robotics club (and since I am typing this here, you can guess that the rest is history)

I completely understand the way immigrants rightfully treasure and celebrate their heritage. But I have always found it puzzling -- especially in college -- to see people from overseas mostly hanging out with their own.

I have heard far too many times how "cold" Americans are, how they aren't friendly to foreigners, etc. At least in my experience, that could not be further from the truth. What I HAVE observed is foreigners like myself failing to leave the safety of their known communities and fulling embracing the experience they supposedly came here for.

Semaphor 1 day ago 0 replies      
Just an anecdote from a medium sized German town:

We had two big bunches of foreign students, Chinese and US Americans. Americans organized a lot of parties, the Chinese were hard to engage with. We had a big international community (language exchange regular meetings, parties, movies, BBQs, etc.) But the general thing from the Chinese group was what seemed like shyness. Even when one managed to get them to join, they tended to leave early and barely interact (not for a lack of trying). There were exceptions of course, and I'm currently subletting my apartment (while in another country) to two Chinese students, one of whom turned out to be very talkative once he opened up. But for the majority I met it's really hard to get them to get them to open up. There is some cultural barrier that's very hard to break.

Of course this might be the same for other nationalities, but as those usually arrived here alone, they didn't have a group of countrymen to fall back to and I couldn't tell.

010a 1 day ago 5 replies      
I can't generalize for every school, or everyone at every school. I attended an engineering program at one of the top 5 schools by total international student population.

Honestly, international students are cliquey. Many of them that I talk to openly admit to cheating on their english proficiency exams universities require you to take before you can attend. Meshing with local students is nearly impossible if you don't understand the language proficiently.

I'd expect you'd see roughly the same numbers if you looked at American students in Chinese universities, or elsewhere. But we have to make this anti-American because its Quartz, and Trump is bad, right?

socrates1998 1 day ago 1 reply      
I am curious as to what the ultimate goal is for these foreign undergrads. Get a job in America? Go back home and get a job?

I guess it all depends on the home country and the American school, but I wonder how valuable an American undergrad degree is worth in their home countries?

I know in Japan it often isn't seen as worth it because in Japanese undergrad programs, the people you meet often are a major part of your network along with the people you went to high school and middle school. And these networks are essential to your career arc.

So, by going to school in the US, you lose out on these networks.

But, if your goal is to get a job at an international company where you speak English or get a job in the US, then I guess it is worth it.

booh 1 day ago 0 replies      
You can become extremely proficient in the language and be highly sensitive to the culture. You can also spend 10 years in the country to the point were you grasp practically everything, no matter how much randomness is involved in the conversation or the place and time it takes place.

It does not matter. Majority of foreigners, westerners or not, won't have real local friends in English speaking countries.

Even though you do acclimatize to the culture and language you might still not be fond of it. I lived in countries were it would have been easier than in others to interact with people, but in the end I wasn't able to because I either wasn't a fond drinker (commonplace in all english speaking countries) or I didn't enjoy being involved in mundane silly-office conversations during the smoke break.

Most of the fresh expats can't even realize what they are getting into when they move into another country. If you are deciding to do so and you come across this post, do it, go and check it out, but beware that your inner you will never completely mold to that place.

b6 1 day ago 0 replies      
It takes a lot of courage to put yourself in a strange environment, away from familiar comforts and friends and family, into the care of strangers, to learn about a million faults they have, and then learn to love them anyway, and, most importantly, to really allow the experience to change you.
ashwinaj 1 day ago 1 reply      
What does close friends mean? A lot of relationships in the US are based on "activity" partners[0]. Someone to go out with, sharing a same hobby, passions etc. Close friends in other countries means a lot more than just activity partners.

I had the same experience in grad school, I only had "activity" partners, than what I would call close friendships with Americans. It always felt "distant". It's sadly true even today.

[0] I couldn't find the relevant article that discussed about this, specifically in the American context.

jccalhoun 1 day ago 2 replies      
I teach at a midwest college and I worry about my international students. Some of them do great but too many of them struggle and I don't have the skills or even the time to help them. I try to make sure that they get mixed into groups with the students from the USA in the hopes of encouraging the creation of friendships but I often feel like the university is just taking their international tuition and not doing enough to help them.
snvzz 1 day ago 1 reply      
> 40% of foreign students in the US have no close friends on campus

Missing important data for contrast: How many non-foreign have no close friends on campus.

lazaroclapp 21 hours ago 0 replies      
Most foreign students are in the US for a two years masters program, which is not that long a time to assimilate to any culture. I am from a relatively similar western culture, I arrived here thinking to myself in English 50% of the time (not my native language, but my main "Internet language"). I still spent my whole first year with my closest group of friends consisting chiefly of Germans, Greeks (from actual Greece, not a frat house), Indians and Chinese. It wasn't that it was easier to breach that cultural gap, but more that other international students were aware that there was a gap to be breached. Americans seemed always more likely to only understand overtures of friendship in their very specific cultural context, and even with extensive American media exposure it takes a few comical mishaps to become fully conversant in that cultural code, independent of language. English fluency was the lesser of the issues for me, but admittedly having a strong accent can make it even harder to be understood at a loud party even if it is perfectly fine for asking questions in class. I don't blame the natives at all for it, but do keep in mind that it is the foreign students who often do 9 parts in 10 of the effort. Wonder what is the statistic - even for colleges that are 50/50 overall - of "American student with no foreign student close friends".

For the curious: I did end up having a much closer group of friends including roughly 50% American students later, though, after bonding over one of the most American experiences possible, a Spring break trip.

raisspen 1 day ago 0 replies      
It is the same for almost anybody studying in a foreign country. It takes a fairly high level of self-confidence to really put yourself out there and cultural/linguistic differences can be confusing to navigate. I myself have studied in 3 foreign countries and can say that it requires a lot of effort to make local friends.
wallflower 1 day ago 0 replies      
See also "When a Chinese PhD Student Meets a German Supervisor "https://news.ycombinator.com/item?id=12769385
dagw 1 day ago 0 replies      
I wonder if they tried to normalise how different people and different cultures define what is meant by "close friends". I mean I'd probably say that I've made maybe 4 close friends in my entire life (5 if you count my wife), half of which I've lost contact with, without ever really feeling I'm lacking friends, acquaintances and people to hang out with.
sametmax 1 day ago 0 replies      
So 60% DO have close friends that have a very different culture and language. I find that very positive. I traveled all my life, and I know how hard it is to do it. Well done !
kop316 1 day ago 1 reply      
I would like to frame my thoughts on this as an American that is dating a Taiwanese person, and as someone who has traveled around the USA a lot. I also ran my thoughts by her and she agreed with what I said.

First, I think there are two types of people who travel to a new place (this can be a different part of the country, or a foreign student to the USA): 1) the person who wants to try new things and enjoy where they are, or 2) the type of person who hates where they are and refuse to try anything new.

The latter type of foreign student will not make any American friends. They will only stick with their culture and friends of their culture. She has also said a lot of people feel very nervous speaking English, because they will they will get something wrong and get ridiculed for doing so.

As an American with foreigners in my classes, there are also people who speak their native language to other students who speak it. I empathize with the fact that it is easier for them, but by doing that, the effect that I get is the feeling of exclusion, so I cannot even attempt to try to befriend them. As an anecdote to that, I can say I have felt resentful when I have been the only American in a group of Taiwanese and they did not speak English at all when I was there, as the message I got is they don't want me in the conversation.

I hate to say, it is very intimidating, but if you want to befriend Americans while in the country, you HAVE to speak English as much as possible. I am, and I would like to think any others are very forgiving in the fact that they know English isn't your first language, and are happy to accommodate that. If you do not, most Americans will feel excluded and not even attempt to befriend you.

morgante 1 day ago 0 replies      
Did they compare against American students though? Modern culture is making everyone increasingly lonely and isolated, so I wouldn't be surprised if a good portion of that number is explained by the fact that going off to college and making close friends can be difficult whether you're from Nebraska or Nanjing.

That being said, the cultural aspect can't be ignored. There's a reason that despite going to two very international schools, almost all of my close friends are European, American, or Australian.

1024core 1 day ago 1 reply      
When I came to the US, I made an effort to reach out to Americans (in the dorms, in class, etc). I found that, after you remove the veneer of programmed culture and biases, under it we're basically all the same: similar goals, fears, aspirations, etc. I formed some very close friendships with Americans, and I'm still close friends with them 20+ years later.

My point is: as a newbie in this country, I had to make the effort; if I didn't do that, I would not have made those friendships.

popobobo 1 day ago 0 replies      
40% of US students in the US have no close American friends on campus too. We all went to college. Come one. That is how it is. I wouldn't call the dude saying hi to me everyday the room next door ma friend. They are all just living around me. We all have to pretend to be nice to each other because none of us wants trouble. Sure we do hang out or get drunk at the party. But we are not friends.The only true friend I got is my college sweet heart.
caidh 23 hours ago 0 replies      
I had a college roommate from Hong Kong for a couple years. He specifically asked for an American roommate, and I think we both benefited from the exchange. His English was far better than most of his friends after that time. It's hard though, if I were in his place, I might have been more insular as well seeking out those with my native language.

He also got to laugh hysterically as he saw me try to pronounce some greetings/messages in Cantonese to his father who would call from Hong Kong, so there was that benefit too.

e40 1 day ago 1 reply      
I went to UC Berkeley in the 80's and I found it a sterile and unfriendly environment. Maybe it was me. I had one friend from all the classes I took. One. I had a lot friends from working in the CS department.

I think the reason was the hyper-competitiveness between students. It doesn't foster any sort of cooperation.

ajaimk 18 hours ago 0 replies      
A very one sided argument and a good example of using statistics for evil.

I came to the US in 2007 for my Undergrad from Dubai (Indian by Birth). I lived on a dorm floor with 40 people and only another Indian and I didn't actually talk to that guy much. Most of my friends are Americans including some of my closest - To be honest, the few Indian friends I have are people from work.

End of the day it comes down to your comfort zone. People who come to another country to get an "American degree" will stick to their comfort zone. For those of us who come to explore and understand the local culture, we're going to assimilate into the local culture (my kickball team calls me a coconut: Brown on the outside and white on the inside).

Do you want to stay a tourist or become a local?

zaxxal 1 day ago 0 replies      
As someone who grew up heavily in both the west and east, it's most definitely a cultural issue. IMO Americans tend to be oblivious to so many things and try to hide their ignorance behind liberal values. Which I'm all for, but conversing becomes a game of "what can I say and not have to have a huge discussion explaining my opinion." Mix that in with the college experience and it starts to make sense. I blame the schools for not properly educating children in this country.
geezer 21 hours ago 0 replies      
If foreign students want to make American friends, they should be prepared to feel uncomfortable and also be willing to make others uncomfortable.

There is no easy way. You have to separate yourself from your comfort zone. That includes others from your home country as well as other international students. Live with an American roommate, go to every (American) party you are invited to. Say no to every (non-American) party you are invited to. Find an American gf/bf or keep trying. Join volunteering activities (food drives, blood drives, salvation army etc) to meet locals.

In a couple of years, you would have made yourself deeply uncomfortable on many occasions, annoyed some people, but by now you'll be talking and walking like an American.

This applies in general to immigrants who tend to huddle together because it is the easier thing to do. That is why in most cases, cultural assimilation takes atleast a generation.

aakriti1215 1 day ago 0 replies      
I personally chose to come to University in the US and had a great experience. While my campus had issues with international students and domestic students being divided, we did a lot of research for our newspaper and wrote opinion pieces on how to bridge the gap.

If you'd like to read about another perspective to how international students feel after coming to the US, here's an opinion piece I wrote: http://www.dailynebraskan.com/opinion/agrawal-us-universitie...

If you'd like to read more about what it's like to be an international student making friends with domestic students: http://www.dailynebraskan.com/opinion/agrawal-making-friends...

Here's an article one of my colleagues wrote about making friends with international students as a domestic student: http://www.dailynebraskan.com/opinion/simon-making-friends-w...

DarkIye 14 hours ago 0 replies      
As a university student in London, I noticed Chinese students (the main foreign contingent of the student body) overwhelmingly tended to socialise with one another, and groups of Chinese students speaking Mandarin to each other was a common sight on campus. If you weren't a Chinese speaker, it was hard to penetrate these groups (I tried and failed). Is this also the case in American colleges?
redditmigrant 1 day ago 1 reply      
For me personally as a foreign student from India the biggest hurdle was cultural. I didnt know much about Star Wars, or American TV of the 1980s-90s which meant I couldnt just jump in to most conversations. The second biggest hurdle was that I just didnt enjoy drinking and drinking games to the extent that is a part of contemporary American college setup. Thus I didnt have stories of things I did while drunk. All this combined meant it became hard to enjoy settings with predominantly American/Western groups.

At some point sub-consciously I stopped trying and went into my comfort zone i.e. other Indian people who got my jokes, and where I didnt have to give cultural context before every life story I was telling. I do regret not having made friends from alternate cultures while I had the best opportunity i.e. in college.

merraksh 14 hours ago 0 replies      
Having lived in a few countries, I wouldn't be surprised if an extended research showed that in country X, a significant percentage of foreigners have no close friend from X.

In fact it is much easier to make friends with other foreigners, even when not fluent in country X's language.

duckehlabs 1 day ago 1 reply      
When I was in school, international students had a week prior to orientation with their own orientation. They formed friend groups from this orientation, so it's not too surprising that they would stick to those friend groups rather than then branching out to the people they have less in common with the following weeks.
lr 1 day ago 0 replies      
I was friends with many foreign students in college (a large school in the midwest), and it was because all of them wanted to meet people unlike them. One friend, from Zimbabwe (who was originally from Sri Lanka), once said to me, "I can never understand why all of the foreign students hang out with only people from their own country." She was pretty exceptional, but it's true in general. A lot of Americans do the same thing when they go overseas. It kind of makes no sense to go all of that way, and then just hang out with people just like you.
Crolle 1 day ago 0 replies      
I think it has more to do with the culture than with the campus context. It seems to me that North Americans value more politeness and independence, whereas Europeans think more in term of honesty and reliability. That's why you hear stories of French people being "rude" and Americans being "superficial" or "fake", for instance. Building strong relationships is not impossible, but it is more difficult and it takes time, and a semester can be too short. I'd imagine it can be even worse for people with even fewer common cultural ties.
aakriti1215 1 day ago 0 replies      
Please note this article is from 2012. I feel like the HN headline should be edited to reflect this.
mkaziz 1 day ago 0 replies      
I was a foreign student, and I ended up assimilating by becoming friends with first gen Americans (children of immigrants). Once I got used to a handful of cultural norms (took about a year or two), it became easy to interact with Americans directly.

Today my closest friends are the ones I made in college.

dbg31415 1 day ago 0 replies      
When I was in college, circa 20 years ago, I studied Korean. (I had a friend growing up who was Korean, and he taught me swear words and such, and I figured if I ever got the chance to study I would.)

Anyway, I was 6 months into a language course that was heavily populated by US-born Korean students who were taking it for the easy A... I needed a tutor to keep up so I reached out to the teacher who introduced me to a few Korean exchange students.

They lived in their own apartment, not the dorms. They cooked their own food, didn't go to the cafeterias. Fast forward a bit, my fraternity had a charity event and I invited a few of them... was a casual invite, said something like, "Hey we're doing this thing, tell your friends!"

A week later at the charity concert like 60 Korean exchange students showed up. Every single one of them was dressed in a tux or evening gown. Totally classed up the place. Had no idea there were that many exchange students until that night.

And they were all really appreciative of the invite. Basically said no one had invited them to any events on campus before... I met some new folks, knew just enough Korean at that point to ingratiate myself and get invited out drinking after the event... and quickly realized I was playing checkers at a chess tournament when it came to drinking with Koreans. Ha.

Made some friends out of the deal, but it wouldn't have happened without everyone going outside of their comfort zone a bit.

kingkawn 12 hours ago 0 replies      
It is because Americans are cultureless robots pumped full of paper thin thoughts, who have replaced any experience of togetherness with consumer preferences
azinman2 1 day ago 0 replies      
Years ago I studied abroad for a year in the U.K. I'm American, so no language barriers, but I found it very difficult to make British friends. I was friends with my flat mates who were British, but otherwise only the Germans were friendly to me. The other international students tended to stick to themselves (particularly the Chinese and Pakistani), despite my attempts at bridging the gap.

Homophily is a known social bias, and certainly affects people in their own country let alone a foreign one. I wonder how much of this is "America" per se, versus standard social forces that affect all populations.

cblock811 1 day ago 0 replies      
I studied hospitality in Switzerland for a while and was surprised that some of my classmates just wouldn't integrate with the students there. It was as if they had some aversion to change/something different. Normally I wouldnt find that behavior odd but the whole point of hospitality is to engage and be welcoming. I could see it being a stonger issue for people in other areas of study
jasonkostempski 1 day ago 1 reply      
What percent of Americans have close friends of any nationality on campus?
coolkarni 1 day ago 0 replies      
I found that sports and trivia knowledge is more important than language skills when interacting with the "locals".
yalogin 1 day ago 1 reply      
If there are 750K foreign students coming in every year to the US why are there only 65K H1 visas per year available? Surely they should expect at least 10% of them to want to work in the US after their education right? With that why is everyone acting as if the Indian outsourcing companies are the problem for the H1 visa shortage?
notadoc 1 day ago 0 replies      
I assume much of this is due to language or cultural barriers, even if it's mostly perceived and not a legitimate barrier. Not surprising that most foreign students congregate with one another, but they're missing out on part of the experience of studying abroad surely. This goes both ways BTW, US students studying abroad in many countries experience the same thing as well.
normalperson123 1 day ago 0 replies      
theres a mad dash for everyone in the world to come to american universities and suck every last drop of credibility from them. for a long time college has been getting worse. we went from college being a selective and prestigious club to what we have now: giant degree mills where actual learning is a fucking foot note. and college has become this way because everyone needs a college degree now because there are no jobs for the ordinary joe who doesnt have a degree or connections. so college has been on this slow decline and now we see a ton of asian students who are coming and getting degrees while they still have some value. so at the end of the day the whole thing just stinks. you walk around and look at all the amazing buildings and amenities that the school spent millions upon millions upon millions of fucking dollars on to attract the rich foreign students and out of state party students and the domestic rich students. meanwhile the quality of the education that students get has plummeted. students dont learn anything anymore. elon musk read his physics text book and showed up to exams. he was able to pass because he learned the material. maybe other people have has a different experience but that has not been possible at my school. i had a single professor who went against the current and taught for understanding and at the start of the class he practically got on his hands and knees and begged the students to read the book. being new, i didnt understand. but soon i realized that it was because students dont actually read the books! every other professor chooses some shit textbook thats filled with irrelevant material and you are forced to only read the equations and game the fucking system and get points. learning has been totally abandoned! and now i see swarms of asian students cheating on every exam and i look at the state of the college and it just depresses me. i was in an algorithms class and we had a quiz. times up, we are walking to the front to turn things in and there was an asian guy standing in front of the desk, looking at other peoples papers and copying the answers. completely brazen. and the nobody did anything. nobody gave a shit. and i go to a well known school. the state of affairs is not good.
preexo 22 hours ago 0 replies      
Living in south east Asia I see more than 40% of US American expatriates who are living here not having even a single native/local friend...
ap46 22 hours ago 0 replies      
Another way to look at this is American students don't engage with 40% foreign students on campus. It takes two to tango.
sauronlord 1 day ago 1 reply      
So you move far away AND prefer the company of yourself... what's the problem here?
nradov 1 day ago 0 replies      
As a point of comparison how many domestic students have no close friends on campus? In other words, what's the differential between foreign students and those who grew up here?
systems 1 day ago 1 reply      
so a solid majority of 60% gets to have a close US friendsounds impressive (too good to be true actually)

how does this compare to foreign students in Canada, Germany, UAE, UK

seattle_spring 1 day ago 0 replies      
I wouldn't be surprised if 40% of all students have no close friends on campus.
jorgec 1 day ago 0 replies      
My experience in the U in my generation, started with 60 students and ended only 6 from different levels/years.
partycoder 18 hours ago 1 reply      
Consider yourself in the following scenario:

You hang out with Indians and they start talking about Cricket, a sport that is widely watched there, but also a sport that most likely you know nothing about. How can you partake in that conversation? you can, probably, but only at a basic level and you might not have a lot to add.

So, the same happens with American football, or baseball, or sports that are not widely followed elsewhere.

Those types of conversations marginalize foreigners, even if it's non intentional. Now, if you have empathy, you might prefer to talk about something else, with the purpose of being inclusive.

known 10 hours ago 0 replies      
Why should they have "close" American friends? http://www.washingtonpost.com/blogs/worldviews/wp/2013/05/15...
amyjess 1 day ago 1 reply      
I remember, several years ago (2010-ish), a friend's roommate brought a couple of his Indian coworkers home. I got to meet them because I was hanging out with my friend at the time. They hung out for a bit, and then he took them to a shooting range, bringing his gun collection along.

Before they left, he told my friend and I that he was doing this because he read a statistic showing that a huge percent of Indian-born workers have never been in an American's home, so he wanted to get his coworkers out of that statistic.

In hindsight, that act was the only decent thing that man has ever done (he is no longer on speaking terms with me, my friend, or virtually everyone else we know for a variety of reasons that have nothing to do with his co-workers).

tn13 1 day ago 0 replies      
My wife studied at an American college. She had no friends. The college had over 100 students 100% of them from India, Middle East and Turkey. The college was nothing but a way for H4 women to get F1 so that they could eventually get OPT and instead of being deadweight at home they could stand on their own legs. This is probably one of the N colleges out there.

There aren't too many American citizens in these colleges and I will not be surprised if these student constitute 50% of total F1s.

BrailleHunting 20 hours ago 0 replies      
Student unions should encourage "welcome committees" to make sure no one whom wants to participate in the social scene is left out. (dorm life was the best.)
aaron695 22 hours ago 0 replies      
"You never meet people in their own country" is a factoid I've noticed.

You have more in common with fellow travelers than you will have with people who are just status quo.

normalperson123 1 day ago 0 replies      
love the foreign students.
muninn_ 1 day ago 2 replies      
What could happen if you refuse to unlock your phone at the US border? arstechnica.com
403 points by nkurz  13 hours ago   436 comments top 43
tptacek 12 hours ago 28 replies      
Worth remembering: US citizens can play chicken with CBP and lose nothing more than their computer/phone hardware (perhaps having it returned 6 months later). But non-citizens, possibly including LPRs, cannot: if an alien attempts to cross the border with a device they can't unlock "because they don't have the corresponding 2FA token with them" (as one friend suggested they do), they'll be detained, their devices confiscated, and then put on a flight out of the country.

The current best practice for border crossings --- really anywhere in the world --- is simply not to carry anything you're unwilling to unlock for Customs.

This is going to get harder still. CBP will begin asking everyone for Facebook logins. You'll think of 10 different ways to conceal your Facebook doings from CBP, but CBP has advance traveler's manifests from flights and will know that people have profiles --- and, sometimes, what was on those profiles.

danbruc 11 hours ago 4 replies      
How is this not in violation of international law, especially the International Covenant on Civil and Political Rights [1] article 17, which would apply to US citizens as well as non-US citizens?

No one shall be subjected to arbitrary or unlawful interference with his privacy, family, home or correspondence, nor to unlawful attacks on his honour and reputation.

Maybe even article 9 if the law does not clearly establish what is allowable.

Everyone has the right to liberty and security of person. No one shall be subjected to arbitrary arrest or detention. No one shall be deprived of his liberty except on such grounds and in accordance with such procedure as are established by law.

EDIT: Turns out the US did not really ratify this treaty. [2] So no human rights in the US, or at least not all of them enforceable in court. There may of course be similar rights from other laws or treaties.

[1] http://www.ohchr.org/EN/ProfessionalInterest/Pages/CCPR.aspx

[2] https://en.wikipedia.org/wiki/International_Covenant_on_Civi...

nyxtom 21 minutes ago 1 reply      
Worth noting that the DHS has the authority to take your device, copy the contents, and share any information they found on a case-by-case basis. It seems likely that the standard practice will be to simply copy the content wholesale and use it for importing into a system for analysis/flagging/later sharing and completing the network identity. Effectively, handing over your phone completes the picture in terms of network analysis. They have your facial/physical identity, all your communications, your photos, downloads...etc. This seems to also apply to laptops as well and it's likely that they can also just copy all that over as well without discretion. Make no mistake, the authority is far-reaching and completely unchecked.


"While CBP Officers are responsible for the examination of electronic devices, only Supervisorsmay authorize the copying of the contents of an electronic device.54 Where an electronic device is to bedetained or seized by CBP, a CBP Supervisor must approve of the detention or seizure, and the CBPOfficer must provide a completed CF 6051D or S, respectively, to the traveler.55 Where a traveler claimsthat the contents of the electronic device contain attorney-client or other privileged material, the CBPOfficer must consult with the local Associate/Assistant Chief Counsel or United States Attorneys Officebefore conducting the examination.56CBP Supervisors may authorize the sharing of the travelers information for assistance or otherlaw enforcement purpose on a case-by-case basis"

tzs 9 hours ago 2 replies      
What happens if you try to mail or ship a smartphone across the border, as opposed to carry it with you when you are crossing the border? I'm wondering if you could ship your regular phone ahead, and use a burner while you are traveling.

If that works, that could be an interesting business opportunity. Set up a series of phone exchange stations near major airports, where outgoing travelers can swap their smartphone for a burner. You ship the smartphone to your station near their destination, and when they arrive they can turn in the burner and retrieve their smartphone.

You could also add in a temporary backup/restore service for laptops. A traveller could bring their laptop to your station when they are preparing to leave, and you do a backup of all user data and then delete it from the laptop.

While they are in transit, you send the data to your exchange station nearest their destination. When the traveller brings their laptop in, the user data gets restored.

megous 10 hours ago 1 reply      
Haha. Terrorism bla bla.

Anyone noticed that the first two example cases are actually political activists being targeted.

- the first is some Chelsea Manning advocate- the second is some pot legalization activist

There you have it for what actually are these rarely used rules used for.

kutkloon7 4 hours ago 0 replies      
Well, at 4444 times a year it doesn't seem that rare of an occurrence.

But, hey, what's new? The US has imprisoned innocent people, and tortured them multiple times a day for years and years. This is public information, and no one gives a shit. The US government really doesn't care about anyone who is not a US citizen. (And sadly, western countries mostly just let the US do whatever they want)

A women is being locked up for 8 years because she checked the wrong box on her voting form, and it is pretty common that unarmed black people get shot by cops without serious consequences.

I understand that people are upset about the possibility of having to unlock their phones, but there are far more serious indications that the US is not the great country it wants be. A lot of common sense and decency it missing in the US - and not just in the Trump administration.

turc1656 9 hours ago 0 replies      
This practice should be banned on the premise that any digital data that could enter the country physically on that device could just as easily cross the border and enter on any internet connection. Searching the device does nothing to prevent anything from entering the country, which is the primary role of the CBP agents - to control the flow of people and physical items. Not data.

If the CBP then claims that it's a useful tool/practice because they can identify bad people who shouldn't be allowed in, they should be asked to provide stats on how many people were denied entrance after their devices were checked. My guess is that number is statistically meaningless, probably in the <1% range for people who actually unlock a device.

One of the only exceptions I can think of is cases like this one:http://cyb3rcrim3.blogspot.com/2016/04/the-laptop-child-porn...where there is already a legitimate investigation going on and a person is suspected of serious criminal activity and they don't have enough for a warrant. In these cases, the CBP uses the rules regarding border re-entry to their favor to compel a search of a device because they have at least some sort of reasonable suspicion that a device may contain data for which the mere possession of is highly illegal. Other than that, I'm hard-pressed to think of any cases where the mere possession of data would be a felony unless they suspect you of having classified information without the appropriate clearance.

civilrightsyep 10 hours ago 1 reply      
These discussions tend to focus on all the inane ways to subvert the border guards.

The real question we should be asking is how to make this a big enough issue that it gets mainstream attention and action.

We shouldn't try to figure out how to continue down the fascist path but how to change it's direction.

biscuitsandbbq 11 hours ago 4 replies      
This happened to me crossing into Canada and to be honest I really wasn't expecting it. I had tons of very private pictures and messages on there stretching back years. I really didn't want to turn over my password so that 3 border agents could look through all that stuff in some back room, but at the same time I knew there wasn't anything illegal on there. Anyway, I was with my partner and didn't want to ruin her vacation as well so we just turned over our passwords (under threat of detention if we didn't).

In any case, if you find yourself in this unfortunate situation (you shouldn't - you should travel with an empty phone when crossing the border, or at least wipe it before crossing then do a restore over icloud later), here's something you can do: Before going in (if you have an iphone), make sure you have a strong alphanumeric passcode. Enable auto-wipe on 10 incorrect passcode attempts. Then, turn the phone off. If CBP then asks to turn your phone on and wants your password, appear eager to comply, but give them an incorrect one, and when that doesn't work, admit that you had just changed it a few days ago to something more secure and that you may have forgotten it, but don't worry, you always just use the thumbprint to unlock the phone anyway and volunteer to open it that way. Always appears helpful. The phone still won't unlock because iphone requires passcode entry after restart. If they ask why you turned the phone off, just say that's what you normally do when entering border control. Unless you're under some kind of serious suspicion of some kind at this time, they're unlikely to detain you for forgetting your phone password, and even if so, it's locked with a strong passcode so there's no much they can do anyway.

askvictor 3 hours ago 0 replies      
Are there any Android distributions that can boot to a different environment depending on what code is entered at boot time? And said partitions are hidden from each other (i.e. once booted, the other partition is impossible to detect)? This was described in Cory Doctorow's "Little Brother" novel, and on the face of it seems like a reasonable proposition - any major flaws in such a plan?
bogomipz 11 hours ago 0 replies      
>"Ars spoke with several legal experts, and contacted CBP itself (which did not provide anything beyond previously-published policies)."

In other words the CBP were completely unhelpful and simply referred them to published legalese online rather than clarifying that legalese. It really feels like the culture of DHS and CBP views the people they are supposed to serving as adversaries.

dandare 5 hours ago 1 reply      
What if I simply reset my Android phone to factory settings just before exiting the plane? Can they force me to log in to my Android account in order to get to my FB?

And what if my battery is dead or my phone stops working? Can they force me to log in to their own Android phone?

What if I don't carry any phone at all, can they simply force me to log in to an empty phone? (Assuming everyone has an Android or Apple phone or is going to spend couple days in detention.)

jdalgetty 11 hours ago 1 reply      
I don't understand America's fascination with snooping through peoples shit.
pyrale 10 hours ago 0 replies      
The simple way to avoid this for me, is to skip avoidable US trips for now. I would have loved to attend strangeloop, but I will wait for better opportunities.

If my work requires me to travel there, I will simply ask to have electronic material provided in the US, and download whatever tools I need once inside the US.

I would advise anyone traveling to have versioned working tools stored online anyhow.

z5h 11 hours ago 1 reply      
I don't know any of my passwords except for my password manager. If I leave my password DB at home, and mail a physical copy to my destination, what can happen? I'll be absolutely unable to access any account.
mariocesar 10 hours ago 0 replies      
Can you just send your phone or laptop through fedex, or some other post service, and just don't travel with anything with a password.

Even better, do a cloud backup and wipe out your phone or laptop, and restore a backup when you are out?

Or am I missing something?

manishsharan 10 hours ago 0 replies      
Maybe its for the best. This gives me a reason to buy a Nokia 3310 or its new version http://venturebeat.com/2017/02/13/hmd-global-will-launch-the...
intopieces 12 hours ago 4 replies      
Is it worth it to clear those devices and restore from cloud backup after crossing?
ipsin 10 hours ago 2 replies      
Are visitors being turned away because they can't provide passwords for social media accounts? Using a local password safe (KeePass2, etc.) should be a great excuse, but I'm not sure what happens next.

It bothers me that the US is providing a terrible example of how to treat visitors.

no1youknowz 11 hours ago 1 reply      
One thing I have thought about. I have dozens of profiles which are just test accounts for social media, to hook up to applications I've been building.

I've been thinking recently to just close all the accounts, including email.

This will leave me in a state where I have pretty much no social accounts, just 1 email account which is used for friends/family.

Are there any cases of individuals giving up their phone and it being a wasteland of information? I personally do not use facebook, google-plus, twitter, linkedin or anything else. I literally do not have an opinion, nor do I care to share it!

Edit: Do they also check when email accounts have been created? How could they know, unless they checked in with the provider? I'm thinking of closing my main account, requesting data be deleted and starting afresh!

pfarnsworth 6 hours ago 0 replies      
Can they ask me to turn over my gmail username and password, even though I don't have a phone or computer on me?
crispyambulance 11 hours ago 2 replies      
If some CBP knuckle-dragger wants me to unlock my phone so he can browse through it looking for whatever as I stand there watching while asking me insipid questions, that's not a huge deal.

The real problem here is downloading of data from the phone. Once that data is off-loaded, the government can make it a part of their "Seven-degrees-of-Kevin-Bacon-Osama-Bin-Laden edition" game, stored forever, perpetually looking for "connections" no matter how remote. That's a scary and slippery slope to surveillance state hell. I don't think such practices are commonplace, not yet?

Has anyone here ever been in a situation where the CBP confiscated their phone/laptop or downloaded data electronically from it?

analogmemory 10 hours ago 0 replies      
This article [1] suggested to reset your phone to a blank slate before you hit customs and then reinstall after you're in. Not sure if that's going to help but I like the idea of the subtle middle finger.

[1] https://medium.freecodecamp.com/ill-never-bring-my-phone-on-...

denzil_correa 12 hours ago 1 reply      
> The short answer is: your device probably will be seized (or "detained" in CBP parlance), and you might be kept in physical detentionalthough no one seems to be sure exactly for how long.

This is a precarious situation to be in an I'm surprised if this does not violate some provision of the law. Worse, there have been conflicting judgments on similar cases [0, 1].

On a side note, can law itself be unbiased due to the nature of the cases heard in courts? A situation X regarding certain law LX might be heard 1000 times and would be more clear in contrast to a situation Y concerning with law LY as it went to court just 3 times. Can one computationally figure out laws which are unclear based on the number of times they are references in court judgments or some other similar parameters?

[0] https://arstechnica.com/tech-policy/2013/12/judge-wont-let-s...

[1] https://arstechnica.com/tech-policy/2015/05/warrantless-airp...

lb1lf 11 hours ago 4 replies      
Question: what would happen if you are asked to unlock your device and you refuse on the grounds that it contains classified material?

Presumably they wouldn't just shrug and say 'OK, then' - but neither could they (if the classifying authority was one they cared about - say, NATO) just say 'Tough luck, now unlock it!', right?

athenot 9 hours ago 1 reply      
Have there been any incidences of people who signed up for CBP's Global Entry program having trouble?

Many of the complaints here are a (justified) fear that privacy invasion occurs as a side-effect of a clumsy and poorly thought-out attempt to prevent something illegal[1]. So if I go ahead and sign up for the background check + interiew to let them see I don't have ill intentions, would this reduce the likelyhood of collateral privacy invasion when actually crossing the border?

I've been considering this for convenience reasons but am curious if it might help in this instance.


[1]: There is also the issue of restricting freedom and declaring people "illegal" when that runs against the spirit of this country, but I'm not focusing on that here.

yawz 10 hours ago 3 replies      
The question I have in mind is "do they have the right to confiscate your device or to disappear with it?". What if I told them "I'm going to unlock it, but I'm the driver. Ask me all that you want to see, and I'll take you to that app?".
SadWebDeveloper 6 hours ago 0 replies      
Personally i like to travel with a burn phone that only has the contacts i need in case something happens to me and dont bring laptop because m usually not traveling internationally for work.
technologyvault 11 hours ago 0 replies      
There are probably thousands of instances similar to this (not necessarily having to hand over your device) where US citizens are at the mercy of law enforcement simply because we don't know what are rights are, and because often law enforcement either doesn't know or doesn't care.

Civil rights often tend to be only as useful as our knowledge of what exactly they are.

danaliv 5 hours ago 0 replies      
So worst case I'm out a thousand bucks and one phone? Sounds like it's worth the trouble.
cousin_it 11 hours ago 1 reply      
When you arrive at the border, it might be too late to get privacy-conscious and refuse to unlock your stuff. Unless you're using non-American devices and social media, your data probably gets inspected anyway when you apply for a visa, and might be monitored at other times as well.
xtf 11 hours ago 0 replies      
This means I have to install a blank Custom ROM before I travel to the land of the free
transfire 12 hours ago 4 replies      
The idea that our constitutional rights end at the border is such a travesty, contrary to everything our forefathers fought for, that alone is worthy of renewed revolution.
gkfasdfasdf 10 hours ago 0 replies      
They can detain you or your device even if you do unlock it. So better just not to unlock it.
twtw99 10 hours ago 0 replies      
Was wondering what happens if they find posts on social media criticizing the current administration? Trump often used to highlight the fact that the thousands of CBP agents has supported/endorsed him.
jacamat 8 hours ago 0 replies      
Reset to factory then unlock...
sandworm101 9 hours ago 1 reply      
Ive got a trick. As a lawyer i keep a note on my laptop deacribing how it is encrypted (linux, home dirctories + trucrypt archives for important stuff) and that content is subject to attorney-client privacy rules. Ive been asked to unlock several times. I say "sure thing, but ill need some sort of court order so i dont get sued." Faced with a reasonable person willing to unlock, they have never pushed the matter. For my phone, phones cannot be trusted. Dont do anything you want to keep private on a phone. Buy a netbook.

I would suggest saying something like "sure thing, but i use this device to talk to my lawyer and i should ask him first if it is ok." They wont want the hassle of dealing with this perfectly reasonable request. (Don't fib. Have a lawyer first. All you need to do is email one a couple times for that statement to he true.)

ballenf 12 hours ago 1 reply      
Same result as if you refused to open a securely locked briefcase?
mpcsh 11 hours ago 3 replies      
Here's what I want to know: what happens if I just spike my phone on the ground?
youdontknowtho 10 hours ago 1 reply      
US Customs will sing you a song called "Punch you in the face." It's a classic.
ronnier 12 hours ago 3 replies      
> 4,444 cellphones and 320 other electronic devices were inspected in 2015

And recently I'm hearing about this -- it largely went ignored during the previous administration but now it's a major concern, articles on HN, my Facebook feed, Reddit, ...

rsync 11 hours ago 3 replies      
If your border crossing strategy involves a wiped/clean/factorydefaults device, always, always, always put a zip bomb[1] in place.

Name it something helpful like "do_not_open_this_file.zip".

They can't say you didn't warn them ...

[1] https://en.wikipedia.org/wiki/Zip_bomb

bflesch 11 hours ago 2 replies      
In a world where thousands of people annually travel to some "freedom fighter" camps in order to get combat training, I very much agree that it is necessary to intensively screen travelers who return to their country of origin once they fit a certain heuristic.

As one single returnee who is not caught might actually cause dozens or hundreds of citizen deaths, there shouldn't be a screening exception for smartphones or computers.

We can all argue that the current heuristics / profiling methods are not good enough, but as an EU citizen I'd be glad if my government would actually be as straightforward about screening travellers as USCBP is. If travelers - citizens or not - want to return after learning to kill or taking part in some sort of criminal activities, or even announcing their support for such criminal activities in social media, they should be held accountable for their actions upon returning by strict border controls.

The most cited deep learning papers github.com
426 points by sdomino  1 day ago   44 comments top 12
cr0sh 1 day ago 2 replies      
I can understand why it probably isn't on the list yet (not as many citations, since it is fairly new) - but NVidia's "End to End Learning for Self-Driving Cars" needs to be mentioned, I think:



I implemented a slight variation on this CNN using Keras and TensorFlow for the third project in term 1 of Udacity's Self-Driving Car Engineer nanodegree course (not special in that regard - it was a commonly used implementation, as it works). Give it a shot yourself - take this paper, install TensorFlow, Keras, and Python, download a copy of Udacity's Unity3D car simulator (it was recently released on GitHub) - and have a shot at it!

Note: For training purposes, I highly recommend building a training/validation set using a steering wheel controller, and you'll want a labeled set of about 40K samples (though I have heard you can get by with much fewer, even unaugmented - my sample set actually used augmentation of about 8k real samples to boost it up to around 40k). You'll also want to use GPU and/or a generator or some other batch processing for training (otherwise, you'll run out of memory post-haste).

pizza 23 hours ago 1 reply      
http://people.idsia.ch/~juergen/deep-learning-conspiracy.htm... oh Juergen

> Machine learning is the science of credit assignment. The machine learning community itself profits from proper credit assignment to its members. The inventor of an important method should get credit for inventing it. She may not always be the one who popularizes it. Then the popularizer should get credit for popularizing it (but not for inventing it). Relatively young research areas such as machine learning should adopt the honor code of mature fields such as mathematics: if you have a new theorem, but use a proof technique similar to somebody else's, you must make this very clear. If you "re-invent" something that was already known, and only later become aware of this, you must at least make it clear later.

kriro 16 hours ago 1 reply      
This might be as good a place to ask as any. Does anyone have suggestions on the problem of annotating natural language text to get a ground truth for things that have no readily available ground truth (subjective judgments of content etc.)? I do own the book "Natural Language Annotation" which is good but not exactly what I need. The part of annotation guidelines and how the annotation was done in practice is often only brushed over in many research papers. I mean I get it at a high level it's basically have a couple of raters, calculate inter- and intrarater reliability and try to optimize that. However like I said I'm struggling a bit with details. What are actually good values to aim for, how many experts do you want, do you even want experts or crowd source, what do good annotation guidelines look like, how do you optimize them etc.? Just to play around with the idea a bit, we did a workshop with four raters and 250 tweets each (raters simply assigned one category for the entire tweet) and that was already quite a bit of work and feels like it's on the way to little side of things.

I feel like I should find a lot more info on this in the sentiment analysis literature but I don't really.

mathoff 15 hours ago 0 replies      
The most cited deep learning papers: https://scholar.google.com/scholar?q="deep+learning"
curuinor 1 day ago 5 replies      
No PDP book? It's old and weird but interesting and has a lot of original ideas, notwithstanding the actual original backprop being from before then. Nor the original backprop stuff?
nojvek 21 hours ago 1 reply      
Someone needs to make a summary of the top papers and explain it in a way lay man can understand. I would pay $500 for such a book/course explaining the techniques.

I've been reading a number of this papers but it's really tough to understand the nitty gritties of it.

pks2006 1 day ago 2 replies      
I always wanted to apply the knowledge of the deep learning to my day to day work. We build our own hardware that runs the Linux on Intel CPU and then launches a virtual machine that has our propriety code. Our code generates a lot of system logs that varies based on what is the boot sequence, environment temperature, software config etc. Now we spend a significant amount of time go over these logs when the issues are reported. Most of the time, we have 1 to 1 mapping of issue to the logs but more often, RCA'ing the issue requires the knowledge of how system works and co-relating this to the logs generated. We have tons of these logs that can be used as training set.Now any clues on how we can put all these together to make RCA'ing the issue as less human involved as possible?
applecore 14 hours ago 1 reply      
Classic papers can be worth reading but it's still useful to know what's trending.

Even a simple algorithm would be effective: the number of citations for each paper decayed by the age of the paper in years.

gravypod 1 day ago 3 replies      
This is a really lucky find for me. I was just about to do something to try and get into machine learning. Right now I need some help getting started with writing some machine learning code. I don't know where to start. I've come up with a very simple project that I think this would work very well for.

I want to buy a Raspberry Pi Zero, put it in a nice case, add to push buttons and turn it into a car music player (hook it into the USB charger and 3.5mm jack in my car). The two buttons will be "like" and "skip & dislike". I'll fill it with my music collection, write a python script that just finds a song, plays it, and waits for button clicks.

I want the "like" button to be positive reinforcement and the "skip & dislike" to be negative reinforcement.

Could someone point me in the right direction?

EternalData 1 day ago 0 replies      
Nice. Super excited to read through and build out a few things myself.
gv2323 21 hours ago 0 replies      
Has anyone downloaded them into their own separate folders and zipped the whole thing up?
husky480 1 day ago 0 replies      
torchcraft is the best way to learn about machine learning.

If you can sim a set of boxes, you can learn whats inside them.

Software developers who started after 35, 40 or 50 belitsoft.com
371 points by clubminsk  17 hours ago   146 comments top 30
aczerepinski 13 hours ago 6 replies      
Software - like anything worth mastering - requires many thousands of hours. The only reason I can think of that it would be harder to pick up at an older age is that often the hours are harder to come by due to family commitments and so forth.

I went through an intense several year period of studying jazz music beginning around age 16. I remember the discipline it required and the speed at which I was able to progress.

When I got into programming at age 33, it felt exactly the same. The discipline, the hours, the speed of progression; all very similar. Having devoted many thousands of hours to mastering a skill in my teenage years and again in my 30s, I'm not aware of any differences in capacity to improve at one age over the other.

I attended a boot camp and had ~25 classmates all over the age spectrum. Some were smarter than others, some worked harder than others, but there was no age correlation on either of those observations. I also didn't see any age correlated patterns in success in the job market. Some old and young students got jobs immediately; some old and young students had to fight it out a little longer.

If I lose my passion for software at age 40, 50 or 60, I would be very open to pursuing something new at any point in my life.

JeremyNT 7 minutes ago 0 replies      
I'm nearly 40 and just started my first software development job.

In my case, I already knew Python and Ruby from working in ops, so it wasn't completely new to me as was the case for the people in the article. What is new to me is having larger programming tasks that require focus and teamwork. The nature of ops tends to involve a lot of context switching and as a result most of my projects were small and self contained. Learning how to collaborate with others on the same code base is a big adjustment.

I am also incredibly impressed with how patient my colleagues are. I know "how to code" broadly, but that's not the same thing as "how to be a software developer." I'm fortunate enough to have the opportunity to learn as I go.

soneca 14 hours ago 8 replies      
That's great that this is getting upvoted at HN. I started studying software development last November, at the age of 37. A few times, in my own and other people's posts stating that we are learning to code, there was more negativity in the comments than I would expect from HN.

It is always a mix of: "quit now, it take lots of years to become a developer worth of its name", "don't do it, software development is not the glamorous job you think, it's awful" and "give it up, you just won't be hired for a good job that easily".

An impression I often have is that there are developers who have some kind of resentment reaction to the "everyone can/should learn to code". Like new kids sneaking in your own private club. Outsiders trying to be like you. There is only one type of developer: that kid that started to code when he was 12. All others are impostors and wannabes.

I'm glad that there is this support sentiment for stories like this as well.

Veen 16 hours ago 3 replies      
Not to rain on anyone's parade, but it would also be useful (probably more useful) to see stories of older people who have tried and failed to become developers.

It's nice to see the success stories but I'm always wary of survivorship bias. If there are ten people who couldn't make a go of it for every one of these stories, it puts a different view on things.

(I say this as a 37 year old freelance writer currently learning Elixir and React in the hope of shifting careers.)

Lordarminius 16 hours ago 3 replies      
This is a fantastic article.

I decided to learn programming at 38, started at 39, and now at 41 I am in the process of releasing a commercial version of software I created. I moved into this field from medicine, to have greater control over my life, scratch my entrepreneurial itch and broaden my horizons. I have no regrets.

You can do it if you really want to, and at any age.

Entangled 15 hours ago 1 reply      
I started learning Swift in my fifties and by god am having a blast. Already finished an iPad app for my daughter and there is no better gratification than seeing her playing and learning with it.
sonabinu 14 hours ago 3 replies      
I started later in life, after being a stay at home mom for 10 plus years. I went back to regular school and now I'm working as a Software Engineer. I need to look up more concepts and ask more questions but I'm getting there. It's a long road but one worth walking.
empath75 12 hours ago 0 replies      
I've dabbled with programming off and on since the 80s, but never really did anything work related until I was 38. I got transferred from the NOC to a sys admin position just as everything here became devops all the time. Really focused on learning python, aws and Jenkins. Now 41 and I just got promoted to being a senior software engineer at a very large tech company.

I actually credit hacker news for a lot of it because repeating what I read here makes me seem a lot smarter than I am, I think.

jcadam 11 hours ago 1 reply      
I honestly don't think it matters. You're going to hit a salary ceiling at about the 8-10 year mark no matter when you start. So unless you're already at or near retirement age, it's not too late :)

I'm 36, started programming at about age 6 or 7 (thanks largely to my mother being a programmer and helping me learn the basics), and still spend a lot of my free time on personal projects. I actually didn't start programming for a living until age 26 (did a stint in the Army right after college).

I haven't had a significant raise in the last 4 years. In fact, I made a higher salary (albeit in a higher COL area) 5 years ago. Early in my career I was getting 20%+ pay bumps just for switching jobs. That doesn't happen anymore. More often than not I have to make my salary expectations clear from the first conversation with a potential employer lest I waste a lot my time only to receive an offer 30% lower than my current salary.

It's the point I'm thinking of leaving the field (I'll never stop programming in my free time, though) and finding something else with some actual upward mobility.

lebanon_tn 16 hours ago 1 reply      
Kudos to these people. A lot of the time when I hear people ask "Is it too late to do x" it sounds like they are asking for an excuse to not try.

I know for certain older workers will face subtle and not so subtle discrimination which makes it incredibly unfortunate that companies don't focus on it more in their workforce diversity initiatives. Is this a problem companies are less willing to confront? Compared to say, gender and race diversity?

makecheck 7 hours ago 0 replies      
One of the great things about programming compared to many professions is that the requirements to start are pretty low: you need a machine and (realistically) Internet access, and a willingness to search for examples and answers. It doesnt require you to clear space in your garage, or invest in large equipment, or go anywhere far away, or have a company backing you to acquire the necessary training; you just start. Even better, progress can be realistically made in spare time without necessarily abandoning whatever job got you this far, assuming you arent working 90-hour weeks.

It is also one of the few disciplines that is included in part-time degrees at some universities. This means you can even be taught radically new things in the field without necessarily giving up your day job.

SuperGent 14 hours ago 1 reply      
I thought that starting out as a developer at 42 was a bit too old. I felt much better when the new guy I started with admitted he was going to be 50 at the end of the month.
omginternets 16 hours ago 3 replies      
This seems to conflate "you can become a dev" with "you can be hired as a dev" after the age of 35, 40 or 50.

This isn't to say it's not worth becoming a dev at a later age -- coding is an increasingly crucial skill for entrepreneurs -- but it seems cruel to entertain the myth that older devs get hired by the handful.

drunkkcunt 12 hours ago 3 replies      
This was motivating.

I'm 25, I started (properly) learning and liking coding about a year ago. Seeing people who are 4-5+ years younger than me with more knowledge and experience is discouraging. It doesn't help than in a job interview I was told "Why should we hire you when there are people younger than you with more experience"

barking 16 hours ago 3 replies      
Might be hard to get a job though.

One thing you'll likely have in your favour is knowledge of another domain to a degree that perhaps no other programmer has.

Even so you might have to go the start up route and then unless you have strong marketing skills you're unlikely to do well.

"Is it just you? What happens if you die?" is a question you'll get over and over.

laythea 8 hours ago 0 replies      
In my opinion, this question is not answerable because:

- it calls into question the individuals motivation and ability to materialise that motivation - this is impossible to measure.

- there is no such thing as a "software developer". We write software, this is true; however software is so ingested into society that one software engineer may be performing an entirely different role to another. And different roles naturally demand differing skill sets - this is impossible to specify.

So, without meaning to offend, bundling up the entire aged population and asking if they can do a job that is hugely variable is a bit of a non-sense.

ClaytonB 10 hours ago 0 replies      
I absolutely feel that it is harder to learn a new career as you get older. You have much less time as an adult than a traditional college student, and my brain doesn't retain things as well as it did 18 years ago. Also, the repercussions for failing to successfully transition into a new career are much more severe for someone approaching 40 than someone in their early 20s.

There was significant risks associated with my decision, financial burdens (loans, credit card debt accumulation), and also some opportunity costs of not earning income for over a year. I quit my primary job, leaving me with no safety net. It was scary at my age to do this, but by taking on such huge financial risks I was more even motivated to succeed - to fail would have been devastating.

happy-go-lucky 13 hours ago 0 replies      
As of now there's no such thing as programming species, natural or genetically engineered. Like any other skill, programming is learnable. One's age should not be a barrier to learning it as long as they enjoy their mental health[1]. Some skills are easy to learn and some are hard and take many years of practice before one can do it with some mastery.

We learn skills out of necessity or out of passion, sometimes out of both. Whether we become good at something depends on a number of factors, and obviously there're efficient older workers and incompetent younger workers in every industry, and this dualism applies to programmers as well.

A computer is a means to an end. People in disciplines such as biology, mathematics, sociology learn and use programming as a tool to solve their day-to-day work-related problems because it helps boost their productivity.

[1] http://www.who.int/mediacentre/factsheets/fs220/en/

mrwebmaster 11 hours ago 1 reply      
I started 2 months ago, trying to follow the courses of Edimburgh Software Engineering ( http://www.drps.ed.ac.uk/16-17/dpt/utswenm.htm ). At the moment I'm doing Haskell, Linear Algebra and Computational Logic, but I see that everybody just do some courses with a more practical approach (Python, NodeJs, Ruby on Rails, etc.) I have some web development knowledge (manage several drupal sites and have 2 linux servers online) and have some mathematics knowledge (I am economist). Do you think I should also take the fast path? Am I loosing my time by learning maths, logic and Haskell? I'm 36
pklausler 6 hours ago 0 replies      
I believe that anybody with the aptitude for abstract thought can learn programming and mathematics at any age, and that anybody without the aptitude for abstract thought is never going to become competent at programming or mathematics no matter how early they start or how hard they work at it.
LiweiZ 12 hours ago 0 replies      
Software is the path I chose to achieve many different short/mid term goals for my life. Finding an ok paid job, exploring another form of freedom of mind, the ability to create things I want, guiding/teaching my kids better with deeper understanding and know how of the development of human-control-machine-to-achieve-result (even though I only have a tiny piece of knowledge of it), sensing the new opportunities, etc.

While I'm still struggling with the first one, I found the rest are all very well achieved.

To all late starters: keep going as long as the resource is available and good luck to all of us.

edit: typo and replace "tech" with a more detailed description.

phodo 4 hours ago 0 replies      
Being a young engineer may make you marketable. But having deep domain expertise and engineering talent at a later age gives you extraordinary super powers.
z4chj 12 hours ago 1 reply      
Really unfortunate that it pitches php development as a way to get started as an older developer. As if an older developer doesn't already feel antiquated and behind, let's make it worse by teaching them an established technology that is not usually the language of choice for companies on the bleeding edge of technology, which i, I imagine, where these people are hoping to be
hopfog 13 hours ago 0 replies      
I almost finished a Bachelor in Business and Economics and worked abroad in a completely different field before starting my career as software developer.

If you have the interest you will pick up the stuff you need for the job fast. Even though many of my colleagues have been programming since childhood I feel that their growth curve has leveled out compared to mine, at the point that we're on par in some areas.

dghughes 11 hours ago 0 replies      
I was expecting to see people like me who try programming every few years (since 1983) but never got into it. My fingers say yes but my brain says no.

The people shown in the article were unexpected I think they are even more interesting to jump right into programming from a jobs that you wouldn't expect.

kemonocode 14 hours ago 0 replies      
I think it's great this article is getting so much attention, considering the huge amount of bias and "ageism" that exists in the field.

Just because you can be an entrepreneur before 30 doesn't mean you'll have the life experience to be good at it, and many junior-level software development jobs expect a level of commitment that's unfeasible for people who "have gotten a life".

dustingetz 14 hours ago 0 replies      
i thought the argument for agism was that, once you hit around 30 your salary is peaked because it only takes 5-10 years for a college grad to catch up to the state of art. and the counter-argument is that if you are so good you can make high risk projects successful, your salary is bounded by the business value of the project
richev 5 hours ago 0 replies      
Inspiring article, but suffers from confirmation bias.
gaspoweredcat 17 hours ago 4 replies      
bottom line, if you can read and type you can do it
65827 13 hours ago 1 reply      
Man, I couldn't imagine that. People stare at me like I'm a martian Elmer Fudd and I'm only 31, really hope to be out of this town by 35.
Pathfinder, a fast GPU-based font rasterizer in Rust pcwalton.github.io
380 points by mmastrac  2 days ago   100 comments top 16
raphlinus 2 days ago 1 reply      
Very happy to see this become public, and it looks very impressive. I'm blushing a bit. Patrick and I indeed had very stimulating conversations, but all the hard work figuring out how to map rendering efficiently to a GPU is Patrick's.
AceJohnny2 2 days ago 3 replies      
This is an exciting demonstration, but I have one worry that's unaddressed: I'm assuming all the other nitpicky details of hinted font rendering etc is handled correctly by all the renderers compared? I mean, they all provide pixel-identical output?

Otherwise the point is mostly useless. A faster rendering implementation is almost useless if the output doesn't look as nice. I wouldn't care if the text I'm reading over 5 minutes takes 200ms vs 800ms to render.

outworlder 2 days ago 3 replies      
> This is the case on the Mac, since Apple has not implemented any OpenGL features newer than OpenGL 4.2 (2011)


Once upon a time, the Mac was a great development platform...

fmap 1 day ago 0 replies      
This looks impressive, but does anybody know why "exact coverage" is apparently considered the gold standard for rendering vector graphics? Mathematically, computing pixel coverage corresponds to sampling a box filtered version of a characteristic function.

In practice I would expect, say, a gaussian filter to be both easier to approximate and less prone to aliasing artifacts. Apparently that expectation is completely wrong though, since nobody seems to implement it that way! What's so special about vector graphics that makes the box filter behave well?

CalChris 2 days ago 2 replies      
Along these lines, well, in parallel and a few doors down, there's Alacritty, a GPU-Accelerated Terminal Emulator written in Rust.


pcwalton 2 days ago 4 replies      
For those seeing broken links for the graphs, please reload.

Happy to answer any questions :)

datenwolf 1 day ago 0 replies      
Impressive work. One of my (far too many) ongoing projects, is also a GPU based glyph renderer. However I'm using a different method, which relies on some preprocessing/conversion of the glyph data. Described in as few words as possible my method could be coined as "trivariate polynomial distance fields". The renderer is mostly done, but there's still a lot of work needed in the glyph preprocessor to be robust and universally usable.

Months ago I posted a few screenshots on Twitter (https://twitter.com/datenwolf/status/714934185564225536), and the comment by Michael IV is spot on. The renderer has no problem with sharp corners, but so far the glyph preprocessor still struggles with it and I have to manually adjust the emitted output to get nice results.

jarjoura 2 days ago 1 reply      
I'd be curious to see how this compares to CoreText on Mac and DirectWrite on Windows. Both are highly tuned for their respective platform so I'd see them as baseline.
ori_b 2 days ago 2 replies      
It looks like this doesn't handle hinting, subpixel rendering, and glyph tweaks that people expect from modern font renderers.

Hinting is not obsolete.

hesdeadjim 2 days ago 1 reply      
Wonder how it compares to using Antigrain Geometry. I once had to implement text rendering in a game engine a bunch of years back and I used FreeType to load font glyphs and then fed the geometry data into Antigrain and had it rasterize (FreeType rasterization is meh while AGG is heavenly). Even on very old iPhone hardware I was able to render in the main update loop and not encounter any frame rate hiccups.

Unfortunately AGG 2.5 is now GPL so if you need to stick it into anything closed source you are stuck using 2.4's modified BSD.

vvanders 2 days ago 1 reply      
First off, impressive stuff.

That said I think having a CPU tessellation path is going to be critical if you want to see wide adoption. Platforms like Android and the like don't always have geo shaders which is why you see FreeType so widely used.

AriaMinaei 1 day ago 0 replies      
I wonder how this compares to the parallel vector rendering engine that Alan Kay has mentioned in a few of his talks [1][2].

[1]: https://youtu.be/XnDYuQUN4J0?t=1060[2]: https://github.com/damelang/nile

vedranm 1 day ago 0 replies      
Thanks for posting this. Added to [1]. Will try running it on Mesa radeonsi OpenGL and OpenCL later.

[1] https://en.wikipedia.org/wiki/List_of_OpenCL_applications

ComputerGuru 2 days ago 0 replies      
I did some work a while back that involved using genetic algorithms to solve a problem pertaining to PC errors. Long story short, the bottleneck ended up being the generation of an error screen (which is then compared to a pre-existing one in the cost algorithm). We were first using GDI (on Windows), but then we switched to DirectWrite, but couldn't get it to be fast enough to make the algorithm feasible. This definitely piqued my interest!
Mithaldu 2 days ago 1 reply      
How easy would it be to integrate this with projects written primarily in other languages?
sushidev 2 days ago 4 replies      
Couldn't run the example on Ubuntu 16.10Would appreciate anyone's help to get it running:

 ~/dev/rust/pathfinder$ cargo run --release --example lorem-ipsum -- resources/tests/nimbus-sans/NimbusSanL-Regu.ttf Downloading clap v2.20.3 Downloading image v0.12.3 Downloading bencher v0.1.2 Downloading quickcheck v0.4.1 Downloading semver v0.2.3 Downloading glfw-sys v3.2.1 Downloading enum_primitive v0.1.1 Downloading nom v1.2.4 Downloading cmake v0.1.20 Downloading gcc v0.3.43 Downloading vec_map v0.6.0 Downloading unicode-segmentation v1.1.0 Downloading ansi_term v0.9.0 Downloading unicode-width v0.1.4 Downloading term_size v0.2.2 Downloading strsim v0.6.0 Downloading gif v0.9.0 Downloading glob v0.2.11 Downloading png v0.6.2 Downloading scoped_threadpool v0.1.7 Downloading jpeg-decoder v0.1.11 Downloading color_quant v1.0.0 Downloading lzw v0.10.0 Downloading inflate v0.1.1 Downloading deflate v0.7.4 Downloading adler32 v0.3.0 Downloading rayon v0.6.0 Downloading deque v0.3.1 Downloading num_cpus v1.2.1 Downloading env_logger v0.3.5 Downloading regex v0.1.80 Downloading aho-corasick v0.5.3 Downloading thread_local v0.2.7 Downloading regex-syntax v0.3.9 Downloading memchr v0.1.11 Downloading utf8-ranges v0.1.3 Downloading thread-id v2.0.0 Compiling adler32 v0.3.0 Compiling utf8-ranges v0.1.3 Compiling ansi_term v0.9.0 Compiling color_quant v1.0.0 Compiling term_size v0.2.2 Compiling enum_primitive v0.1.1 Compiling lzw v0.10.0 Compiling scoped_threadpool v0.1.7 Compiling bencher v0.1.2 Compiling winapi-build v0.1.1 Compiling inflate v0.1.1 Compiling unicode-width v0.1.4 Compiling unicode-segmentation v1.1.0 Compiling glob v0.2.11 Compiling nom v1.2.4 Compiling gif v0.9.0 Compiling num-integer v0.1.32 Compiling rand v0.3.15 Compiling memchr v0.1.11 Compiling aho-corasick v0.5.3 Compiling regex-syntax v0.3.9 Compiling deflate v0.7.4 Compiling gcc v0.3.43 Compiling semver v0.2.3 Compiling strsim v0.6.0 Compiling deque v0.3.1 Compiling winapi v0.2.8 Compiling num_cpus v1.2.1 Compiling vec_map v0.6.0 Compiling lord-drawquaad v0.1.0 (https://github.com/pcwalton/lord-drawquaad.git#171a2507) Compiling num-iter v0.1.32 Compiling kernel32-sys v0.2.2 Compiling clap v2.20.3 Compiling num-complex v0.1.35 Compiling thread-id v2.0.0 Compiling thread_local v0.2.7 Compiling num-bigint v0.1.35 Compiling rayon v0.6.0 Compiling cmake v0.1.20 Compiling jpeg-decoder v0.1.11 Compiling num-rational v0.1.35 Compiling num v0.1.36 Compiling glfw-sys v3.2.1 error: failed to run custom build command for `glfw-sys v3.2.1` process didn't exit successfully: `/home/mich/dev/rust/pathfinder/target/release/build/glfw-sys-66de5311db1a83bd/build-script-build` (exit code: 101) --- stdout running: "cmake" "/home/mich/.cargo/registry/src/github.com-1ecc6299db9ec823/glfw-sys-3.2.1/." "-DGLFW_BUILD_EXAMPLES=OFF" "-DGLFW_BUILD_TESTS=OFF" "-DGLFW_BUILD_DOCS=OFF" "-DCMAKE_INSTALL_PREFIX=/home/mich/dev/rust/pathfinder/target/release/build/glfw-sys-79c50ef4a5edfcd6/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_C_COMPILER=/usr/bin/cc" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC -m64" "-DCMAKE_CXX_COMPILER=/usr/bin/c++" "-DCMAKE_BUILD_TYPE=Release" -- The C compiler identification is GNU 6.2.0 -- Check for working C compiler: /usr/bin/cc -- Check for working C compiler: /usr/bin/cc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Looking for pthread.h -- Looking for pthread.h - found -- Looking for pthread_create -- Looking for pthread_create - not found -- Looking for pthread_create in pthreads -- Looking for pthread_create in pthreads - not found -- Looking for pthread_create in pthread -- Looking for pthread_create in pthread - found -- Found Threads: TRUE -- Could NOT find Vulkan (missing: VULKAN_LIBRARY VULKAN_INCLUDE_DIR) -- Using X11 for window creation -- Configuring incomplete, errors occurred! See also "/home/mich/dev/rust/pathfinder/target/release/build/glfw-sys-79c50ef4a5edfcd6/out/build/CMakeFiles/CMakeOutput.log". See also "/home/mich/dev/rust/pathfinder/target/release/build/glfw-sys-79c50ef4a5edfcd6/out/build/CMakeFiles/CMakeError.log". --- stderr CMake Error at /usr/share/cmake-3.5/Modules/FindX11.cmake:439 (message): Could not find X11 Call Stack (most recent call first): CMakeLists.txt:192 (find_package) thread 'main' panicked at ' command did not execute successfully, got: exit code: 1 build script failed, must exit now', /home/mich/.cargo/registry/src/github.com-1ecc6299db9ec823/cmake-0.1.20/src/lib.rs:573 note: Run with `RUST_BACKTRACE=1` for a backtrace. Build failed, waiting for other jobs to finish... error: build failed

Go 1.8 Released golang.org
320 points by petercooper  9 hours ago   134 comments top 21
grey-area 5 hours ago 5 replies      
OK, so now this is out, how about we discuss what is actually in this release!

The sort pkg now has a convenience for sorting slices, which will be a nice shortcut instead of having to define a special slice type just to sort on a given criteria, you can just pass a sorting function instead.

HTTP/2 Push is now in the server, which is fun, but like context might take a while for people to start using in earnest. Likewise graceful shutdown. Is anyone experimenting with this yet?

Plugins are here, but on Linux only for now - this will be interesting long term for things like server software which wants to let other compile plugins for it and distribute them separately, presently that has to be compiled in to the main binary.

Performance: GC times are now down to 10-100 microseconds, and defer and cgo are also faster. Compilation time improving but still not close to 1.4.

GOPATH is optional now, but you still do need a path where all go code is kept, perhaps eventually this requirement will go away - GOPATH/pkg is just a cache, GOPATH/bin is just an install location, and GOPATH/src could really be anywhere, so I'm not sure if long term a special go directory is required at all if vendoring takes off, then import paths could be project-local.

There's a slide deck here with a rundown of all the changes from Dave Cheney:


Finally, as someone using Go for work and play, thanks to the Go team and everyone who contributed to this release. I really appreciate the incremental but significant changes in every release, and the stability of the core language.

schmichael 8 hours ago 2 replies      
Linking to the Github release seems more like a grab for fake Internet points than a useful post.

A useful post would be waiting for golang.org to be updated and linking to the official release notes.

Edit: Thanks to whoever updated the link to point to something useful at least. Still would have preferred this came down until the release was actually posted.

clumsysmurf 7 hours ago 3 replies      
I'm wondering if "The Go Programming Language 1e" (Donovan / Kernighan) is still relevant enough to be used as a first book for self teaching.


jbergstroem 8 hours ago 0 replies      
I reckon h2 push (https://beta.golang.org/doc/go1.8#h2push) support will be big for web servers like caddy and traefik.

Caddy already has a few interesting ideas on how to use this: https://github.com/mholt/caddy/pull/1215#issuecomment-256360...

old-gregg 3 hours ago 0 replies      
1.7 release was great for finally addressing the issue of ever-growing size of produced binaries. [1] Moreover, some additional improvements were promised in 1.8 which I was looking forward to.

But after upgrading to 1.8 I am now observing 3-4% binary increase vs 1.7, so the trend is again reversed back to fatter binaries. :(

[1] https://blog.golang.org/go1.7-binary-size

gravelc 3 hours ago 0 replies      
I'm seeing significant performance increases in my app (small RNA aligner). Haven't profiled yet, so not sure where the gains are coming from, but happy all the same.
KAdot 6 hours ago 0 replies      
Official release blog post https://blog.golang.org/go1.8.
modeless 8 hours ago 0 replies      
I see stop-the-world garbage collection pause times have been reduced to microseconds which is great. But for many applications the pause times for single threads still matter. Any numbers for that?
speps 7 hours ago 0 replies      
> The DefaultTransport.Dialer now enables DualStack ("Happy Eyeballs"[1]) support, allowing the use of IPv4 as a backup if it looks like IPv6 might be failing

[1] https://tools.ietf.org/html/rfc6555

tempw 8 hours ago 3 replies      
A little not so related question, what is/are the most used stack for web applications in Go? How common if at all would be a Go backend & React for instance frontend stack?
enahs-sf 6 hours ago 2 replies      
random question here, but what are people using in the way of ORM/sql access with go? Just database/sql or are there any other good packages worth checking out. I don't expect a full-blown activerecord style ORM, but something that takes a bit of the pain out of mapping structs to records.
claudiug 5 hours ago 0 replies      

so go is becoming more and more better for lower GC tasks

watbe 4 hours ago 0 replies      
Final release notes: https://golang.org/doc/go1.8
Ice32 8 hours ago 2 replies      
Why is the latest stable version on their website still 1.7.5? Where can I download 1.8 installation for linux (not source code)?

I've just removed my previous version and then I realize there's no 1.8 on their website.

jraedisch 6 hours ago 0 replies      
I am just leaving the release party in Hamburg, Germany and looking forward to simpler sorting and all the other improvements. Awesome stuff and awesome people!
perseusprime11 1 hour ago 0 replies      
What is the best place to start if I want to learn Go? Any books? Tutorials?
0xdeadbeefbabe 5 hours ago 0 replies      
Yay Mips32!
bitmapbrother 3 hours ago 1 reply      
Does anyone know the roadmap for Go 1.9?
eknkc 8 hours ago 1 reply      
rshetty10 8 hours ago 0 replies      
Hurray, Finally
The Biggest and Weirdest Commits in Linux Kernel Git History destroyallsoftware.com
390 points by gary_bernhardt  2 days ago   51 comments top 14
curuinor 2 days ago 4 replies      
Clauset Shalizi Newman 2007 has not-nice things to say about the classic physicist's idiot trick of fitting power law distributions by drawing a straight line on a log-log graph: it's got huge bias. https://arxiv.org/abs/0706.1062

However, the other difficult thing about power law distributions is that the dataset size requirements for proper determination of the fact that it's a power law distribution are occasionally incredibly difficult. So their critique is very strong, given the comparative lack of data. It is often the case that computer systems, with the overflowing reams of data, are still not enough. Note that the paper I cited up there suggests MLE and then a Kolmogorov-Smirnoff test, so it'll say a lot of things aren't power laws that could well be.

Another way to look at it is from a more geometric point of view. The metric entropy of any generic system of variables is defined as the sum of the positive Lyapunov exponents: and as an "entropy" that quantity does have a lot of commonalities with the other entropies. But to have positive Lyapunov exponents is often to have a chaotic dynamics, so it could just be conjectured that the time series of commits and merge octopus sizes in kernel git history is chaotic, so the evolution of the time series will be fractal in nature.

But it's also really fucking hard to confirm or deny that one, because there are varied and strange definitions of chaos itself and the methods that have been suggested to measure Lyapunov exponent in real systems are arcane and difficult. You could try some synchronization methods, but they remain arcane and crap. Fractal measurement methods are also shitty and full of dark magic.

One neat little trick might be to discretize the series, symbolic dynamics-style (it's already discretized but discretize further, into like percentiles or something) and run it through one of the dynamical machine learning dealies to see if there's patterns. Not too much literature on that but it's a thing that some randoes in like 2004 or something did

cpobrien 2 days ago 3 replies      
There is a mention of the 66 parent merge from Linus himself:


geofft 2 days ago 0 replies      
Another interesting piece of trivia: the very first more-than-two-parent merge in the kernel history is a mistake. The second and third parents are the same commit.

 commit 13e652800d1644dfedcd0d59ac95ef0beb7f3165 Merge: 4332bdd 88d7bd8 88d7bd8 Author: David Woodhouse <dwmw2@shinybook.infradead.org> Date: Sun May 8 13:23:54 2005 +0100 Merge with master.kernel.org:/pub/scm/linux/kernel/git/torvalds/linux-2.6.git

gsylvie 2 days ago 1 reply      
I don't like OP's definition of divergence. I prefer to take the size of the diff along first-parent instead.

Here's how I would do it:

 time git log -m --first-parent --shortstat --pretty="%H" --min-parents=2 | grep -v '^$\|3e1dd193edefd2a806a0ba6cf0879cf1a95217da' | sed 's/.* file.* changed,//' | sed 's/insertion.*,/+/' | sed 's/deletion.*//' | sed 's/insertion.*//' | sed 's/^\ \(.*\)\ $/\$\(\(\1\)\)/' | xargs -d '\n' -L 2 echo echo | bash | sort -k 2,2 -g 
Note: I skip 3e1dd193edefd2a806a0ba6cf0879cf1a95217da because that commit has no diff along first-parent, and thus screws up my xargs result (which depends on every 2nd line having the --shortstat output).

Of course "--first-parent" doesn't guarantee that we're walking the mainline (see: https://developer.atlassian.com/blog/2016/04/stop-foxtrots-n... ), but it usually is.

On my laptop it takes 3 mins 30 seconds. Here are the 5 biggest merges by this definition:

 099bfbfc7fbbe22356c02f0caf709ac32e1126ea 463702 3f17ea6dea8ba5668873afa54628a91aaa3fb1c0 466320 ce519e2327bff01d0eb54071e7044e6291a52aa6 500074 7ea61767e41e2baedd6a968d13f56026522e1207 504965 f063a0c0c995d010960efcc1b2ed14b99674f25c 569691
And here's "git show" for those 5:

 099bfbfc7fbb 2015-06-26T13:18:51-07:00 Merge branch 'drm-next' of git://people.freedesktop.org/~airlied/linux 3f17ea6dea8b 2014-06-08T11:31:16-07:00 Merge branch 'next' (accumulated 3.16 merge window patches) into master ce519e2327bf 2009-01-06T17:04:29-08:00 Merge git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging-2.6 7ea61767e41e 2009-09-16T08:11:54-07:00 Merge git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging-2.6 f063a0c0c995 2010-10-28T12:13:00-07:00 Merge git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging-2.6

SEJeff 2 days ago 0 replies      
Some of my favorite commits come from Rusty Russel, who wrote the lguest toy hypervisor documentation as a story:


brongondwana 2 days ago 0 replies      
It only has one parent, but this would be the commit that I'm least proud of (not in Linux, obviously):


Showing 126 changed files with 14,128 additions and 20,617 deletions.

(ok, I'm pretty proud of reducing code size by 6k+ lines while improving lots of stuff, but the commit is a shitshow)

userbinator 2 days ago 1 reply      
GitHub's logo always reminds me of the octopus merge; not sure if it was chosen for this reason, but I think it's quite suitable.
cperciva 1 day ago 1 reply      
Octopuses are more common than you might expect

The etymologically correct plural is octopodes. (Some people accuse "octopodes* of being pedantic, but as I see it "pedantic" is just a euphemism for "correct in a way I don't like".)

metrognome 2 days ago 3 replies      
I think Gary's commit counts are off:

 $ git log | wc -l
This should count the number of lines in the entire git log, including metadata (not just commits). I think he means this:

 $ git log --oneline | wc -l
The number of commits for Rails should be closer to 61,000.

majewsky 1 day ago 1 reply      
I used octopus merges once for a deployment system that I built when my team switched from SVN to Git. Since there were a lot of developers working on different parts, it was many times required to test multiple different changes in parallel in the QA system.

I built a small web UI where developers could select and unselect development branches, and it would octopus-merge all selected branches into the master branch, and force-push that state onto the QA branch (and deploy it to QA, of course). So QA would always be master + all development branches that were currently being verified. By using a Github webhook, it would update the QA system whenever master or one of the branches being verified was pushed to. I'm not in that team anymore, but I think that deployment tool is still humming along nicely.

smallnamespace 2 days ago 1 reply      
Slight article nitpick: a distribution that 'looks like a straight line' in a log-log plot is often not power-law distributed.

One could say that the distribution has a fat one-sided tail though.

tomatokiller 2 days ago 2 replies      
Has anyone asked Laxman Dewangan what he was up to with that initial commit and merge thing?
kijin 1 day ago 2 replies      
> "Christ, that's not an octopus, that's a Cthulhu merge"

Perhaps git should throw a warning when you try to do an octopus merge with more parents than an octopus has legs. If you really want to proceed, add the --cthulhu option. The default behavior would be --no-cthulhu.

behm 2 days ago 1 reply      
That was the worst diagram today.<1 Commits on the y-axis? Where would be 30 on the x-axis? Can't tell if you only have 3 markers on a log axis.
SecureDrop An open-source whistleblower submission system securedrop.org
337 points by spaceboy  1 day ago   77 comments top 13
eganist 1 day ago 0 replies      
For those who don't know Garrett Robinson (who heads SecureDrop's development), he's been extremely dedicated to user privacy issues and first amendment concerns. I may occasionally differ from his views, but I admire the passion he's poured into both his work at Mozilla and into SecureDrop.


hackuser 1 day ago 5 replies      
SecureDrop uses Tor Browser, as do many other public interest security solutions. However, a respected security expert here on HN recently said of Tor Browser:

the Tor Browser might be the least safe browser to use of all available browsers that can be installed on modern computers. It is a perfect storm of "inferior security design" and "maximized adversarial value per exploit dollar spent". / Don't use Tor Browser.

He recommends Chrome (presumably over the Tor network). I tend to believe the expert, because IME real security expertise (as opposed to technically sophisticated people reading about security and trying to DIY) is rarely utilized and applied even by prominent organizations and projects. But I wish someone would reconcile all of this.

EDIT: Some clarifying edits

tptacek 1 day ago 7 replies      
This is a trivial Flask file uploading application, with a "code name"-based feedback system, wrapping GnuPG's Python bindings, intended to be run on Tor.

The security it provides is marginal, but it's so simple that it's not the part of anyone's stack that's most likely to be compromised.

I think a significantly better version of this could be built. What makes doing that tricky is that you want to retain the almost hello-world simplicity of this app, because the big reason not to run something like this is the likelihood that the server itself will have flaws.

On the other hand, it's 2017, and you can also accept files over secure messengers.


Amusingly, people seem to think that these are bad things to say about an application like SecureDrop.

h4waii 1 day ago 1 reply      
SecureDrop is also in use by CBC, a publicly-funded National broadcaster in Canada, and is actually implemented and managed properly -- regardless of the quality of SecureDrop itself.


The gateway site is only accessible over HTTPS, then it's to an .onion via a link to Torbrowser, and mentions of TAILS, all caveats with using the stated software applies though.

benwikler 1 day ago 1 reply      
RIP Aaron Swartz, who originally built this. He'd be 30 now.
secfirstmd 1 day ago 1 reply      
Also worth shouting out to Global Leaks, a similar sort of system with some interesting other features.


unicornporn 1 day ago 1 reply      
Do not forget https://onionshare.org/

An excellent alternative to SecureDrop. At least so it seems...

mindslight 1 day ago 0 replies      
Tangential and more applicable to a different style of leak, but I'd be interested in seeing the development of some protocol ideas for authenticating leaks to gain confidence the leaker is actually within a given organization. Otherwise we're left not knowing if a casual leaker is for real or just entertainment twitting.

One rough idea is that large organizations make specific press releases or announcements, that a precommitment could demonstrate privileged access to.

Another idea would be inclusion of some internal communication, which other members of the organization could confirm. This would require those other members to be sympathetic to the leaking, and also not worried about reprisals for speaking publicly like so. This probably isn't useful on its own, but the basic mechanism could be combined with other means to derive utility without public attestation.

The biggest issue is (of course) an adversarial organization subtly changing to-be-published information, to sniff out the actual leaker. Which is why I'm envisioning the need for some formality that could quantify and mitigate such leakage.

saycheese 1 day ago 1 reply      
Recently review the SecureDrop and was suprised how many main stream media companies to not provide a way for leakers to safely leak information to them.
amelius 1 day ago 3 replies      
Is this based on Tor? Or are whistleblowers expected to use Tor on top of this?
benevol 1 day ago 9 replies      
I'm not sure the problem is a lack of leaking solutions that we can trust, especially as long as WikiLeaks is around.

The problem I see is that there will be no more important leaks:

a) Given how around 50% the US population was brainwashed by government and media into believing Snowden is a traitor,

b) Given the fact that America has elected a president who wants Snowden executed,

c) Given that the NSA has locked down their systems completely since Snowden's revelations.

Who would want to take these risks to leak anything just to be put on "the list" by their own country and People?

If Snowden's leaks were not enough to get people thinking then the only thing that will is serious pain and suffering. And that is what I personally expect to come (for the lower and middle class, at least).

elcct 1 day ago 1 reply      
One could use Bitmessage for leaks - just create a channel and let people publish data to it.


eptcyka 1 day ago 0 replies      
If a site like this doesn't yell at you for accessing over just https and not tor, you can only expect it to be run by three or four letter agencies.
DuckDuckGo Donates $300K to Raise the Standard of Trust Online spreadprivacy.com
316 points by AdamSC1  15 hours ago   77 comments top 14
AdamSC1 15 hours ago 5 replies      
Disclaimer: I work at DuckDuckGo and this is on our company blog at spreadprivacy.com

At DuckDuckGo our overall goal is to raise the standard of trust online. To do that we've focused heavily on search, but try and support organizations that push privacy forward in other ways.

For the 7th year in a row, we've announced our donations to organizations and FOSS projects that help keep everyone a little safer in our digital world.

Donation Recipients:

$100,000 - Freedom of the Press Foundation

$75,000 - World Privacy Forum

$29,000 - Whisper Systems

$25,000 - Privacy Rights Clearinghouse

$25,000 - Tor

$25,000 - Electronic Frontier Foundation

$5000 - American Civil Liberties Union

$5000 - Access Now

$2500 - The Calyx Institute

$2500 - Center for Democracy & Technology

$1000 - Restore the Fourth

$1000 - Patient Privacy Rights

$1000 - Online Trust Alliance

$1000 - Tech Freedom

$1000 - Demand Progress

drallison 12 hours ago 0 replies      
Kudos to DuckDuckGo for supporting these organizations which are trying to keep everyone a little safer. HN readers who agree should donate to the organization(s) with which they resonate. Even small donations can make important differences.
mbaltrusitis 10 hours ago 9 replies      
I'm sincerely confused by people saying that DuckDuckGo has vastly inferior search results. Mind you, I search mainly programing related things so maybe DDG better services that niche? What types of searches are people doing that DDG doesn't provide suitable results?
mrschwabe 12 hours ago 1 reply      
DDG has been nailing it lately - really digging their Twitter feed too; they do not pull punches.
jasonkostempski 9 hours ago 2 replies      
DDG has 300K to donate? I guess they're doing much better than I expected. How in the world do they make money?
JimWestergren 11 hours ago 2 replies      
Great. I think you should consider Let's Encrypt as well.
bsclifton 6 hours ago 0 replies      
I just recently (last week) switched over to DDG; it's been solid- I've gotten really great results versus last time I tried them out (5 or 6 months ago?)

I love their vision, the services they provide, and what they've done here (with the donations) <3

twsted 5 hours ago 0 replies      
More than two years using DDG on every device, very satisfied. I'm suggesting it to everybody.
almost_usual 6 hours ago 1 reply      
I've been using 100% DuckDuckGo in combination with Brave browser for the past 3 months and have not looked back.
thr0waway1239 10 hours ago 0 replies      
It seems the fastest way to spread privacy is to bring a sudden stop to its erosion. My suggestion: ban all acquisitions by the top software companies for the next 5 years :-)

As a bonus, you will foist some competition on the market as companies which thought they were on an acquisition path will now be forced to show real profits.

X86BSD 10 hours ago 1 reply      
I've moved all our devices to DDG recently. It used to be goog on all our idevices. Mainly I was just to lazy. But I like DDG results and the engine and that they give back. Also they use FreeBSD which is a plus.

Seems like a great company and search engine to me.

snackai 12 hours ago 8 replies      
Great to see support for all those important projects, but maybe DDG should use that money to hire someone who makes their Search Results better so we would not have to go back to Google on every other more sophisticated search.
technologyvault 10 hours ago 1 reply      
Great idea, but it seems like it's going to take a lot more than $300k, right?
Dirlewanger 7 hours ago 4 replies      
DuckDuckGo most likely tracks everything users do. Their founder sold his previous startup along with all its user's data. Outside of the cozy story they spin, I don't know why users think this company is a champion of user privacy.
Is PostgreSQL good enough? renesd.blogspot.com
387 points by richardboegli  2 days ago   136 comments top 22
brightball 2 days ago 3 replies      
I've been attempting to preach the PostgreSQL Gospel (http://www.brightball.com/articles/why-should-you-learn-post...) for a few years now about this exact same thing.

When you look at your database as a dumb datastore, you're really selling short all of the capabilities that are in your database. PG is basically a stack in a box.

Whenever I started getting into Elixir and Phoenix and realized that the entire Elixir/Erlang stack was also basically a full stack on it's own...and that by default Phoenix wants to use PG as it's database...I may have gone a little overboard with excitement.

If you build with Elixir and PostgreSQL you've addressed almost every need that most projects can have with minimal complexity.

yorhel 2 days ago 2 replies      
There was a similar talk[1] at FOSDEM, where the speaker describes how, as an experiment, he replaces a full ELK stack plus other monitoring tools with PostgreSQL. He even goes as far as implementing a minimal logstash equivalent (i.e. log parsing) into the database itself.

It wasn't an "we do this at scale" talk, but I'd love to see more experiments like it.

For the impatient: Skip to 17 minutes into the video, where he describes the previous architecture and what parts are replaced with Postgres.

1. https://fosdem.org/2017/schedule/event/postgresql_infrastruc...

cel1ne 2 days ago 5 replies      
I do use PostgreSQL wherever possibly. Add http://postgrest.com/ and nginx as a url-rewriting proxy and you have a performant, highly adaptable REST-Server.
einhverfr 2 days ago 0 replies      
Nice writeup though I would add a few things.

Listen/Notify work great for short-term job queues. For longer term ones, you have some serious difficulties on PostgreSQL which require care and attention to detail to solve. In those cases, of course, you can solve them, but they take people who know what they are doing.

Also in terms of storing images in the database, this is something that really depends on what you are doing, what your database load is, and what your memory constraints are. At least when working with Perl on the middleware, decoding and presenting the image takes several times the RAM that loading it off the filesystem does. That may not be the end of the world, but it is something to think about.

Also TOAST overhead in retrieved columns doesn't show up in EXPLAIN ANALYZE because the items never get untoasted. Again by no means a deal breaker, but something to think about.

In general, PostgreSQL can be good enough but having people know know it inside and out is important as you scale. That's probably true with any technology, however.

Aeyris 2 days ago 7 replies      
Is anyone actually utilising a recent version of PostgreSQL for full-text searching beyond a hobby project? How do you find the speed and accuracy versus Elasticsearch?
jaequery 2 days ago 2 replies      
is it good enough? yes. in fact, its probably an overkill for most. i think the question of good enough wouldve been perfect for sqlite.
mattferderer 2 days ago 0 replies      
Very nice article for those of us who have never used PostgreSQL much. I've been starting to use it with Elixir & this gives me a good understanding of why someone would use it, especially when when starting a new app.

Out of curiosity, does anyone have a favorite article saved that does a great comparison of when to use certain databases?

seibelj 2 days ago 0 replies      
Yes, for all projects and small businesses I start, Postgres and Redis is what I use from the beginning. Then if it ever gets to the point where I need a different DB for something, I replace components with the new tool. People get fascinated with these fly-by-night data stores and put their operations at serious risk. Start with the tried and tested technologies, then carefully augment your stack as needed.
rosser 2 days ago 7 replies      
Using an RDBMS as a work queue is an anti-pattern, but if you're going to do it, you probably can't do much better than LISTEN/NOTIFY.
api 2 days ago 2 replies      
From my experience with both PostgreSQL and RethinkDB (and other NoSQL stores):

For SQL, complex queries, and data warehousing: yes. It's an excellent database and I'm not sure why you'd pick another SQL DB unless it were a lot better on point two.

For high availability and scaling: no, absolutely not.

The problem with the latter is an arcane deployment process and arcane error messages that provide constant worry that you're doing something wrong. It's a many week engineering project to deploy HA Postgres, while HA RethinkDB takes hours -- followed by some testing for prudence... our testing revealed that it does "just work" at least at our scale. We were overjoyed.

The docs for Postgres HA and clustering are also horrible. There are like five different ways to do it and they're all in an unknown state of completion or readiness.

Of course if/when we do want complex queries and more compact storage, we will probably offload data from the RethinkDB cluster to... drum roll... a PostgreSQL database. Of course that will probably be for analytics done "offline" in the sense that if the DB goes down for a bit we are fine. HA is not needed there.

TL;DR: everything has its limitations.

mamcx 2 days ago 1 reply      
Now imagine if we understand that the relational model is no for "just data storage" but also can be use for everything.

The closest thing(1) was dbase/foxpro. You can actually build a full app with it. Send email from the database? Yes. Is not that wrong? Is wrong just because RDBMS (2) made it wrong, not because is actually wrong. Why is better to split in separated languages/run times/models a app than one integrated?

(1): Taking in consideration that neither Fox or any "modern" rdbms have take the relational model to it full extension.

(2): A RDBMS is a full-package with a defined role, and limited capabilities. A relational-alike language will not be a exact replica of that. Not even is demanded to implement a full-storage solution.

The biggest mistake the relational guys have commited is to think always in terms of full-databases instead of micro-database. Ironically, kdb+ (or lisp? or rebol?) could be the closest thing to the idea (where data+code are not enemies but friends).

agentgt 2 days ago 1 reply      
We had to learn this the hard way. We have many of the data/services the article mentions and while we still use them when ever it gets massive we actually will go back to Postgresql.

For example for our internal analytics/logs/metrics we use ELK and Druid but believe it or not these tools despite their purported scaling abilities are actually damn expensive. These new cloud "elastic" stuff cheat and use lots and lots of memory. For a bootstrapped solvent self-funded startup like us we do care about memory usage.

For customer analytics we use... yes Postgresql.

For counters and stream like things we don't use Redis we use Pipelinedb (Postgresql fork). For Cassandra like stuff we use Citus (Postgresql extension).

Some of our external search uses SOLR (for small fields) but Postgresql text search is used for big fields.

The only part of our platform we don't really leverage on Postgresql is the message queue and this because RabbitMQ so far has done a damn good job (that and the damn JDBC driver isn't asynchronous so LISTEN/NOTIFY isn't really useful).

anko 2 days ago 4 replies      
I love Postgres, but the one thing I think sucks is it's COUNT() performance.

I've read all sorts of hacks but I would love for someone to solve this for me!

mooneater 2 days ago 0 replies      
Postgres' awesome extensible type system means it will continue to increase in functionality much more easily than most comparable DBs.https://www.postgresql.org/docs/9.6/static/extend-how.html
TheAceOfHearts 2 days ago 1 reply      
It's really amazing how far you can with a relational database. If you have very minimal constraints, keeping everything in a single place can make life so much easier. Configuration hell is real. I hadn't considered using PG for storing binary data, but I've hacked together a few toy projects where I used mongo and just shamelessly shoved everything in there.

I have a some slightly tangential questions, which I'd love to hear people's thoughts on: How do you decide where to draw the line between what's kept and defined in the application and database? For example, how strict would you make your type definitions and constraints? Do you just accept that you'll end up duplicating some of it in both places? Also, how do you track and manage changes when you have to deal with multiple environments?

crudbug 2 days ago 1 reply      
Has someone played with threading model within Postgres.

I was reading the documents, looks like for every client request Postgres forks a new Process and uses shared memory model.

Using multi-processor threads/coroutines might be useful for scaling it further.

mooneater 2 days ago 0 replies      
How do you get a "hotstandby replica for $5/month"?
mooneater 2 days ago 0 replies      
Another big plus for postgres: PL/Python, PL/R, etc
ckdarby 2 days ago 0 replies      
Why is the font so small on this site?
ausjke 2 days ago 0 replies      
still using php+mysql here as I can find so many documents about their various usage easily.
hartator 2 days ago 2 replies      
I am more the MongoDB bandwagon. Shemaless makes prototyping so much easier. And no migrations!
treve 2 days ago 6 replies      
Or just MySQL. Popular choice, unpopular opinion. I trust it more because it gives me a tried and tested path when I need replication (which tends to happen rather early). My understanding is that Postgres replication is not nearly as battle-tested.
22 Years of Delphi and It Still Rocks marcocantu.com
298 points by vs2  1 day ago   331 comments top 56
alex_duf 1 day ago 11 replies      
I remember using Delphi 2 when I was younger, that's how I learned programming. If you don't mind hearing my personal story, keep reading, otherwise move on to another comment.

By an interesting coincidence I was given a CD with tons of software on it, including Delphi 2. I started playing with it and quickly realised I could create my own programs using that tool.

Later I went to the book store and found a book about it that I couldn't afford. So I came back every week to read as much as I could, then heading back home to try it on my illegal copy of Delphi. (I was 11)

Then I realised there was a help manual embedded in the distribution, so I learned as much English as I could so I could understand that manual.

Little by little I learned about conditions, loops, object programming and made a bunch of terribly crappy yet working games.

No amount of studies could have taught me as much as I learned through using that software. No manual work could have given me the fun I had when programming my very own games. Reading "Delphi" as a headline on hacker news made me feel nostalgic and I figured I'd share my story here.

So thanks Borland, the 11 year old me may not have paid to use that software, but it was enough of a revelation to make me the programmer I am today.

yoodenvranx 1 day ago 10 replies      
Delphi is still unrivaled when it comes to rapid GUI prototyping in combination with easy deployment. It's much easier than QT/GTK and you usually get a standalone .exe with no external dependencies.

I really want to cry when I see the current alternatives... Node/Electron with dozens of MB of runtime and all that Javascript stuff? What went wrong that we end up with this?

mamcx 1 day ago 2 replies      
I'm moderator in one of the oldest/largest delphi forum in latin-america:


Ironically, I don't use Delphi anymore. Yet, I still help people with it and defend it when is possible.

Delphi is amazing. It only have a HUGE problem: His owners.

You can re4ad why Delphi fade away here:https://www.quora.com/Why-did-Borland-fail?share=1

"Borland lost its way when executive management decided to change the company to pursue a different market. ......In the height of the enterprise transformation, I asked Del Yocam, one of many interim CEOs after Kahn, "Are you saying you want to trade a million loyal $100 customers for a hundred $1 million customers?" Yocam replied without hesitation "Absolutely."

analognoise 1 day ago 4 replies      
Shout-out to Lazarus/FreePascal. It is like all the best parts of Delphi 5-7, and it is open source, free, and you can make commercial applications in it!

I have started using it in earnest and it is wonderful - and a company can never take it away from you; there's no "we're shifting focus..." announcements. No paid support for broken updates.

I honestly can't recommend it enough.

krylon 1 day ago 0 replies      
I've mentioned it before, but I'll repeat it. ;-)

At work, I inherited maintenance of an in-house application a coworker that left the company wrote for our accounting department. In Delphi.

When people speak highly of Delphi, they always mention how great the IDE is. Personally, I am not a big fan of IDEs. I was not disappointed, but my mainly, what I do is read and edit the source code.

And I have to admit, that part was a pleasant surprise! I had never looked at or touched Pascal code before, but I was able to make a change to the source - and it worked, the very first time! - within a few days. And the majority of that time was spent figuring out how the code was structured[1] and what the accountants wanted me to do (they always talk to me as if I knew the first thing about accounting).

But after that, it was smooth sailing. It sure helped that my predecessor wrote very readable code, but it seems that ObjectPascal made it very easy to write it that way.

Or, more briefly: Happy Birthday!

[1] The application is about a 10 KLOC in total, which is not small in my book, but not "very large", either. Also, I was both a developer and a sysadmin and a helpdesk monkey, so the phone was ringing about every fifteen minutes.

codewritinfool 1 day ago 2 replies      
As a pascal user since the early 80's, then an Object Pascal user, then a Delphi user since the first version, I love Delphi but to be honest I cannot afford the yearly updates.

For someone that uses it for home projects, $916 for an upgrade is out of the question.

Rambling here, but... Delphi 3/5/7 were incredible design packages. I feel they lost focus when they jumped on the .NET bandwagon - maybe it is just my perception, but maybe some of their internal developers were less focused on native code. Anders leaving was also a big hit. Kylix was yet another distraction.

silveira 1 day ago 4 replies      
The documentation and help pages that came with Delphi were absolutely the best documentation I ever had. Every topic came with examples that you could copy and reuse. Not just snippets but fully functional blocks of code. It was brilliant.
madiathomas 1 day ago 1 reply      
I was a Delphi Developer when I started developing software professionally. I loved programming in it. I jumped ship when Anders Hejlsberg, the Chief Architect of Delphi moved to Microsoft to be a lead architect of C#.NET. I used Delphi 6 and 7. When it was time to move to Delphi 8, Borland decided to target .NET.

The choice was between learning Delphi 8 for .NET or learn Visual Studio .NET. I chose MS.NET because MS is the custodian of the .NET Framework. They are the ones on the driving seat, not Borland.

Looking back, I don't regret the move because Delphi is dying. Posts requiring Delphi are nowhere to be seen on job sites in South Africa.

interfixus 1 day ago 7 replies      
A shocking aspect of Delphi 1 was the gargantuan executables it insisted on producing. There was much forum posting and gnashing of teeth.

Hello World ran to about 100 kB.

Funny thing is, I am just getting into Go, and you know what?

brunock222 1 day ago 0 replies      
12 years ago I did a rude system, part of it using Delphi 7.

They still need, once every 6 months, to compile the EXE file with a small change.

I get paid monthly just to press CTRL+F9 (generate new .EXE file) every 6 months, basically.

I liiiike it :)

frik 1 day ago 1 reply      
I started with QuickBasic/QBasic and Pascal. Though I went with VB4-6 instead of Delphi.

It's sad that RAD isn't popular anymore. 1995 - 1999 was great with Win95 era, everything was so consistent, good documentation. Then Microsoft realised their "The Microsoft Network" lost against the open free WWW and then the announced dotNet (which took until 2003 for them to release v1) - that was the beginning of the end of the great Win32 platform and RAD. HTML with Frontpage and Dreamweaver was just an okay RAD andbthe situation got worse with "no tables, use div" and XHTML 1/2 movements.

masterponomo 1 day ago 3 replies      
I attended a Clipper user group in Atlanta in the mid 1990's, with my dad. He was into Clipper, was trying to get me interested. The Clipper folks were also into Turbo Pascal. A Borland rep was there to give out swag and talk up the imminent release of Borland Delphi. The Clipper crowd was divided--some excited, some not so much. The older guys, like my dad, had come up programming with punched cards and tape drives. For them, a PC with Clipper and Turbo Pascal was plenty advanced enough for small business apps. Dad never did go for Delphi. 20+ years later, I'm doing a quick study of Object Pascal to prep for working on a legacy application at my work, in Delphi.
pjmlp 1 day ago 1 reply      
Delphi was a great environment, but Anders and other key persons leaving Borland, while the company lost track which customers they should target, besides the "clever" idea to change the company's name, killed it.

I know of a few companies in Germany and Netherlands that still use it, but it is hard to get offers.

And we had to wait 15 years until .NET started to offer a compilation model similar to Delphi (only for UWP apps).

While Java kind of outsourced it to third party JDK vendors due to Sun's attitude against AOT compilation, oh well at least Java 9 will bring the first steps towards support it.

aylons 1 day ago 2 replies      
One of the first environments I ever programmed was Delphi. Had lots of fun with it, and it had a special place in my heart...

Fast forward 20 years, and I'm a hardware developer using Altium a lot. As some here may be aware, Altium is a multi-gigabit piece of CAD/EDA behemoth written - and still maintained - in Delphi. 3D, DirectX, everything in Delphi.

Just yesterday and today I had easily reproducible BSODs by using a very basic feature (routing nets). Memory leaks galore - most people I know have the habit of shutting Altium down now and then just to avoid a crash. Success is low - it crashes a lot, I'm getting tired of unhandled exceptions windows.

And this way, I realize even great development tools age badly. Of course this is not all Delphi's fault, but it shows how tech is a Red Queen's race: you must run faster and faster to keep in the same place.

eli_gottlieb 1 day ago 0 replies      
I got my start programming with Delphi 5. It was an excellent environment to learn in, since I could always see something happening based on what I had done.

Better: the Delphi component libraries made writing a GUI application easy. I've never seen anything of their quality and ease up through today, and I still miss being able to get an application running by just subclassing some standard components and writing the core logic I needed.

fredsted 1 day ago 1 reply      
Sometimes I kind of miss Delphi 7. The way it makes it easy to create apps made programming so fun.

Such a shame it went downhill after the .NET stuff.

kweinber 1 day ago 1 reply      
After the death of Python, Oracle and Delphi go really well together... give it a shot if you want to see the future.

Edit: this is a historical reference... search for "Oracle Delphi Python"

Joeri 1 day ago 0 replies      
I learned VB3 at 13, and moved to delphi at 14. It was amazing. As easy as VB, but all the power of C++. I made my own DLL's to use in VB, just because I could. It was the coolest thing ever.

Fast forward years later, and plenty of intermediate languages (C, C++, Java, ...) I got hired into my first professional software developer job ... as a delphi developer in a company with a big multi-million line delphi codebase they wanted to migrate to the web. I had done some web development, so I ended up writing features on both the delphi and web teams, often the same feature. So I became intimately familiar with the trade-offs of delphi vs web development.

The thing about delphi: it was/is insanely productive. In the beginning it took about 3 to 5 times as long to write a comparable feature on the web side. It ended up driving me to research cutting edge web dev techniques to find some way to approach the productivity levels that delphi gave. In the end we almost got there, using rich frameworks and a component-driven UI. But to this day the delphi team can still get a feature done faster than the web team, and that's despite an IDE which is much weaker than webstorm/intellij. Object Pascal and the VCL are just that good.

However, I wouldn't do a new project in Delphi. You're locked into an ever more expensive product with an uncertain future, and the productivity advantages just aren't worth it anymore. You can get close enough using an open source dev stack, and the value of having all your tools be open and free is significant.

binaryapparatus 1 day ago 2 replies      
I am bit older than average so I used Delphi in its golden days. Even today few of the old customers need old apps maintained so I keep Delphi 7 ready.

It _is_ great IDE/language.

orionblastar 31 minutes ago 0 replies      
I learned UCSD Pascal on an Apple 2e, and then Turbo Pascal on an IBM PC XT in high school. 1984 to 1986. It helped me learn more languages like Ada, we had Janus Ada for DOS.

Delphi 1.02 I got from some CD in a UK magazine sold in the USA. They gave away free copies of commerical software for buying the magazine for $10 or so.

I did better in Visual Basic because I got jobs that required it.

mannykannot 1 day ago 0 replies      
Verity Stob has chronicled the tribulations of Delphi enthusiasts in a series of commentaries:


ahacker15 1 day ago 7 replies      
Delphi is still popular in the south of Brazil. It's dying, but slowly.

There's a lot a legacy application written in Delphi here. Some old programmers, that only know to program in Delphi, may even build new apps with it.

It may not be that modern today, but there's the "pay the bills" mindset.

I think Delphi will die, but only because it costs a fortune today. If it had a resonable price, many people would continue to use it indeterminately.

johnny_reilly 1 day ago 0 replies      
My first gig was writing telecoms software using Delphi 3. Good times! Then Inprise bought Borland and shipped a buggy version of Delphi 4 which was a pain. Like many others I jumped ship when I heard what Anders was doing with C#. I don't regret the move but I thought Delphi was great.
mmgutz 16 hours ago 0 replies      
I remember Borland vs Ms

 Delphi vs VB TASM vs MASM Borland C++ OWL vs Microsfot C++/MFC Philippe Khan vs Gates
Delphi was the raddest RAD tool. The free VCL components for it were often better than commercial VB ActiveX components. Delphi executables were faster. Delphi could statically link everything.

I eventually had to move to VB when my Delphi job dried up. VB felt like a downgrade. I remember constantly referring to Dan Appleman's book so I could use Win32 APIs to work around the limitations of VB. I'm not knocking VB as it's one of my favorite tools too, but Delphi was the cream of the crop.

bigtunacan 1 day ago 4 replies      
I never used Delphi, but I have a couple of friends that were die hard Delphi fanatics. Eventually they moved on to C# as all of the Delphi jobs dried up with Delphi's continually decreasing use. They still swore how they thought it was the best thing they ever worked with, but at the end of the day you have to pay the bills.

Maybe it really is great, I don't know. I was a bit surprised looking at Tiobe (http://www.tiobe.com/tiobe-index/) that they rank it as the 9th most popular language right now. Where is this actually being used? And the price tag is incredible!

vram22 1 day ago 0 replies      
Somewhat long and interesting thread about Delphi from a while ago, on HN:

Delphi why won't it die? (2013) (stevepeacocke.blogspot.com)


Did a search in hn.algolia.com just now to find the thread, and it was the top result when the setting was "By popularity":


netrap 1 day ago 0 replies      
Free Pascal/Lazarus is great. Glad they finally have Delphi Starter free but still it's not enough and the limits suck.
jenkstom 23 hours ago 0 replies      
I too often see a fundamental misunderstanding of what made Delphi so great. Yes, you can drag and drop controls onto a form in a lot of tools. It can compile native code. And yes, Object Pascal was nice to read and maintain.

But the real issue is rapid database applications. Being able to create business apps very rapidly was always Delphi's thing. It was a really fantastic replacement for Paradox and brought Turbo Pascal and Paradox together to create something amazing.

For that there still isn't anything. There are a lot of things that are close, but nothing with the same level of power and speed. The closest thing I've found is Django. Hopefully that clarifies: it's not about GUI anything.

ChuckMcM 1 day ago 0 replies      
I was just having this conversation with an acquaintance about the software glut. When you do something well, you don't have to do it again, just enough to keep it compatible with the other things it needs to work with. So it takes a team of 50 people to make a product and then 10 people to maintain it forever. So where do those 40 people go?

The latest answer has been annual licenses.

gerryk 10 hours ago 0 replies      
I used to code in Clipper in a past life. Many ex-Clipper devs became Delphi devs when the Great Fragmentation happened (multiple competing OO frameworks for Clipper).
igivanov 1 day ago 1 reply      
Hey Marco, I still have your Delphi 5 book :-)

FWIIW, where I work, I started using Delphi 5 when it came out. As of now we have about a dozen D5 desktop applications all essential for the core business that are in active use and supported as necessary, running happily on Windows 10. I don't see why it won't continue like this for another 15 years... The bosses (who are non-programmers) don't really care that D5 is out of fashion - because it all just works (and rocks).

The only reason I couldn't upgrade to a later version (Unicode is one thing that would be useful) is I am stuck with one critical and long discontinued grid component.

Looking back I am not sure how we could have lived w/o Delphi. Well maybe it's an exaggeration, but certainly life would have been more difficult. Everything else from before .NET/C#/Windows Forms looks like a nightmare.

nnq 1 day ago 9 replies      
Was wondering... is there any modern all-in-one RAD-tool nowadays? And if not... why?

Has there ever stopped being a need for this?

tmsldd 1 day ago 0 replies      
Thanks for making me feel so old... we had a programming lecture twice a week and I was carrying one of those 10 kg "Delphi The Bible" book in my backpacks... that made me feel like a programmer somehow.
reitanqild 1 day ago 1 reply      
I guess I should have started on a greenfield project.

For me it was less rosy.

I remember my boss trying to get me started on Delphi.

We basically had two or three Delphi developer workstations available because setting up one that could compile the projects we had in our vcs was a three day task - and it was only possible with the help of our resident Delphi consultant.

Stuff like that has made me love Maven and Java.

Same goes for Visual Basic that I once used to love.

Still I feel I could have loved it if I didn't start with 10 years of accumulated references to unsupported packages. :-/

jksmith 1 day ago 0 replies      
I've written tons and tons of code in Delphi and remember it fondly. A few issues I had with it toward the end included Hjelsberg turning it into a kitchen sink, me too language, much like IMO the direction C# has gone in, and walking into projects with tons of business logic right behind the forms, empty except blocks, and codebase full of warnings when compiled. Fix your warnings people.
Havoc 1 day ago 1 reply      
It's good for teaching purposes because it forces a very structured clear line of thinking.

Not convinced it's still best as a practical language though.

no_wizard 1 day ago 0 replies      
Admittedly I haven't used Delphi but his reminds me of learning to program with realBASIC (Xojo).

The book with the IDE was free and easy enough to comprehend and it introduced all the basics of OOP.

Great times. I wonder if lots of people have had this same experience with VB .NET

duracel 1 day ago 0 replies      
Delphi was much more better choice than many others in 90's and early 2000. Fast compiler and quite optimised binaries also.
eggy 16 hours ago 0 replies      
I've used Lazarus and it's amazing, however I'm hoping Red lang becomes the new Delphi in a way.
raister 1 day ago 2 replies      
It rocks to the point of server unavailability.
adam-falafel 1 day ago 0 replies      
My first job out of college was as a tech support intern for Delphi, first answering pre-sales and installation calls, and eventually moving up to answering paid support calls. From there, I joined a consulting company and wrote and maintained LOB applications written in Delphi. Good times. Jumped ship to .NET in 2003.
barking 1 day ago 0 replies      
Back then there was no internet and I knew no programmers.I remember seeing the yellow borland boxes at PC World and not really understanding what borland were. Afterwards I bought my first copy of vb secondhand (lots of manuals came with it), from someone who was getting into Delphi instead.
fithisux 1 day ago 0 replies      
People seem to forget that some Delphi libraries are intertwined with assembly and work around Delphi inefficiencies. This made some libraries very hard to port to FreePascal. A typical example is DevC++ and its dependencies.
nottorp 1 day ago 2 replies      
I can't help but notice that all the article talks about is Delphi versions up to 6... which is where it started to go downhill.

Are recent versions of Delphi actually usable? I haven't tried anything beyond 8, and from my (admittedly rusty) experience the best version was 6.

snarfy 1 day ago 0 replies      
If I had to write a native COM application in Windows, I'd still use Delphi.
julius 1 day ago 0 replies      
I remember a tool called "Dirty Little Helper" (if I am not mistaken), which had all the Delphi answers, I would ask SO nowadays (eg. how to make a windows tray icon?)But it was an offline application.
mrded 1 day ago 0 replies      
Delphi 6 was my first one :)
acqq 1 day ago 3 replies      
> I'll start blogging on the Delphi language coming back to Linux tomorrow!

To taste the cross-platform IDE for Rapid Application Development today:



"Why use Lazarus?

No dependencies!

With Lazarus you can create programs which do not require any platform dependencies [1]. The result of it is the user of your program does not need to install any further packages, libraries or frameworks to run your software.

[1] Linux/BSD applications may depend on GTK2 or alternatively QT. Some add-on packages may also add dependencies of their own

Can be used in commercial projects

Some IDEs restrict their license to only non-commercial development. Lazarus is GPL/LGPL [2][3] which permits using it in building commercial projects.

[2] LGPL with additional permission to link libraries into your binaries.[3] Some additional packages come with various licenses such as GPL, MPL, ..."

huan9huan 1 day ago 0 replies      
After several years programming and using Borland, finally, I got to know what "Turbo" meaning (I am Chinese for not good in English)
b4xt3em4n 1 day ago 0 replies      
The best environment I've used to build software.
lunchladydoris 1 day ago 0 replies      
I love the magazine scans. They're almost always a good reminder of how much things have changed, and how cheap things have become.
pcunite 1 day ago 0 replies      
Delphi 6 "Personal Edition" was my intro into Pascal. I absolutely loved it.
melling 1 day ago 5 replies      
"Fastest compiler -- 85,000 lines per minute"

What do you get 22 years later on an Intel i7?

alsadi 1 day ago 1 reply      
my favorite color is better than yours.

Would someone please point to his/her blog saying that Tcl/tk being the universal scripting language?

Yuioup 1 day ago 0 replies      
Delphi is dead. Nothing to see here. Move along.
avodonosov 1 day ago 0 replies      
ah, Delphi...
YC Annual Letter 2017 samaltman.com
328 points by sama  9 hours ago   140 comments top 23
optimusclimb 7 hours ago 10 replies      
> 4) Does this company have a clear and important mission?

> Without this, I usually get bored. More importantly, companies that dont have this usually have a hard time recruiting enough great people to work with them, and thus struggle to become very large.

I find this one interesting. When Google started, there were many incumbent search engines. Their mission was to provide better results than what existed, i.e. improve on an existing product.

Uber's "mission" was to get privileged people with smart phones and disposable income a sexy/cool ride home from the bar.

Facebook was a social networking site targeted at college students...what was it's mission? To allow people to see what that cute person in their Psych class was up to this weekend?

Snapchat...the clear and important mission to allow for safely sending risqu pictures?

While I believe Sam's belief that they WANT to work with companies with clear important missions, that doesn't seem to be a/the deciding factor in becoming "very large", or attracting great people. If anything, it seems like the "spaghetti at the wall" approach is what happens - some startups gain momentum, and then it's the high growth trajectory that attracts a ton of talent.

Keverw 8 hours ago 2 replies      
> In future years, we also hope to explore how something like financial aid might work for people that need a small amount of capital to help get their startup going.

Sounds interesting. I always kinda hated how people fresh out of high school with zero credit can have so much money handed to them for going to school. While if they wanted to get a business loan or buy a house they would be much more scrutinized.

I wonder with that money instead and right mentorship if they could come out ahead of their peers who used that money for college instead.

I know some people who just go to college because they feel like they have to, and get degrees they don't even use. Imagine one spending 4 years going into debt, the other coming out a head with a real world profitable startup. One thing I always hate about school work is I feel like I'm just doing stuff for the sake of being graded or being busy. I rather spend my time creating stuff that's more real instead, learn as I go. There's a bunch of free and even paid training on-demand on the internet to fill in your skill gap.

AndrewKemendo 9 hours ago 6 replies      
Companies like Amazon, Facebook, Google, Apple, and Microsoft have powerful advantages that are still not fully understood by most founders and investors....This trend is unlikely to reverse without antitrust action, and I suggest people carefully consider its implications for startups.

I think the "startup community" needs to have a serious conversation about this [2]. If we agree that machine learning is going to be a major underlying technology moving forward and that as a result, data is the new Oil [1], then the top 5 technology companies serve as the new Oil Barons.

It goes further than that actually in my mind, as these firms are disrupting themselves and innovating as fast if not faster in many areas than small startups can. They are in every industry and if they can't move as fast, then they buy the talent or the whole team. Granted, a small team will always move faster, but I think that these mega corps know enough now to know that they can pivot on trends if needed.

I do worry about the likelihood of a startup that could beat google, facebook, microsoft etc... with humble roots. Maybe if a "startup" came in with 100M+ in funding, but more than likely at this point those companies would be funding that startup, if only for the intelligence value.

So while "we've seen this before - someone always comes out of nowhere" which gave rise to the current crop of major companies, it would have to be pretty left field at this point for a company to compete with the major players without investment from an arm of theirs. Considering Snapchat is the biggest IPO in the coming year, I don't have a lot of hope that there will be something coming through.


[2] https://medium.com/@andrewkemendo/the-ai-revolution-will-be-...

danvoell 8 hours ago 1 reply      
Hi sama, great letter. The only thing I would suggest for the next round is to identify some misses. What are some RFPs that you tried to solve and didn't and what are your insights on this? The two annual letters I like the most are Bill Gates and Warren Buffet. From Bill Gates, I don't mind the endless optimism since it is a foundation. From Warren Buffet, I expect the ole hoe hum, we'd be trillionaires if we had seen this trend or we were totally wrong about this ...
icpmacdo 9 hours ago 5 replies      
"Hacker News has 3.4 million users per month and 350,000 users per day, with 4 million pageviews a day. There are just under 1 million registered accounts, with several hundred added each day. Users post around 1,000 articles and 6,000 comments to the site per day."

First time I have seen those numbers, very impressive. I thought there were a lot less users.

ridruejo 9 hours ago 3 replies      
"Our mission is to enable the most innovation of any company in the world in order to make the future great for everyone." I know many people will read that and roll their eyes back and believe it is just a marketing gimmick for the gullible and like most investors YC just cares about the money. The thing is that they do deeply care about founders and making things better in the world, as naive as that may sound to outsiders.
nopassrecover 6 hours ago 1 reply      
We are only about 30 years into the age of software, about 20 years into the age of the internet, and about 2 years into the age of artificial intelligence

Which innovations over the last 2 years most indicate a new age of artificial intelligence?

I undertook some research, and noticed:

 3 Months Ago: Google Translate AI had a breakthrough 5 Months Ago: Amazon/Facebook/Google/IBM/Microsoft launch an AI Partnership 1 Year Ago: AlphaGo has a big win and OpenAI is founded 2 Years Ago: Google (now Waymo) performs its first public driverless ride 3 Years Ago: Google acquires DeepMind and interest in Deep Learning takes off 6 Years Ago: IBM Watson wins Jeopardy
All indications suggest increased interest and investment in AI, but from an external perspective I'm interested to know whether quantitative changes (big data, cloud infrastructure, GPU processing speed etc.) and increased interest have genuinely spurred a qualitative new era of AI free of the pattern of "AI Winters" (https://en.wikipedia.org/wiki/AI_winter).

bartmancuso 7 hours ago 1 reply      
Re YC Research, please consider a project looking at the challenges surrounding climate change communications. To state the obvious, the science on climate change is settled, but many people are still skeptical/resistant or in outright denial. Heated arguments and throwing around citations to journal articles really doesn't change anyone's mind. There is a deeper, nuanced communications challenge at play, and it's the exact type of thing that YC would be great at addressing. There are some very interesting ways that technology (and the creativity of the YC community) could be used to tackle this issue.

Climate change may be the defining issue of the next century, and it would be a shame to be stymied by the communications challenges. I hope you'll consider it.

blacksmythe 8 hours ago 2 replies      

 >> funded over ... 1,470 companies >> more than 50 of our companies are worth more than $100 million each.
3.4% is a remarkable ratio.

derrickc 8 hours ago 0 replies      
I'd love to know the 50 companies with >$100mm valuations. Anyone know of a list or can start one? I saw this was 32 companies last year and really curious which were added.
yawgmoth 8 hours ago 2 replies      
On the HN front - I would like to see a discussion and then the implementation of measures to encourage meaningful discussion, discourage low-effort comments, but also to discourage "negative lurking". Like other users noticed, there are a lot of lurkers -- some of which downvote and move along. Because frequently I do not understand why a post has been downvoted (my own or otherwise), I am discouraged from contributing. A byproduct of both negative lurking and the fact that HN has concluded itself to be an intelligent community is that some readers, including myself, feel judged by the score next to our comments. There is someone out there reasonably asking "why do you care?" but it is also important to acknowledge that this phenomenon exists. Part of the longevity of a user's participation is the willingness of the community to accept it.
notatoad 8 hours ago 2 replies      
>I expect that people will talk about software as one of our secret weapons.

can you give a broad overview of what software YC creates? What does it do? is it internal tracking and stats, or exclusive libraries that YC-funded companies get to use to build their products?

bdrool 5 hours ago 2 replies      
Aside from a throwaway line about a possible "macroeconomic meltdown", there seems to be little attention paid to big signs of an overall negative trend in tech jobs:

* "Bay Area: Tech job growth has rapidly decelerated" (from here: http://www.siliconvalley.com/2017/02/10/tech-job-growth-slow...)

* "Tech Jobs Took a Big Hit Last Year" (from here: http://fortune.com/2017/02/16/tech-jobs-layoffs-microsoft-in...)

The first of those two articles has been submitted twice to HN with zero comments both times. The second has not been submitted at all as far as I can see.

Is it possible there is a bit of a collective head-in-the-sand phenomenon going on?

krishna2 6 hours ago 0 replies      
@sama : Great write up. Regarding the admission process you had mentioned on Dalton Caldwell that he took "a process that used to be stressful and deeply imperfect and improved it by a huge amount."

Can you please elaborate on that?

partisan 4 hours ago 0 replies      
I appreciate the letter. The following caught my eye, because I was sure we had all learned that the ends do not justify the means. Surely the Zenefits debacle was a lesson in ethics and not one about avoiding getting caught doing the wrong thing.

> we have to work harder to find people doing a startup for the right reason: to bring an idea theyre obsessed with to life, and willing to do something unreasonable to see it happen

danenania 5 hours ago 0 replies      
Hey Sam,

Here's a question for you (or other YC folks) on YC recommendations.

Give that I'm planning to apply for the next cycle, would it benefit me to ask for recommendations from 7 or 8 folks who I've worked with in the past and have strong relationships with?

It's not totally clear to me whether it would be kosher to ask for those or if it's something you'd rather that people do spontaneously. I'm also wondering if 7 or 8 is too many given how little time I know you have to spend on each applicant. Thanks!

mbrock 9 hours ago 3 replies      
I think it's a mistake to conceive of Reddit as a place to "waste time."

It's also the new platform for lots of important online communities to discuss and disseminate information.

gregoire 9 hours ago 4 replies      
Is there a reason why this is published on Sam's blog and not on YC's blog?

Oh and a remark on the letter's typography: the double spaces after periods should be single spaces.

applecore 8 hours ago 0 replies      
Minor nitpick: n^2 isn't the correct factor to describe hyperscale networks. It's much slower, probably n x log n.
angersock 7 hours ago 2 replies      
Nice writeup sama, but the history is a bit revisionist:

> We are only about 30 years into the age of software, about 20 years into the age of the internet, and about 2 years into the age of artificial intelligence.

Software has been around--and actively making bank!--for over 40 years. The internet has been around longer in the form of academic institutions, email, and BBS than you mention.

And AI has, of course, been around at least as long as software.

I appreciate you have a message to send, but maybe don't ignore the path we took to get here.

nilved 5 hours ago 1 reply      
Did anyone else CTRL-F for Thiel and close the page?
jtmarmon 9 hours ago 2 replies      
"The percentage of women who apply to YC is roughly the same as the percentage of women who get funded. The same is true for Black and Latino founders."

I'm guessing what he means to say here is that if x% of the applicant pool is woman/black/latino, then x% of the accepted pool is woman/black/latino, right? The way it's phrased it sounds like every woman/black/latino who applies is accepted

rdtsc 6 hours ago 1 reply      
> the fact that one of their "partners" is such an obviously destructive force within society.

Can you expand on the "obvious" and "destructive" force? What exactly did Thiel do that is obviously destructive to our whole society? You mean Palantir? Wasn't that there for a while?

Inferring Your Mobile Phone Password via WiFi Signals fermatslibrary.com
339 points by pogba101  2 days ago   63 comments top 13
sounds 2 days ago 0 replies      
For those who want more information on CSI (Channel State Information):


This allows you to use a custom firmware developed for the Intel 5300 wireless adapter and read the CSI values with each packet.

Every 802.11n implementation that I am aware of keeps a CSI vector (IQ values, typically as integers) within the wifi chip. Both the Wifi AP and STA do this. The CSI vector is updated with every packet, using the training data at the beginning of the packet. (802.11 is CSMA [2] so there is a fixed transmission to start the packet)

In other words, Intel has this nice tool for one of their (now somewhat dated) chips. But CSI is not restricted to Intel chips. Atheros chips have a decent but limited CSI readback method, not quite as nice as Intel's [3]. But CSI has been used for experiments on all major wifi chips out there.

With 802.11n this is used to determine the quality of signal likely to be received on each sub-carrier within the signal.

CSI is useful for many other things: RF experiments, indoor position sensing, and now apparently also password cracking.

[2] https://en.wikipedia.org/wiki/Carrier_sense_multiple_access_...

[3] http://pdcc.ntu.edu.sg/wands/Atheros/

gefh 2 days ago 8 replies      
Holy shit.From a brief scan it looks like the paper concentrates on recovering a numeric pin, but these attacks never get worse, only better, so I assume full keyboard access is not too far off.What's the defense? Have your phone manage the passwords and unlock via fingerprint?
user659 2 days ago 0 replies      
This paper is available through Google scholar if you search for "CCS 16 password WiFi" or click here: https://www.a51.nl/sites/default/files/pdf/p1068-li.pdf

I've been a part of a similar paper that detected exact keystrokes. This one seems to build on a similar idea. The thing to keep in mind is that these systems need user and environment specific training. That is if the user is changed or the user or something in the environment moves, the system needs to retrain.

kardos 2 days ago 2 replies      
Direct link to PDF without the (infuriating) popups/overlays: http://delivery.acm.org/10.1145/2980000/2978397/p1068-li.pdf
andai 2 days ago 1 reply      
See also: detecting and motion tracking people behind walls, with the ability to recognise specific people ( also using wifi ).


Of particular interest: It can determine breathing patterns and heart rate.

program_whiz 2 days ago 3 replies      
Read the section "limitations". Only works on 10 users right now, must be trained for the pattern "per user", phone must be sitting on stable surface, gesture must be performed as close to "the same" every time. This is just clickbait and "please fund our research" IMO.
danielhooper 2 days ago 1 reply      
Some weeks back I read a post here about detecting people in rooms by measuring how the physical body interferes with the wifi signals. I wouldn't have imagined someone could extract useful information at this small of a scale. wow!
adynatos 2 days ago 0 replies      
LTE and HSDPA (and maybe older gens) have Channel Quality Indicator, which afaik has the same role as CSI. So I wonder if the same trick can be achieved with LTE signalling? To pull that off you would need access to a BTS, but today with open source stacks, like OpenBTS or OpenAirInterface,you could roll out your own.
saycheese 2 days ago 0 replies      
RELATED: "Keystroke Recognition Using WiFi Signals"


freyr 2 days ago 0 replies      
It looks like they're inferring the right 6-digit password about 20% of the time on their first try, presumably using the Xiaomi phone. But if they can try 20 candidates before getting locked out, they can guess the 6-digit password about 50% of the time.

With the Samsung phone, which has a much lower 1-digit recovery rate, it seems that it would be closer to 6% on the first try, and 20% by the twentieth try.

baby 2 days ago 1 reply      
I like this Fermat thing but it would be cooler if it could add a date to the papers who, for some reason, do not have a date.
leejoramo 2 days ago 0 replies      
Also never enter a password in any location where a hidden video camera could be observing you. Or where a hidden microphone could be listening to your typing. Or where ruffians holding crowbars could be lurking in the next room.
Amorymeltzer 2 days ago 2 replies      
Moral of this (and every other) story: Never, ever connect to a free, public wifi.

ETA: This was meant to be glib, given the frequency of such stories seen on HN, and the many children below are quite correctly pointing out that the real moral is https://news.ycombinator.com/item?id=13645694

Live slow, die old: Mounting evidence for caloric restriction in humans geroscience.com
276 points by discombobulate  2 days ago   213 comments top 24
ellyagg 2 days ago 4 replies      
There's a totally different way to live that also has mounting evidence, and it's a way that sounds a lot more satisfying to me.

For awhile now, the "obesity paradox" has been a thing, where segments of the population who are a little heavier than one would expect actually have the best all-cause mortality rates.

Recently there's been some pushback on this "paradox", but the one I want to call attention to is here:


The problem with the obesity paradox is that it's been based on the flawed BMI. In this study, they actually did DEXA scans of elderly women's body fat percentage, and those with the highest BMI and lowest body fat percentage had the best all-cause mortality rates.

This suggests plenty of calories plus strength training is in the running for a longevity lifestyle. This makes sense intuitively and if one is familiar with the panoply of beneficial physiological effects from exercise on the human body. And, it would not come as a shock to find that increased strength from greater muscle improves, e.g., balance and coordination to prevent accidents in the elderly, nor that increased muscle mass provides a protective tissue reserve for fighting disease without the concomitant downsides of adipose.

crudbug 2 days ago 3 replies      
2016 Nobel prize was awarded to Yoshinori [0] for his work on Autophagy [1] - Cells eating dead cells and auto-correcting body functions when there is lack of calories/energy in the system.

Ancient philosophy also says about fasting 2 days a week. The Autophagy process provides a scientific foundation for those claims.

[0] https://www.nobelprize.org/nobel_prizes/medicine/laureates/2...

[1] https://en.wikipedia.org/wiki/Autophagy

jaggederest 2 days ago 4 replies      
As a counterexample, the Minnesota Starvation Experiment found a substantial variety of negative effects from calorie restriction:

> Among the conclusions from the study was the confirmation that prolonged semi-starvation produces significant increases in depression, hysteria and hypochondriasis as measured using the Minnesota Multiphasic Personality Inventory. Indeed, most of the subjects experienced periods of severe emotional distress and depression.[1]:161 There were extreme reactions to the psychological effects during the experiment including self-mutilation (one subject amputated three fingers of his hand with an axe, though the subject was unsure if he had done so intentionally or accidentally).[6] Participants exhibited a preoccupation with food, both during the starvation period and the rehabilitation phase. Sexual interest was drastically reduced, and the volunteers showed signs of social withdrawal and isolation.[1]:123124 The participants reported a decline in concentration, comprehension and judgment capabilities, although the standardized tests administered showed no actual signs of diminished capacity. This ought not, however, to be taken as an indication that capacity to work, study and learn will not be affected by starvation or intensive dieting. There were marked declines in physiological processes indicative of decreases in each subject's basal metabolic rate (the energy required by the body in a state of rest), reflected in reduced body temperature, respiration and heart rate.


leovonl 2 days ago 1 reply      
There is mounting evidence that fasting and a few dietary approaches (like the ketogenic diet) are useful tools for dealing with a whole sort of issues, from autoimmune diseases and generally any excessive inflammatory responses (including CHD).

Calorie restriction OTOH is something that gives mixed results, and I suspect it may be related to the diet adopted and how it is implemented (ie, LCHF vs LFHC, fasting vs frequent small meals).

I suggest anyone interested in food/health to spend some time on google scholar searching about these approaches, it's enlightening.

prodmerc 2 days ago 5 replies      
Wait just a minute. What if I don't want to live longer? What if I want to live a shorter life full of energy?

Everyone is focusing on longevity when quality is probably more important. 120 years as a couch potato would be shit compared to 60 years of being active every day (whether physically or mentally).

People are against TRT because it shortens lifespan, while completely ignoring the quality of life improvements, which is just stupid.

Give me something to burn fast and bright, not slow and dim, thank you very much.

nacc 2 days ago 4 replies      
I have a wild theory about longevity. For all the mammals, increased heart rate correlates with lower life span [0]. But heart rate itself may just be a measurement, and what's important is rate of metabolism - higher metabolism, higher heart rate, lower life span. And vice versa.

If we very boldly assume these relationships are causal and predictive to individuals, it follows lower energy consumption -> longer life. Caloric restriction might be one way to do this.

However, with lower metabolism, stuff in the body tend to get older. This will make them fail more easily, then you are likely to get sick, which will shorten your life. Therefore the optimal strategy to maximum life expectancy seems to be control metabolism (by exercise / calory intake etc.) to a certain point where you are just unlikely to get sick, then stop.

[0] biology.stackexchange.com/questions/20489/is-there-any-relationship-between-heartbeat-rate-and-life-span-of-an-animalH

coldcode 2 days ago 19 replies      
Interesting but is it worth living being hungry all the time? Are the extra years (you might not get anyway due to illness or accident) on average worth it to you the individual?
reasonattlm 2 days ago 0 replies      
Of note, Geroscience is a new popular science of aging online magazine supported by the Apollo Ventures investment fund, devoted to longevity science startups. You should absolutely take a look at their site to see the sort of sea change that has occurred for the perception of aging research in the funding community:


They take a very Hallmarks of Aging view of the causes of aging, which I can quibble with around the edges, but it is good to seem more people putting their money into the game of building ways to treat the causes of aging.

The principals at Apollo became involved in this space and raised a fund both because they are enthused by the field of therapeutics to treat aging and want to see it succeed, but also because they recognize the tremendous potential for profit here. The size of the market for enhancement biotechnologies such as rejuvenation treatments is half the human race, every adult individual.

Publishing a magazine on aging research is a way to help broaden their reach within the community, find more prospective investments, talk up their positions, and raise the profile of the field as a whole, all of which aligns fairly well with the broader goals of advocacy for longevity science. Many hands make light work, and we could certainly use more help to speed up the growth of this field of research and development.

anonu 2 days ago 2 replies      
You might live longer on a calorie restricted diet but you might just die of boredom. Seriously, food is amazing. Why would you deny yourself so much of it.

I'd be a bigger advocate of intermittent fasting. This has also been shown to improve health and longevity.

memoryfab_com 2 days ago 0 replies      
I actually believe starvation and caloric deficiency has more correlation with inflammation. When you introduce foreign sustenance (food) into your body, it has to spend resources to work through it, absorb it and expel waste. The higher the caloric count the more work it has to do and usually more inflammation from working through it instead of working on recovery.

What studies (HMS,Johnhopkins) have shown is exercise (blood flow vital to removing waste and in result inflammation) and sleep (recovery) is equally as if not more important than diet.

These focus points are the same in fighting age related diseases: Alzheimers and sleeplessness.

fast_throwaway 2 days ago 0 replies      
Dr. Jason Fung, a Canadian Nephrologist is an encyclopedic resource for the impacts of calorie restriction and fasting. While he has a couple (really good) books, the following presentation is a nice overview of his work, findings and results of therapeutic fasting:

'Therapeutic Fasting - Solving the Two-Compartment Problem'https://www.youtube.com/watch?v=tIuj-oMN-Fk

wallace_f 2 days ago 1 reply      
Given the inconsistent, even capricious prescriptions coming from nutritional science and medicine, here's three reasons I don't worry too much about nutritional "science."

First, n anecdote:

1.) One year I decided to run a marathon, and joined a running group including some elite runners.

One was a former state champion. In that particular year he'd run an Iron Man, a marathon, and the Goofy Challenge. That is a big deal.

He was carved out of stone. At running clubs, women would surround him. They'd paw at him, giggle as they pointed and touched this man body of steel.

After witnessing this for months, once at a pub I saw him order, IIRC: 2 hotdogs, a hamburger meal (with its own sides), and sweet potato fries, which he washed down with several beers.

I was shocked. I asked if he always ate that way. He said he just eats what he feels like eating, with one exception: added sugar.

So that's what I do. I eat what I feel like, with (3) exceptions. Small portions, no added sugar, and limits on artificial flavours. I've seen great results in energy, mind and physique. It appears my body is actually really well tuned to tell me what it needs without me having to do anything at all--as long as I follow those three rules.

2.) We need to be careful what we call actual science. If you can't rigidly follow the scientific method and test your theory to see if it matches with nature, it's not science.

3.) Nutrition is ridiculously important to our well being. It's more important than sex, shelter, and ego. Our bodies have been evolving for millions of years to learn how to tell us what they require. It's arrogant to declare yourself to know better here without having hard scientific evidence of it. That's what we're seeing now in the "scientific" literature. Butter is no longer a heart risk. Eggs are now healthy. All of the sudden sugar is the devil. Etc.

thomk 2 days ago 0 replies      
Eat To Live, by Dr. Joel Fuhrman describes a diet that is low calorie high nutrient and it's a very good, easy read for anyone who is interested. I personally lost 70+ pounds on it: https://www.wikiwand.com/en/Eat_to_Live

Health = Nutrients/Calories!

sebringj 2 days ago 0 replies      
See about Roy Walford http://www.walford.com/ for the origins of this. He was able to experiment with humans in the biosphere team.
stefap2 2 days ago 2 replies      

He existed and even thrived on a diet of subrancid cheese and milk in every form, coarse and hard bread and small drink, generally sour whey, as William Harvey wrote. "On this sorry fare, but living in his home, free from care, did this poor man attain to such length of days".

Thomas Howard brought him to London to meet King Charles I.

Parr was treated as a spectacle in London, but the change in food and environment apparently led to his death.

reasonattlm 2 days ago 0 replies      
Some of the more interesting recent research, from the perspective of evidence for meaningful health benefits, and some degree of additional longevity in our long-lived species.

Caloric restriction improves health and survival of rhesus monkeyshttps://dx.doi.org/10.1038/ncomms14063

Will calorie restriction work in humans?http://dx.doi.org/10.18632/aging.100581

ziotom78 1 day ago 0 replies      
Interestingly, I read about this idea in Corps et mes (1943), a marvellous novel by Maxence Van Der Meersch (a physician). The wife of the main character is able to recover from tuberculosis thanks to a diet with very low calories.
bikamonki 1 day ago 0 replies      
Meh, I'll just wait for the iHeart, the iKidney and the iLiver. Surly hope I do not end up with an Android-powered brain b/c my insurance plan does not cover the always reliable and reasonably priced AWS Cognito Plus, while I'm definitely not rich enough to dream about the iBrain 7. Until then, the triple-b diet will do: beer, burgers and bike to work.
grondilu 2 days ago 0 replies      
Yet as far as know longevity records among humans did not have any particular diet, did they?

As caloric restriction is being more and more shown to be the only efficient method to increase longevity, I suppose more and more people will try it and soon enough we'll get to break records. Time will tell.

wcummings 2 days ago 1 reply      
A low-calorie plant-based diet

>much of it from plant-based material like the Japanese sweet potato, their staple food, in contrast with the rice-heavy cuisine of the mainland.

hartator 2 days ago 0 replies      
I wonder what's the best way to apply this to daily life. Restriction at every meal vs alternative day restrictiona vs fasting a few days every once in a while.
Hambonetasty 2 days ago 2 replies      
Why the fuck would you want to do that? Being old sucks.
beegeezuz 1 day ago 0 replies      
Live fast, die young. Quality over quantity. I'm bout dis life bitch.
Video calls for Signal now in public beta whispersystems.org
286 points by okneil  2 days ago   81 comments top 11
mirimir 2 days ago 3 replies      
Well, but ...

> But anyone testing the beta who links their iPhone to iCloud and wants the same level of privacy Signal has always offered should consider an extra step, too: Disabling a setting that uploads a calls metadata to Apple. The beta upgrade to Signal will use CallKit, Apples framework for allowing VoIP calls like Signals, to be integrated more completely into the calling functionality of the phone. But that also means calls will be recorded in the iPhones call log and, for iCloud users, shared with Apples server. iOS treats CallKit calls like any other call, however that also means some information will be synced to iCloud if enabled, Open Whisper Systems warns. This information includes who you called and how long you talked.


weinzierl 2 days ago 6 replies      
This is great news. I'm on iOS but I hope it will come to Android soon.

I have a technical question:

> We immediately realized that protocols like SIP, which traditionally required holding open long-lived connnections in order to receive incoming calls, were not going to be compatible with the mobile environment.

Ok, so far so good.

> Instead we built our own simple REST-based signaling protocol [...], and used push notifications instead of long-lived connections to notify the client of incoming calls.

So, no long lived connection but a "simple REST-based signaling protocol". How is that supposed to work without a long lived connection?

> Actual push notifications hadn't been invented yet, though, so we created our own push infrastructure by sending encoded SMS messages that the app would silently intercept and interpret instead.

OK, that's pretty clear again.

> Over time, we switched to push notifications when they were created by Google and Apple [...]

But don't push notifications basically work over a long lived connection? Of course it's better to have just one long-lived connection to Apple instead of one for every communication App, but in the end if you want real time signalling in a mobile environment you won't get around a long lived connection, don't you? At least that is my understanding, but I'm always happy to learn something new.

izacus 1 day ago 8 replies      
I really don't get the direction of the Signal project. iMessage got a huge following because it allows you to trabnsparently send both SMS and encrypted messages (significantly lowering barrier to entry) and brings that functionality even to the desktop. Hence using it means that average user doesn't have to explicitly think about which channel to use and with spread of use more and more communications are secure.

On the other hand, Signal SMS support is broken (datastore and MMS issues), they don't want to bring SMS support to their "desktop" app (which STILL needs you to install Chrome to work) and they still don't support the use of multiple devices. Instead they're wasting resources implementing video chat which noone really asked for and won't help the adoption nearly as much as having a secure drop-in replacement for SMS client. Even worse, enabling SMS support will prevent any other SMS apps that let you have conversations via the computer from working.

It seems like they're actively trying to shoot their own foot.

The best Signal can do is to make a proper desktop application (even Electron would do now, even though Telegram's approach is significantly better UX wise) and make SMS seamlessly integrated into it both on the phone and on the desktop. Video chatting is nice, but it's not where the most important requirement for cross-platform private communication is.

subliminalpanda 2 days ago 2 replies      
Tried it today with a relative, the audio quality is significantly better, almost no stuttering.

Video was very smooth as well.

flyx86 1 day ago 1 reply      
So I still cannot send a voice message in the chat (like Android users can), but I am now able to do video calls. Shouldn't a feature be fully completed before moving on to the next one?

And it still frequently happens that I get the same text message from Android users six times. Where are the priorities of this project?

bubblethink 1 day ago 1 reply      
Does this change mean that the server is open source too ? I remember that server for voice (redphone) wasn't open earlier.
lmedinas 1 day ago 0 replies      
Is there plans to bring Video calls to Desktop also ?

I will be interested in doing Video calls in a bigger screen, like iPad for example, is there any plans to expand this ? or is there any technical limitation like the phone number ?

newman314 1 day ago 0 replies      
Has anyone tried Signal with CallKit enabled and iCloud disabled as well as seeing if it leaks metadata in this form?

This config works for me with other VoIP apps. I tried earlier but was not able to get a call through...

jrv 2 days ago 3 replies      
Great. Signal doesn't do group calls though, right? Is there any app that does end-to-end-encrypted group calls?
exodos 1 day ago 1 reply      
Signal is a pretty polished app, more than expected. However I'm annoyed that they're adding large features without fixing small things.

Sometimes I'll MMS from other people inside other chat windows where they don't belong.

It's also really annoying that I have to attach a an item BEFORE I enter the message. Gets old after while.

Apps crashes quite frequently.

Using T-Mobile Wifi Calling feature makes MMS inconsistent, though at least it allows me to use.

Signal should add support for Windows 10 Mobile as an app for all platforms, and SMS/MMS should carry over, not just Signal-to-Signal.

I know there's a few other bugs, but I can't think of them at this moment.

BTW, I use the LG G4 unrooted, custom recovery, and unlocked bootloader

ripdog 1 day ago 1 reply      
Eh, doesn't work for me. I turned the beta on for two android phones, then called one from the other. No video options, audio quality unchanged.
Realtime Analysis of the Oroville Dam Disaster github.com
305 points by rodionos  1 day ago   166 comments top 22
kolbe 1 day ago 9 replies      
I like what you're trying to accomplish, but you are missing a very helpful piece of information in your analysis. I'm sure you noticed how poor of a predictor rain fall is in your assessment of how much water will be added to Lake Oroville. This is largely because Oroville is filled from Sierra Nevada runoff. If a particular storm system is warm (as was the case with the latest one), it will mostly rain in the Sierras. This has a twofold effect: one, the precipitation doesn't stay put at higher elevations (as snow does), and two, it melts the existing snow, causing it to also be added to the downstream water accumulation. So a key part of forecasting is not only to look at expected precipitation, but also the expected snow levels.
stuckagain 1 day ago 3 replies      
Wow almost every statement here is wrong.

#1: the dam is managed in the winter about 1 million acre feet below the top, for flood control purposes. The spillway has been operating all winter long.

#2: The operator cannot decide to use the emergency spillway. Water just goes over it when it approaches the top of the dam.

#3: no water has gone over the dam. That would destroy it.

mturmon 1 day ago 1 reply      
Kind of a fun study, enabled by the presence of so much public-domain data from CA DWR.

One piece that's missing from this analysis is upstream precipitation. You are sensing precip at Oroville itself, but water in the dam comes from everywhere upstream (3600 square miles according to [1]). As well as (potentially) snowmelt.

People doing forecasting of reservoir levels will be integrating distributed precipitation information, snowpack, and temperature, with a soil runoff and routing model to get predictions.

[1]: http://cdec.water.ca.gov/cgi-progs/profile?s=ORO&type=dam

harlanlewis 1 day ago 0 replies      
This is terrific. Would be a great resource for journalists to identify and understand some of the key points that can then be shared in a more public-friendly format. I wish all stories had this kind of accessible data behind the news that could be explored!

Here's a great article on climate change's effects on our water infrastructure in light of the Oroville Dam's situation: https://www.nytimes.com/2017/02/14/opinion/what-californias-...

An example - the bulk of California's water storage is in the natural reservoir of mountain snowpack. The large volume of precipitation this year, much of it as high-altitude rain falling on snow, has caused an extraordinary amount of snowmelt in a short period of time. The natural reservoir of snowpack is rapidly released into our manmade resorvoirs and strains or exceeds their capacity. This cycle of alternating extremes, drought to wet, is expected to continue, and our water infrastructure's capabilities must be planned in light of that.

seanp2k2 1 day ago 3 replies      
mxfh 1 day ago 2 replies      
Would it be feasible to dig out another minor temporary overspill or diversion out of the basin into another one somewhere far away from the dam, and more importantly the emergency spillway?

Dry Creek basin seems to have some points were it's just a few meters of a hump over the emergency spillway level.




caseyf7 1 day ago 3 replies      
This is a great analysis.

However, this seems a little off: "Based off of our estimate, for every inch of rainfall at the Oroville dam, 136,790.5 acre-feet will be added to the reservoir." The relationship probably isn't linear. Much less of the first inch of rain makes it to the reservoir vs. the 5th inch of rain.

Hopefully, the next iteration will project how many inches of rain it will take this weekend to top the spillway again.

clamprecht 1 day ago 1 reply      
Several pilots have published videos of overflying the Oroville Dam. Here's one from a guy who makes some good videos:


The money shot is at 2:25

astrodust 1 day ago 1 reply      
As always, the quirky units Americans insist on using never cease to amaze.

Acre-feet. Of course.

The only thing missing here is nautical miles and football fields.

NDizzle 1 day ago 0 replies      
Thanks for putting this together. Keep me in your thoughts, guys. There's a lot of misinformation about what's going on up there. I'm in Yuba City, 2 miles away from the feather and 30 ft up.

They reduced the mandatory evacuation order but more and more emergency personnel are arriving. There are (unfounded at the moment) local reports of water seeping underneath the emergency spillway weir.

ransom1538 1 day ago 1 reply      
Oroville dam feeds into the Feather river, the Folsom dam feeds into the American river. Here is the kicker: both the American and Feather feed the Sacramento river - a major bottle neck.

Folsom dam has turned on the spill at insane rates the past few days. I wish people living in Sacramento county (my parents) would take things more seriously.


Symbiote 1 day ago 0 replies      
I realise this is done to show off the ATSD tool, but it would also make a really good example using a Jupyter Notebook. GitHub will process and render those itself.

Example: https://github.com/benlaken/Comment_BadruddinAslam2014/blob/...

anigbrowl 1 day ago 0 replies      
It's great that you're building on this dataset but I'm not sure you're well-placed to pronounce on it being a disaster. Database expertise doesn't make you an expert on civil engineering.
alextheparrot 1 day ago 0 replies      
I really enjoy the graphs and analysis produced by this project, but does it irk anyone else that the Y-axis does not start at 0 in all the graphs? There are some graphs further down the page that start and 0 and they seem to show the data so much better (Water is increasing, but not 50x or whatever).
chris_7 1 day ago 1 reply      
What ever happened to the Mosul dam? Stories from about a year ago made it sound like collapse was imminent, but it hasn't happened yet. Did they do repairs, or are they just unsure when it will happen?
Fiahil 1 day ago 1 reply      
It looks like someone had spend quite some time doing this on working hours. It is to showcase something ("where is the ad?"), or someone asked and paid for this analysis?
rismay 1 day ago 0 replies      
This is a very educated estimate. Let's hope it rains less than 2 inches per day.
dkarapetyan 1 day ago 1 reply      
Will a lot of infrastructure survive the new weather patterns?
pm24601 1 day ago 0 replies      
"Disaster" ? not yet (and I hope not ever)
AnimalMuppet 1 day ago 1 reply      
Please tell me Betteridge's Law of Headlines does not apply here...
mathgenius 1 day ago 0 replies      
What is the point of all this data and analytics if the author does not even speculate on an answer to the question that is being asked?
erikpukinskis 1 day ago 1 reply      
I never considered it, but maybe climate change will help clear some of our waterways of harmful dams.
SNES preservation project on hold due to missing $10,000 package byuu.org
358 points by coldpie  2 days ago   142 comments top 23
mkarr 2 days ago 5 replies      
As a hopeful note of optimism: I recently sent a large, expensive package (worth around $1500) cross-country via USPS. The package was sent out mid-December. According to the tracking it made it all the way to the destination Post Office. After that...nothing. It just never progressed. I filed both a mail search requests, and an insurance claim. I received nothing but radio silence on those as well. Fast forward to about a week ago when the package just shows up on my doorstep without notice. Hopefully byuu's package eventually does show up.
scott_karana 2 days ago 4 replies      
> I've already maxed out my 401K loans on the Japanese game set, and it will probably be 2-3 years before I finish dumping and scanning all of those games to sell them for money.

I always feel a sinking feeling in the pit of my stomach when I read things like this.

Is any labour of love worth your potential future? As much as I admire his dedication, it's not a choice I'd make... :(

Maybe it's just because I'm a bit of an ops guy, and it feels like leveraging a single point of failure. :P

bane 2 days ago 2 replies      
I'm very sorry to hear about this. Shipping stuff internationally can be hair raising when it goes wrong. I used to regularly ship about $500k worth of computer equipment all over the place and it one time got caught up in German customs. A colleague of mine in Germany had to drive from Munich to Frankfurt and back many times over the course of several weeks to get it sorted out. IIR the problem was some minor form had been filled out incorrectly (by me). Huge mess. Hopefully this collection will turn up soon.

One thing

>Although I can't afford to host this content (it weighs in at hundreds of gigabytes)

I really urge you to get in contact with Jason Scott about getting a copy of your archive to the Internet Archive. These scans and dumps could be priceless and it makes sense to have as many backups of it as you can.

At any rate, thank you very much for all your hard work, time and expense in doing what it is you do byuu.

chrishacken 2 days ago 1 reply      
I'm a developer at USPS; if you send me the tracking number I can try to pass this along to someone.
froh42 2 days ago 1 reply      
I was just checking DHL web site over here in Germany for the cost but found something that may be interesting:

Aufgrund der Streiks bei Lufthansa im Dezember 2016 wurden fr Sendungen in die USA Ableitungen ber alternative Transportrouten notwendig. Unter anderem ber den Seeweg. Das fhrt aktuell noch zu strkeren Verzgerungen bei der Zustellung von Economy Paketen in die USA.


Due to a strike at Lufthansa in december 2016 packages with destination USA need to be redirected using alternate transport routes, amongst other by ship.This results in even bigger delays delivering economy packets into USA.


If you have the DHL tracking number you should try tracking.

There's also the DHL facebook page where you can start a facebook chat, with good luck your supporter will speak english. (Just try!) https://www.facebook.com/DHLPaket/app/1609168226005546/

From what it looks there's a high chance your package is on a ship somewhere. And even more importantly, the sender should create a "Nachforschungsauftrag" with DHL (ha, one of our bureaucratic german words I can't even find a translation for - it's a request for inquiry of a lost package). Even with domestic packages over here the tracking information sometimes is plain wrong, and I expect it even more so to be wrong if they don't use their usual ways of transport. (No tracking of all those packages in a shipping container on a freighter.)

Good luck. (Btw, once I have given up on a package from the US to Germany after 6 months. The day after I gave up and purchased an alternative product my packet arrived ... UGH).

userbinator 2 days ago 3 replies      
Why didn't you send the dumping hardware to the PAL donor?

This makes me think of another idea to avoid shipping the cartridges themselves around: an interface that allows eventually accessing them over TCP/IP. It could be exposed as a block device or a custom protocol, it doesn't matter as long as you can essentially send an address and get some bytes back, allowing to read the entire address space.

gargravarr 1 day ago 0 replies      
Last year, I tried to buy a very-difficult-to-find bluetooth adapter for my car stereo. I found one on eBay in the US, won the auction for slightly more than I wanted and then paid to have it shipped to the UK. After a week, the tracking information stopped being updated, right after USPS took over the package. After a month, I tried to get in contact with the couriers. This was difficult, because the package was shipped with the eBay Global Shipping Program, which does a fair job of obfuscating which carrier is actually taking it. USPS didn't want to know, the tracking number wasn't valid as far as they were concerned. Eventually I contacted eBay asking where the package was. Overnight, I was refunded, saying the package had obiviously been lost.

I did my best to contact them and ask for the package to be found, but nobody wanted to know. I eventually spoke to a woman in eBay's customer services, but nothing ever came of it. In my situation, at least I got my money back, and I know it's not quite the same as this poor guy's, but I can understand the frustration of dealing with tracked shipments. Especially for rare items that are difficult or impossible to replace - all you want is the package itself. Shipment tracking is supposed to prevent this happening - after all, if the package gets lost, it's likely in the same place as the last scan! You'd think the money the couriers have invested in the tracking system would incentivise them to use it to find lost packages.

tudorw 2 days ago 0 replies      
Good luck, I appreciate your archiving efforts.
filmgirlcw 2 days ago 1 reply      
Would physically visiting the facility in Jersey City help at all? I don't know how far byuu lives from there, but perhaps he could send a surrogate in his place to try to get information face to face. That might be grasping at straws (theoretically they should have location information available over the phone), but it might be worth a thought.
tiglionabbit 2 days ago 0 replies      
I love the Super Nintendo. So it's good to see someone is aiming to preserve it perfectly. I thought existing ROM dumps were good enough, but seeing as he already has this "icarus" program to read them, there's something more he needs for perfection?

Since he's handling the cartridges personally, it's a bit of a shame he has to give them all back. Wouldn't it be kinda neat to make an uber-cartridge that contains all chip configurations from every Super Nintendo cartridge in existence, so it could play any game on the hardware that game expects?

sethx 1 day ago 1 reply      
DHL from the US to Germany or Eurpope is absolutely rubbish.The problem is that you most likely used DHL Global Mail, which is not the same as DHL Express.DHL Global Mail delegates the shipment to the cheapest carrier possible, which in this case is USPS.Once they do the handover, you effectively lose all tracking ability.

My story with them:http://webtrack.dhlglobalmail.com/?id=27838&trackingnumber=G...

That's more than one full month for a package that contained nothing more than a teeshirt.On top of that, the package never arrived at the final destination, and might be stuck in customs, for all i know. I never got a notification, and will actually contact DHL soon myself, because their service is absolutely laughable.

As a logistics company, they had one job. And they failed.

rewrew 2 days ago 2 replies      
I'm confused -- has it been verified that this package cleared customs? Because if not that could be the delay here.
Neliquat 2 days ago 0 replies      
Had some shoes ordered that got hung up in the NJ hole for nearly a month. The company had already started a claim by the time they got here. Also DHL to USPS, which the shipper said was the only practical option from their region (poland). I can relate. Hoping his shows up late like mine, but googling the issue did not leave me with much hope. A lesson to insure your valuable shipments at the least.
prirun 1 day ago 0 replies      
I sent a laptop from IN to TN, with tracking (back in the 90's) and insurance. It never got there. After several weeks, I filed a claim with the USPS, and they weren't going to honor it. Their records showed that I turned it in to the local post office, then nothing - no further tracking whatsoever. I had to threaten to escalate things before they would pay, and then it took forever to get the money.

They said in the future, if you want to be sure of a delivery, you have to send it registered mail: with that, every time a transfer of any kind occurs, it is done with paperwork, signatures, tracking, and under lock and key. Of course it costs an arm and a leg too.

anonymous_iam 2 days ago 2 replies      
I suppose that you have now learned to not trust the United States Postal Service with your $10,000 package. FedEx or UPS do cost a little more, but in this case I think you would agree that it would have been worth it.
Larrikin 2 days ago 6 replies      
>First, because I don't speak any German, and feel very uncomfortable traveling to a foreign country on my own. I know most Germans speak English, but that doesn't ease my concerns all that much. It's scary to me to travel alone, sorry.

Found this odd. What is scary about traveling to a presumably a big city in one of the richest countries in the world?

ryanlol 2 days ago 7 replies      
>Obviously, it should have been insured for the full value. But please understand that hindsight is 20/20. The sender chose the insurance amount, and had no reason to suspect the package would be lost, and that we would be given no help on the matter. It's not unreasonable to believe a mail carrier would try to avoid the need to pay out 1000 Euros worth of insurance by not losing said package.

That's some really high expectations there.

If you want to send something to another country and aren't willing to lose it? Buy a plane ticket.

It's essentially impossible for the carriers to prevent this from happening given their volumes.

rayrod2030 1 day ago 0 replies      
I live in Jersey City and once had a $4200 apartment deposit check stuck in the USPS system for about two weeks before it finally turned up at its final destination. It was mailed from Jersey City to a Hoboken address so literally should have taken two days tops. Hopefully this is a similar situation and this package shows up soon.
jasonkostempski 2 days ago 0 replies      
Maybe it blew over to the neighbors front law, I just found my $130 package over there.
BrianT 1 day ago 0 replies      
What about contacting the postal inspectors?


They maybe able to open a case for you

2 days ago 2 days ago 1 reply      
rasz_pl 2 days ago 0 replies      
>The package was insured for 1000 Euros.

end of story.

phonon 2 days ago 2 replies      
Why not sue DHL?(USPS has sovereign immunity.)
       cached 17 February 2017 03:11:02 GMT