hacker news with inline top comments    .. more ..    27 Aug 2016 Best
home   ask   best   3 years ago   
Planet Found in Habitable Zone Around Nearest Star eso.org
1176 points by Thorondor  2 ago   422 comments top 48
mjhoy 2 ago 9 replies      
The fun stuff is buried in footnote [4]:

> The actual suitability of this kind of planet to support water and Earth-like life is a matter of intense but mostly theoretical debate. Major concerns that count against the presence of life are related to the closeness of the star. For example gravitational forces probably lock the same side of the planet in perpetual daylight, while the other side is in perpetual night. The planet's atmosphere might also slowly be evaporating or have more complex chemistry than Earths due to stronger ultraviolet and X-ray radiation, especially during the first billion years of the stars life. However, none of the arguments has been proven conclusively and they are unlikely to be settled without direct observational evidence and characterisation of the planets atmosphere. Similar factors apply to the planets recently found around TRAPPIST-1.

taliesinb 2 ago 2 replies      
Wow, amazing result. And talk about synchronicity - just last night I watched an interesting 2015 talk about the search for planets around Alpha Centauri using the radial velocity technique: https://www.youtube.com/watch?v=eieBXGpNYyE

The speaker even mentioned the previous incorrect HARPS announcement, which was later found to be an artefact due to the windowing function they used - a pretty embarrassing mistake. This new finding involves a completely different period: 11.2 days instead of the previous 3.24 day signal.

Also, link to the Nature paper for the lazy: http://www.eso.org/public/archives/releases/sciencepapers/es...

kjell 2 ago 2 replies      
Just in time for the third book in Liu Cixin's space opera ("Remembrance of Earth's Past") to be released in english next month: https://www.goodreads.com/book/show/25451264-death-s-end

Previously on HN: https://duckduckgo.com/?q=site%3Anews.ycombinator.com+cixin+...https://hn.algolia.com/?query=Cixin%20Liu&sort=byPopularity&...

afreak 2 ago 19 replies      
Keep in mind that at best it would take maybe 1,000 years with current technology to get there with a probe or human-supporting ship. It would be highly unpopular however as it involves exploding nuclear bombs behind the craft to get it there that fast--that and it would probably cost trillions to build the thing.
bikamonki 2 ago 1 reply      
Since we're all rolling out our best fiction here, here's mine:

We'll get there animating matter by means of beaming laser instructions onto it. We just need to discover how we can move atoms by simply shinning a laser onto them, a controlled pulse of different light frequencies that allows us to arrange atoms in such way that they become tiny building blocks of nano-machines, like making pizza dough: twisting, throwing, rolling, until we have the right shape. Once the first Lego pieces are ready, we use the same laser to instill the energy required to move them. These animated nano-machines will then auto-assemble and become a bigger machine until we effectively, and remotely, build and operate a full-featured robot. Said bot will send back everything we need: images, audio, chemical reads, etc. Furthermore, our bot can build more bots and eventually build the laser that can be beamed into the next planet to repeat the process and expand colonization.

api 2 ago 7 replies      
How big of a space telescope would we need to see this planet in any actual detail?

One of my sci-fi fantasies is to take a photo of an extrasolar planet and see someone else's city lights. :) Of course if we could see that we could also probably detect their radio emissions, but seeing someone else's lights would somehow be cooler.

thedangler 2 ago 1 reply      
So I guess we start sending light patters to that planet and wait 8 years for a response?
ngoldbaum 2 ago 0 replies      
And here's the paper describing the discovery: http://www.eso.org/public/archives/releases/sciencepapers/es...
oli5679 2 ago 0 replies      
Its hard to draw definitive conclusions when you're speculating from a sample of one! Imagine showing a child with no knowledge of animals a snake and asking her to describe what she thinks the other animals on earth are like and the habitats they occupy. I think there'd be a risk that she'd describe a range of snakes and possibly lizards, but wouldn't be able to imagine something radically different like a whale/eagle. That's the risk we run when our only sample is the earth.
partycoder 2 ago 0 replies      
It's not only temperature, presence of water and distance to star. It's also a large variety of factors.

For instance... what is the atmospheric pressure? boiling point of water is affected by atmospheric pressure. Even if temperature is low, if atmospheric pressure is also low, water would boil at a lower temperature. In Mars for instance, water boils all the time.

Some people might say you can probably create more atmospheric pressure by terraforming the atmosphere. But not all planets can retain an atmosphere. Solar activity, planet magnetic field and gravity can affect that.

Then, gamma ray exposure. Radiation can sterilize a planet. It would be good to measure what is it like there.

owenversteeg 2 ago 1 reply      
Although we can't image it with current technology (JWST and Hubble both have resolution of 100 milliarcseconds) we might be able to within a few years.

IR inferometers will be able to give us some data in just a few years, and the E-ELT/TMT will also let us "image" it. The "image" won't be anything you can really look at (E-ELT has resolution one milliarcsecond) but it'll give us important data.

Diederich 2 ago 3 replies      
Does anyone here have an idea of which kind of resolution https://en.wikipedia.org/wiki/James_Webb_Space_Telescope will provide? I'm assuming that this little rock might not even occupy a single pixel, but I'd love to be wrong.
lutusp 2 ago 0 replies      
This discovery will greatly increase interest in gigantic telescopes, to allow a closer look at the planet and its atmosphere.
bcjordan 2 ago 2 replies      
Would it remain in a habitable state longer than Earth?
shmerl 2 ago 1 reply      
Is it feasible to send deep space probes to such planet? Let's say the probe is accelerated to high sub light speeds with ion thrusters. Can it reach it in some sensible time then?
hoodoof 2 ago 2 replies      
Is any article ever published on an exoplanet in without speculating that it might harbor life?
natch 2 ago 1 reply      
What's super confusing to me is: If the planet is so much closer to its star, and the star is so much larger than ours, why does the artist's conception show the star as being so "small" (perceived size, not actual size) as viewed from the planet? Was the artist just not thinking straight that day, or am I missing something? Yes I understand it's an "artist's conception" but the question remains.
Symmetry 1 ago 1 reply      
I wouldn't hold out much hope for it being particularly habitable. Without the early development of life you don't have a high oxygen atmosphere for most of a planet's history. Without oxygen no ozone. Without ozone UV light breaks up water molecules high in the atmosphere and the planet loses hydrogen on the solar wind. And then you end up like Mars.

In terms of planets to establish a colony on I'd actually look for ones a bit outside the traditional habitable zones. You'd need a bit more in terms of solar panels and heating but lacking hydrogen is a big handicap.

DrNuke 2 ago 0 replies      
For the foreseeable future, terraforming of Mars is much more attainable than any going remote.
sakopov 2 ago 1 reply      
Posted this story when it came out a week ago but it got no traction. [1] This is quite exciting but as far as I understand we are not quite there in terms of technology to reach it within my lifetime.

[1] https://news.ycombinator.com/item?id=12302489

misiti3780 2 ago 0 replies      
Nick Lane's book/hypothesis really change the way I think about life on other planets. His hypothesis is basically that the chances of Eukaryotic cells emerging from bacteria (via natural selection) are so rare (it only happened once in two billion years on our planet) that we really shouldnt expect to find intelligent life on other planets - rather the life we will most likely find will be small cells like bacteria and archaea, that lack a nucleus (and never get very big). He does a much better job of explaining why, but it is interesting nonetheless.

[1] https://www.theguardian.com/books/2015/apr/22/the-vital-ques...

sampo 2 ago 0 replies      
Here is slides and video for a talk "Adaptive Optics Imaging of Extosolar Planets" from 2015.

Especially the review of history of the study of exoplanets is amusing. When only our solar system was known, everyone believed in the theory of "inner rocky planet region, outer gas giant planet region". When astronomers finally started to have instruments to actually detect planets in other solar systems in the mid-1990s, almost none of the detected exoplanets fit the theory (slides 5-8).


jomamaxx 2 ago 0 replies      
"Major concerns that count against the presence of life are related to the closeness of the star."

I think the 'major concerns' are that we don't exactly know what 'life' is, and that since we have no information about any other 'biological entities' such as ourselves anywhere else, we can't entirely assume that it's a common thing.

I suggest that if we find life out there, it will be very common. But it's not entirely plausible that this is the case.

It's an interesting statistical game, made very difficult by the fact we don't fully grasp how 'we' became in the first place. I mean, we have the gist of it, but there's so much that remains unknown.

markingram 2 ago 2 replies      
Let's say the chance of Aliens visiting Earth is 1 in 1 vigintillion, then the chance of the Aliens being at similar levels of technological advancement as humans is 1 in 1 centillion...

In other words, they are so advanced that they can visit us without us knowing, unless they wanted us to know. They can wipe us out without us knowing, unless they wanted us to know.

I am so glad all these are still theoretical.

daveheq 2 ago 0 replies      
I read this days ago, in fact I heard about it from some YouTube conspiracy theorist just before that, saying NASA was covering it up when instead the scientific evidence just wasn't conclusive that it was actually a habitable planet if even a planet because it could have just been two stars' orbits or other rocky bodies.
mrfusion 2 ago 0 replies      
Has anyone considered this planet could actually be a smaller planet with a large moon tidally locked to each other?
transfire 2 ago 1 reply      
This planet is "Nemesis". If you are an Asimov fan, you know why. Damn he was eerily prescient about this one.
withinrafael 2 ago 0 replies      
Melnorme are known to hang out around that system.
thatha7777 2 ago 0 replies      
Sorry for the , but if you took a Space Shuttle to Proxima Centauri it'd take 160,865 years.


cvarjas 2 ago 0 replies      
Two submitted articles on the habitability of Proxima Centauri b:http://www.ice.cat/personal/iribas/Proxima_b/publications.ht...
dghughes 2 ago 0 replies      
If the planet is tidally locked I say that may be good can you imagine an 11 day "year" but also rotating? It would be like living on the Scrambler carnival ride.
Cortez 2 ago 1 reply      
There's too many factors to say the zone may be habitable for life.
Pica_soO 1 ago 0 replies      
well obviously we need the world smallest factory gunned there, directly towards the star, slowing down on solar sails, drifting out into the local orb-equivalent, manufacturing drones and tight-beam equipment.
sangd 2 ago 0 replies      
That will take the New Horizons 73,796 years to fly by.
mirekrusin 2 ago 0 replies      
Would moon also align its orbit? In the icy eyeball it could make some interesting tides like on the football stadium.
stormbrew 2 ago 0 replies      
Was there an error in an early version of this article? There are two comments in here saying 500ly away. Proxima is only ~4ly away.
nxzero 2 ago 0 replies      
Makes me wonder for AI and robotic "life" what would be the best "habitat" for growth.
ralusek 2 ago 0 replies      
Funny that there have been exoplanets found on so many stars, but our closest neighbor can still surprise us.
KamiCrit 2 ago 1 reply      
Well, sounds like it's time to spin up another Space Odyssey book. 4001 anyone?
bordercases 2 ago 0 replies      
Pod Recovered
ianai 2 ago 0 replies      
For some reason this really reminds me of Asimov's Nemesis book.
meganvito 2 ago 0 replies      
Ye, I am at an age too many untangile zeroes that I have no nexus.
rubyfan 2 ago 0 replies      
We should send Matthew Mcconaughey there asap.
justinzollars 2 ago 0 replies      
I hope we can get off of Earth.
Dowwie 2 ago 0 replies      
and maybe the life on that planet just discovered us in the process
derflatulator 2 ago 0 replies      
But is everything on a cob?
ommunist 2 ago 0 replies      
If there is oil on it, centaurian bloody dictatorship cannot be tolerated by progressive democratical forces.
NSO Group's iPhone Zero-Days used against a UAE Human Rights Defender citizenlab.org
1023 points by dropalltables  1 ago   238 comments top 33
linkregister 1 ago 5 replies      
Amazing work by Lookout and Citizen Lab.

Until this point I was not aware that Lookout provided any value-add for mobile devices. I was under the impression it was the McAfee of mobile.

It sounds mean but this is the first reference to actual vulnerability discovery done by themselves on their blog, which usually reports on security updates that Google's Android security team discovered. Previous entries include such gems as "Now available: The Practical Guide to Enterprise Mobile Security" and "Insights from Gartner: When and How to Go Beyond EMM to Ensure Secure Enterprise Mobility."

I can't wait to see more great work. Lookout is now on my radar.

toufka 1 ago 5 replies      
There is a frustration, as a user, that as the value of the iOS exploits increase, they become more and more 'underground'. The time between OS release and public jailbreak is continually growing - and it doesn't seem to only be due to the hardening of the OS. People are selling their exploits rather than releasing them publicly. And the further underground they go, the more likely they will be utilized for nefarious purposes rather than allowing me to edit my own HOSTS file. The most recent iOS jailbreak (to be able to gain root access to my iPhone) lasted less than a month before Apple stopped signing the old OS. Yet its clear this (new) quick action on Apple's part does not (yet?) stop persistent state-sponsored adversaries.

It is more and more clear that to accept Apple's security (which seems to be getting better, but obviously still insufficient) I must also accept Apple's commercial limitations to the use of a device I own. And I suppose that the dividing line between the ability to exploit a vulnerability and to 'have control' is a sliding scale for every user: one man's 'obvious' kernel exploit is another man's 'obvious' phishing scam.

It is not a new tension, but it does seem the stakes on both sides seem to be getting higher and higher - total submission to an onerous EULA vs total exploitable knowledge about me and my device. Both sides seem to have forced each other to introduce the concept of 'total' to those stakes, and that is frustrating. More-so when it's not yet clear which threat is greater.

guelo 1 ago 2 replies      
NSO sells tools that when used violate the CFAA act. It is an Israeli company but a majority share was bought by a San Francisco based VC [0]. It doesn't seem like it should be legally allowed to exist as an American owned company. Maybe Ahmed Mansoor could sue the VC in American courts.

[0] http://jewishbusinessnews.com/2014/03/19/francisco-partners-...

0x0 1 ago 6 replies      
An untethered stealth jailbreak that installs without user interaction from a webview, that's almost as bad as it gets. And for iOS 7.0.0 - 9.3.4 inclusive. And with exfiltration of audio, video, whatsapp, viber, etc etc. So thorough and so bad :-/
hackuser 1 ago 5 replies      
Should exploits like this be treated as munitions, with sale to foreign governments restricted? Or any sale at all restricted? Some thoughts:

* The only uses for the exploits are either illegal or by government security organizations

* I don't think you can just make an explosive and sell it to a foreign government; I think there are strict export controls (though I know very few details, I only read about companies applying, getting approval, etc.).

* In the 1990s, strong encryption was called a 'munition' and export was restricted. That turned out to be impractical (it was available in many countries and the Internet has no borders), morally questionable (restricting private citizen's privacy), and it fell apart.

While I believe in liberty and freedom-to-tinker, as I said, this stuff has no legitimate use.

micaksica 1 ago 2 replies      
The UAE really hates on activists, and appears to be hiring a bunch of people specifically to suppress activists/dissidents within the country. [1] Unfortunately, due to the amount of wealth the country has, it won't stop almost anybody from dealing with them unless Western sanctions are placed on the country, which are unlikely given the current geopolitical situation.


bkmintie 1 ago 1 reply      
Vice has a nice writeup on the exploits as well:https://motherboard.vice.com/read/government-hackers-iphone-...
Miner49er 1 ago 1 reply      
This vulnerability sounds like this:


It was claimed November of last year. I wouldn't be surprised if this "Trident" was sold by Zerodium. Glad it's patched.


I just saw the Citizen Lab article on this:


They mention the Zerodium bounty as well.

epistasis 1 ago 0 replies      
Not having heard about NSO Group before, they've been claiming to have this ability since 2014:


What other 0-days do they have in their pockets?

jtchang 1 ago 3 replies      
The article mentions how this may have been use all the way back in iOS 7 which is crazy.

If you are being targeted for surveillance smartphones are a very bad idea depending on your adversary. A cheap phone that is refreshed regularly will probably be your best bet.

gergles 1 ago 0 replies      
timeal 1 ago 0 replies      
You can be sure that this vulnerability was probably discovered by some researcher, then sold to grey markets like https://www.zerodium.com or https://www.exodusintel.com/ (they pay up to $1 million for a highprofile iOS exploit), who then resold it to some government who is now trying to exploit this dude's phone...
dropalltables 1 ago 2 replies      
Make sure to update to 9.3.5 on all of your iOS devices ASAP!
driverdan 1 ago 1 reply      
To people who work for companies that sell / invest in products that are used in unethical ways (Francisco Partners, NSO, Cisco, etc), how do you justify it to yourself?
scosman 1 ago 3 replies      
Does anyone know if the iOS 10 developer beta 7 (public beta 6) got this patch, or are we vulnerable?
firloop 1 ago 0 replies      
Apple made its bug bounty program public a few weeks ago and the past few iOS updates have all been patching security vulns. It could be a coincidence, but from an outsider's point of view, it looks like the program is working.
artursapek 1 ago 2 replies      
Will 9.3.5 disable/remove the spyware on infected phones? Or does it just prevent one from becoming infected?
SanPilot 1 ago 1 reply      
I'm a beginner when it comes to software development (mostly web development), but it seems to me that the majority of complex exploits like this involve some type of memory overflow and subsequent code execution.

Shouldn't there be methods for detecting these kinds of things in source code or more priority given to preventing it in the C/low-level community?

Osmium 1 ago 1 reply      
Aside, but does anybody else find the switch from right-to-left to left-to-right really jarring in this screenshot?


It has the effect of introducing a line-break into the middle of a line, rather than at either end. I've never encountered this before and it took my brain a few seconds to catch on.

I'd be really curious how native bilingual readers of both a right-to-left and left-to-right language would read that. Does it look natural? Where do your eyes go first?

e28eta 1 ago 3 replies      
I thought it was interesting that they're using Cydia Substrate to hook into specific third-party apps for monitoring.

I wonder if we'll ever see privacy conscious apps using some sort of obfuscation. So that every time you update your app, the attacker will have to reverse-engineer the symbol names again.

It seems like a compile or link time tool could find method call & selector references. As long as your app isn't calling methods using strings, or doing something else tricky, I think it could work.

Or you could just write the app in swift. It's the Objective-C runtime that makes it so easy to intercept method calls.

walrus01 1 ago 2 replies      
This is a REALLY, REALLY good reason why "activists" of any variety should be trained in how to acquire an old Thinkpad and install Debian on it (plus a reasonably xorg/XFCE4 desktop environment). If you're dealing with authoritarian regimes you can do a lot to reduce your attack surface. However at the end it all comes down to rubber hose cryptography. If your government, for example Bahrain decides to detain and torture you, you're pretty much fucked.
eggy 1 ago 1 reply      
Unless you are a high-value target, Apple's security seems fairly sufficient for normal use (I have Android ;)).Companies like NSO Group that state that they play both sides without any moral compass seem like a great target for Anonymous or others. Imagine the client list, and banking information as a trail to blaze!
maglavaitss 1 ago 0 replies      
So, basically three things to notice:

1. never click on links in e-mails.2. if you're targeted by a nation state, you're screwed.3. everybody is vulnerable to rubber-hose cryptography.

Jerry2 1 ago 0 replies      
How does one monitor the infection of an iOS device and how do you capture and store all the stages of an infection?

I've never done any reverse engineering so I'm not sure how you'd go about recording what an infection like this does to your device...

Tepix 1 ago 0 replies      
It's curious that Signal was missing in their list of apps that can be intercepted. Are the targets not using it? Or was it just not mentioned?
matt_wulfeck 1 ago 0 replies      
He wasn't hacked, he was being "lawfully intercepted"!

Just kidding. The difference here is that a government doesn't want to do such as provide reasonable suspicion or go publicly in front of a judge.

metafunctor 1 ago 2 replies      
Is there any way to check if an iOS device has Pegasus installed, without installing and registering for the Lookout app?
abecedarius 1 ago 1 reply      
I have an iPad 1 which long ago was left behind by upgrades. It'd be nice to know when the vulnerabilities were introduced too. Should I stop doing anything networked with it?
chenster 1 ago 0 replies      
I think NSA is trying to acquire them.
dboreham 1 ago 1 reply      
"we did not have an iPhone 6 available for testing"

Big budget operation!

okket 1 ago 5 replies      

 > Alarmingly, some of the names suggested a willingness on > the part of the operators to impersonate governments and > international organizations. For example, we found two > domain names that appear intended to masquerade as an > official site of the International Committee of the Red > Cross (ICRC): icrcworld.com and redcrossworld.com.

landr0id 1 ago 1 reply      
This is off-topic but at first I thought I was on a Spotify blog page. Lookout has very similar branding.
themihai 1 ago 2 replies      
<< Instead of clicking, Mansoor sent the messages to Citizen Lab researchers.

The story is great but I really doubt this. I'm wondering what made him suspect the link? Does he send all the links he receives to Citizen Lab?

Nasa just made all its research available online for free independent.co.uk
773 points by signa11  6 ago   49 comments top 18
thearn4 6 ago 3 replies      
I'm actually curious what is new within this. As it stands, NASA research pre-prints are available on the tech reports server (http://www.sti.nasa.gov/), typically right after the export control review process.

edit: maybe it's related to certain journals where there wasn't access on the STI server? Seems odd to me. For the 8 years that I've been at NASA, it's always been expected that my group's work had to be accessible.

edit2: not all rules are universal across the agency, so my experience may be too specific to Glenn Research Center/my division. In any case, the more open, the better.

WalterBright 6 ago 2 replies      
> Nasa announced it is making all its publicly funded research available online for free

This is the way things ought to be for all publicly funded research, not just NASA. Thank you, NASA, for leading the way.

The beginning of technological progress for mankind started with writing, the beginning of the industrial revolution started with the printing press. Having all the world's knowledge available on your desktop just a click away is the beginning of another exponential leap forward.

codyb 6 ago 1 reply      
Link to the actual data [0].

Looks pretty neat actually. This seems to stem from an executive order by President Obama in 2013. Mobile browsing is okay but I'm excited to check out some of the APIs when I get back to my computer a bit later on. They're seperated by category (Earth Science, Aerospace, etc).

Seems like a lot of the data is already queryable by their api's and I assume there are data dumps and research papers available as well.

Very cool. There is a serious wealth of data and apis available for tinkerers and builders these days from watson to NYC data to Nasa!

[0] - http://www.nasa.gov/open/researchaccess/pubspace

dmix 6 ago 0 replies      
The mars tsunami link is a fun read:http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4872529/

> We conclude that, on early Mars, tsunamis played a major role in generating and resurfacing coastal terrains.

That's a fascinating thing to to think about. Reminds me of the the ocean planet they landed on in Interstellar. Although Mars had giant tsunamis caused by impacts instead of perpetual waves caused by the gravity of a black hole.

iamleppert 6 ago 3 replies      
Great that they're finally getting around to doing this 3 years after the order was signed.

Shouldn't it be a law that any research done with public dollars should be made available to the public for free? Any costs associated with the publishing should be built into the funding.

owenversteeg 6 ago 1 reply      
Ok, so maybe I'm missing something, but what's new here? Is it just the portal?

I've been reading NASA technical papers and publications for years, and although most of the research I was reading was very focused in a specific field, everything I wanted to see was freely available. Even some of the publications linked in the article have been online for a decent amount of time.

joeyrideout 6 ago 0 replies      
Searching the web portal leads to a PMC database query with the filter "nasa funded". Here's a link to a query with just that filter, which returns all 863 articles:


nxzero 6 ago 0 replies      
Saying that "all" of NASA's research is being made available is a stretch given it's known they work on a lot of classified projects with the military, intelligence, etc.:


paul_f 6 ago 0 replies      
This article is so misleading. NASA is requiring that the pubished research papers be public. The research discoveries are still owned by the Universities that did the research - the Bayh Dole Act is still in effect. https://en.wikipedia.org/wiki/Bayh%E2%80%93Dole_Act
thefifthsetpin 6 ago 0 replies      
Title should read "some" rather than "all"
todd8 5 ago 4 replies      
Not NASA, but related: I really want the raw climate data and atmospheric models to be made open and publicly available. I don't understand how researchers can claim that climate change is the most serious problem facing our species while at the same time hiding what they are doing. I know that some data is available, but considering that our government funds most of the research why haven't they put this stuff up on github?

Is the secrecy really necessary in order to get tenure and win grants?

girishso 6 ago 0 replies      
At last... I remember Richard Feynman complaining exactly about this!
halis 6 ago 0 replies      
Great! I glanced over it and I think the next step is to have them provide a For Dumbasses version of pretty much all of it...
eecks 6 ago 0 replies      
Has anyone looked at the resource? Are there data sets and other similar things that could be used in novel ways?
Azuolas 6 ago 0 replies      
following Space X as a top competitor
6 days ago 6 ago 1 reply      
jpeg_hero 6 ago 1 reply      
monksy 6 ago 1 reply      
Including about set design of the moon for the landing?

(For those who just threw a fit.. it's a joke)

Japanese writing system basics candyjapan.com
956 points by bemmu  7 ago   338 comments top 48
viraptor 7 ago 4 replies      
This was an interesting symbol to choose for an internet explanation. It took me a while to realise that the rectangle is actually the symbol that I'm supposed to see, rather than a missing glyph.
imron 7 ago 2 replies      
In Chinese is pronounced sh.

As someone who speaks Chinese, I got a chuckle out of reading 'put your favorite snack in your and t it!' due to the association in my mind of that character and its Chinese pronunciation, immediately followed by a 't'.

primitivesuave 7 ago 4 replies      
Another interesting aspect of traditional Chinese characters is that complex words are expressed by combining simpler symbols. For example, the Chinese word for computer is . The first character represents "electricity", and the second character represents "brain". Which is really what a computer is, an electric brain. Similarly, a computer programmer is , where the three symbols are "rule", "order", and "person" - one who orders rules.

An interesting consequence of this is that you only need to learn around 3000 symbols to read a Chinese newspaper, just like how you can ascertain the meaning of an unfamiliar English word by having knowledge of a small set of Latin/Greek roots.

anqurvanillapy 7 ago 4 replies      
In Chinese, 'eat' is usually ''. We use ' (le)' to say we 'ate' and ' (zi) ' or ' (zh)' for 'eating'.

It is really interesting that in China people often ask their friends '? (Have you eaten?)' rather than ' (Hi)' in daily life. So initially I thought this post was describing something in Chinese w/o my noticing the URL.

euske 7 ago 6 replies      
Yeah yeah, this all makes sense until you see , which isn't "a three-mouthed monster" but "goods".

Languages are weird, man.

andreygrehov 7 ago 5 replies      
I liked the beginning and expected the next word after "mouth" to look almost the same, with a subtle change, which would be a logical extension. But it was quite a jump from a simple square () to god-knows-what ().

Also, why is translated as "Eclipse" in Google?

Sniffnoy 7 ago 2 replies      
Not sure how great an explanation that really is. I like Zompist's explanation: http://www.zompist.com/yingzi/yingzi.htm
zatkin 7 ago 1 reply      
Aren't we sort of starting to doing this with the introduction of emojis? They're a little bit ambiguous, but they do have meaning behind them, nonetheless.
rett12 7 ago 4 replies      
Sometimes I wonder if all these people that criticize or that think that a Latin alphabet can be adapted seamlessly to all languages have tried to study past a beginner level anylogographic language.
dhfromkorea 7 ago 2 replies      
Another interesting side-effect that the compactness of Chinese symbols(other variations: Hanja in Korean, Kanji in Japanese and so forth)allowed was a higher chance of survival against natural disasters like wild fires or crimes like thefts or vandalism.

It was/is far easier to ensure redundancy of scripts and books since the costs of reprinting/copying was far lower compared to other forms of phonetic systems.

The compactness explains how so many archaic, buddhist scripts could survive to this day.

Grue3 7 ago 2 replies      
means "enter", means "mouth". means... "entrance". Actually for most kanji there is no single meaning. Some meanings might even have nothing in common with each other, because they've been based on ancient Chinese wordplay or something.
daveheq 7 ago 1 reply      
"Since we already have symbols for all the sounds we can pronounce"... Not with 26 letters we don't. Other languages have other sounds that English can only try to emulate, and even English has sounds that require multiple letters.
thaumasiotes 7 ago 4 replies      
If substitutes for mouth -- the adjective form of "mouth" is "oral", an etymologically (and audibly!) distinct word. Should that use too?

If king is , kingly is ly, and royal is al, what is regal?

If mouth is and mouthed is ed, why would ate be t rather than ed?

Japan misinterpreted the Chinese writing system (already terrible) into easily the worst writing system known to mankind. It won't look cute when you go beyond two symbols.

partycoder 7 ago 1 reply      
The only problem is that you learn all 2000 at the very minimum and more than that if you actually want to do something practical.

Each one has more than 1 reading, a particular stroke order, and many other things.

dghughes 7 ago 0 replies      
I just discovered NativLang on YouTube so this really is in my zone of interest today.

I've just spent the last few hours learning all about languages how they developed and each culture's spin on adding as much meaning as efficiently as possible to written symbols. I've always loved languages so this was more of a brush up plus learning.

It seems and rightly so ambiguity is death to any characters and efficiency is also fundamental to the character.

I'm not Korean but I like their style literally I like how their language style is so efficient in context to mouth position. It was created because Chinese characters didn't suit Korean language. Japan also streamlined Chinese characters to better suit their culture.

Mayan is another wild language full of meaning in such compact symbols. I had a hard time following their characters.

falcolas 7 ago 1 reply      
Everything old is new again.

I have no , but I must ...

Edit: Nevermind, HN swallowed the Emojis.

paradite 7 ago 0 replies      
This has the added advantaged of being recognized by both native Japaneses and Chinese speakers instantly, as long as we keep it to kanji.

I recognized what the author is doing from the start as a Chinese speaker.

kevindeasis 7 ago 6 replies      
How many symbols are there? And how would the keyboard look like though?
rezashirazian 7 ago 2 replies      
That was cool. Interesting enough, you can come to the same conclusion with emojis.
OOPMan 3 ago 0 replies      
"The English alphabet has only 26 letters, which most kids can master with little difficulty. But we are adults now, why limit ourselves to the 26?"

Less is more. Why waste time learning 2k pictograms (Just for the standard stuff, there's over 50k in total) when you can achieve the same thing with less?

andrezsanchez 6 ago 0 replies      
I think it would be interesting if there were a Latin equivalent of Chinese characters. Different roots could be represented as different characters, some could be used for each of the suffixes like "ly", "tion", etc., and the characters would be joined together to create words like in Chinese.

Different Romantic languages could be represented this way. In the same way that Mandarin and Cantonese use the similar character sets with different pronunciations, and with some characters specific to each one, different languages that have Latin roots would have a few of their own characters specific to their own language, but mostly drawing from the Latin pool.

The pronunciation for each would have to be memorized of course.

frostymarvelous 7 ago 3 replies      
After viewing this a couple of hours ago, ads for Candy Japan are popping up for me on Facebook.
Joof 7 ago 2 replies      
Japanese is an unusual language to write. It's influenced by Chinese (in two separate eras), English and perhaps many other systems. Symbols alone aren't a perfect fit for the language (since they add tense and such), but neither is an English style alphabet.
nxzero 7 ago 1 reply      
>> "Since we already have symbols for all the sounds we can pronounce [in English]"

English has 26 letters, but there are 40+ sounds; which is to say that there are NOT symbols for all the sounds.

codedokode 7 ago 0 replies      
The problem is that many characters look similar but have nothing in common. For example, the article mentions the character meaning "food". So when after reading the article you see similar character like you might think it is somehow related to food. Well, it is not, it just means "good". And another similar character meaning "long" or "leader" also has no relation to previous two.

And when you get to more complicated characters like it becomes even more confusing.

sideproject 7 ago 1 reply      
Really like the live streaming of Google Analytics at the bottom. :) Now I have an idea how much traffic you receive when you reach the #1 spot on HN on the weekend.
danielrhodes 7 ago 2 replies      
I'm interested in how this compactness changes the expressiveness and evolution of the language. In English, as in other languages written in the latin alphabet, you can make changes to the spelling of a word and create new meanings rather fluidly and it's usually easy for the reader to comprehend. I have no knowledge of Japanese, but are there similar possibilities? How do new words/slang get created?
drwicked 7 ago 2 replies      
I'm kindof fascinated by the idea of livestreaming the pageview statistics. I had a surreal moment realizing the weird recursion. Is this something people do now?
giancarlostoro 7 ago 2 replies      
Ancient Pictographic (before Paleo) Hebrew is like this too. Except they only have 22 letters, there's a 23rd but it is no longer part of the 'Aleph Bet'. Hebrew originally had no vowels, and thus today they only put the vowel system in when it's a word people don't normally know how to pronounce. Chinese is similar as well, and I'm sure there's a couple more languages.
fogleman 7 ago 2 replies      
That was fun. Will there be a second lesson?
ryao 7 ago 1 reply      
and are also Chinese characters for mouth and eat, although I had to check google for . Initially, I thought that he was teaching Chinese until I read was eat (which needed Google) and that it was "Japanese".
wch4999 7 ago 0 replies      
Well, reading the first part I just realized this is japanese/chinese! In Chinese all characters are like or . Sometimes we can also break the character down to several parts to understand its meaning.
sova 7 ago 0 replies      
As someone who has spent years of their life dedicated to mastering Japanese, I must say, you sir are a genius.

By the way, I'm happy your site is rockin! I saw it when you launched candyjapan and I am happy for you =)

dingo_bat 7 ago 2 replies      
Slightly off-topic but can anyone here comment on the candies they ship? Are they just ordinary sweets? I've never heard about Japan being famous about candies like Switzerland is about chocolate.
raverbashing 7 ago 0 replies      
alecsmart1 7 ago 1 reply      
This is off topic, but I can't find any pricing on the candyjapan.com website when visiting on mobile (iPhone). Anyone can tell me how much it costs per month?
haddr 7 ago 0 replies      
This article is cool, I love the way is actually encourages to learn something by starting with something simple that you can grasp in seconds.

I was like ()!!!

bootload 7 ago 1 reply      
@bemmu is that a square symbol or something else?
seanmcdirmid 7 ago 3 replies      
billmalarky 7 ago 0 replies      
Love the live video analytics! Such a simple yet effective hack.
BlakePetersen 7 ago 0 replies      
That path tho, /%E5%8F%A3 rendering as /, so slick
smnplk 7 ago 0 replies      
I'm struggling to find a comment that is actually related to the story of CandyJapan and not that character.
fiatjaf 7 ago 1 reply      
Can I believe this? Does Japanese really mix characters that mean things with characters that mean sounds?
ChuckMcM 7 ago 1 reply      
Ok, thats a fun hack.
itaysk 7 ago 1 reply      
why is this post getting so many voteups?
weinzierl 7 ago 2 replies      
Flagged for moderator attention: The link of the article just goes back to this page on HN. Is this an error or did I miss the joke?

Other articles are fine. Reload, clear cache and reload didn't help.

EDIT: Unflagged and sorry for the noise. I clicked on the article's domain and not on the article's title () just as fenomas suspected.

monomaniar 7 ago 0 replies      
Nobody said these are all Chinese letter? Japanese is "invented" and forced educated by Meiji government one hundred years ago. https://en.wikipedia.org/wiki/Meiji_Restoration
disruptalot 7 ago 0 replies      
Finding it surprising no one has corrected the article in that you technically learned Chinese and by proxy Japanese. Chinese traditional characters + meanings largely carried over from Chinese as well.I'm studying mandarin and i was enjoying it until I was told I just learned "Japanese".
Web Scraping in 2016 franciskim.co
832 points by franciskim  3 ago   383 comments top 33
mack73 3 ago 3 replies      
Corporations will abuse your personal integrity whenever they get a chance, while abiding the law. Corporations will cry like babies when their publicly available data (their livelyhood) gets scraped. They will take you to court.

They consider their data to be theirs, even though they published it on the internet.They consider your data (your personal integrity) to be theirs as well, because how can you assume personal integrity when you are surfing the internet?

I have high hopes that the judicial system some time not too far from now will realize that since the law should be a reflection of the current moral standings it will always be behind, trying to catch up with us and that those who break the law while not breaking the current moral standings are still "good citizens" unworthy of prison or fines.

I guess Google won this iteration of the internet because of the double-standars site owners stand by, to allow Google to scrape anything while hindering any competitors from doing the same. There will only be a true competitor to Google when we in the next iteration of the internet realize that searching vast amounts of data (the internet) is a solved problem, that anyone can do as good a job as Google, and move on to the next quirk, around wich there will be competition, and in the end that quirk will be solved, we'll have a winner, signaling that is it time to move on to the next iteration.

minimaxir 3 ago 15 replies      
Keep in mind that companies have sued for scraping not through the API, for example LinkedIn, which explicitly prevents scraping via the ToS: http://www.informationweek.com/software/social/linkedin-sues...

OKCupid did a DMCA takedown for researchers releasing scraped data: https://www.engadget.com/2016/05/17/publicly-released-okcupi...

Since both of these incidents, I now only scrape if it's a) through the API following rate limits or b) if there is no API, and the data has the explicit purpose of being shared publically (e.g blogs), I follow robots.txt. Of course, most companies have a do-not-scrape clause in their ToS anyways, to my personal frustration.

(Disclosure: I have developed a Facebook Page Post Scraper [https://github.com/minimaxir/facebook-page-post-scraper] which explicitly follows the permissions set by the Facebook API.)

fake-name 3 ago 7 replies      
I do a significant amount of scraping for hobby projects, albeit mostly open websites. As a result, I've gotten pretty good a circumventing rate-limiting and most other controls.

I suspect I'm one of those bad people your parents tell you to avoid - by that I mean I completely ignore robots.txt.

At this point, my architecture has settled on a distributed RPC system with a rotating swarm of clients. I use RabbitMQ for message passing middleware, SaltStack for automated VM provisioning, and python everywhere for everything else. Using some randomization, and a list of the top n user agents, I can randomly generate about ~800K unique but valid-looking UAs. Selenium+PhantomJS gets you through non-capcha cloudflare. Backing storage is Postgres.

Database triggers do row versioning, and I wind up with what is basically a mini internet-archive of my own, with periodic snapshots of a site over time. Additionally, I have a readability-like processing layer that re-writes the page content in hopes of making the resulting layout actually pleasant to read on, with pluggable rulesets that determine page element decomposition.

At this point, I have a system that is, as far as I can tell, definitionally a botnet. The only things is I actually pay for the hosts.


Scaling something like this up to high volume is really an interesting challenge. My hosts are physically distributed, and just maintaining the RabbitMQ socket links is hard. I've actually had to do some hacking on the RabbitMQ library to let it handle the various ways I've seen a socket get wedged, and I still have some reliability issues in the SaltStack-DigitalOcean interface where VM creation gets stuck in a infinite loop, leading to me bleeding all my hosts. I also had to implement my own message fragmentation on top of RabbitMQ, because literally no AMQP library I found could reliably handle large (>100K) messages without eventually wedging.

There are other fun problems too, like the fact that I have a postgres database that's ~700 GB in size, which means you have to spend time considering your DB design and doing query optimization too. I apparently have big data problems in my bedroom (My home servers are in my bedroom closet).


It's all on github, FWIW:

Manager: https://github.com/fake-name/ReadableWebProxy

Agent and salt scheduler: https://github.com/fake-name/AutoTriever

prashnts 3 ago 3 replies      
A neat trick I sometimes use to "scrape" data from sites that use jquery ajax to load data is to plug in a middleware in jquery xhr:

 $.ajaxSetup({ dataFilter: function (data, type) { if (this.url === 'some url that you want to watch!') { // Do anything with the data here awesomeMethod(this.data) } return data } })
I remember last using it with an infinite-scroll page with a periodic callback that scrolled the page down every 2 seconds, and the `awesomeMethod` just initiated the download. Pasted it all in dev-tools console, and the cheap "scraper" was ready!

danso 3 ago 1 reply      
This good list of tactics underscores, for me, how the state of the Web has made it a lot more difficult to teach web scraping as a fun exercise for newbie programmers. It used to be you could get by with an assumption that what you see in the browser is what you get when you download the raw HTML...but that's increasingly less common the case. So now you have to teach how to debug via the console and network panel, on top of basic HTTP concepts (such as query parameters).

(Even more problematic is that college kids today seem to have a decaying understanding of what a URL is, given how much web navigation we do through the omnibar or apps, particularly on mobile, but that's another issue).

I've been archiving a few government sites to preserve them for web scraping exercises [0] (the Texas death penalty site is a classic, for both being relatively simple at first, and being incredibly convoluted depending on what level of detail you want to scrape [1])). But I imagine even government sites will move more toward AJAX/app-like sites, if the trend at the federal level means anything.

That said, I think the analytics.usa.gov site is a great place to demonstrate the difference between server-generated HTML and client-rendered HTML.

But as someone who just likes doing web-scraping, I feel the tools have mostly kept up with the changes to the web. It's been relatively easy, for example, to run Selenium through Python to mimic user action [2]. Same with PhantomJS through node, which has vastly improved how accurately it renders pages for screenshots compared to what I remember a few years back

[0] https://github.com/wgetsnaps

[1] https://github.com/wgetsnaps/tdcj-state-tx-us--death_row

[2] https://gist.github.com/dannguyen/8a6fa49253c1d6a0eb92

XCSme 3 ago 6 replies      
Tbh I didn't enjoy the article, it just seems like someone who has just learned about Node.js tried to explain (and mostly failed) how to use some packages to scrape a page. I was expecting to learn some new techniques, but all it explained was how to make a few API calls in order to solve a very specific problem. Also, there was the overall arrogant tone: "I found their interview approach a bit of a turn off so I did not proceed to the next interview and ignored her emails ", this just shows a lot of immaturity.
Jake232 3 ago 2 replies      
Not wanting to thread hijack, but just going to post an article I wrote a few years back as it covers a few other things that are still relevant; and often still gets referenced. May it'll help some people out in combination with OP's post.


stupidcar 3 ago 2 replies      
I wrote a fairly complex spidering and scraping script in Node a few months ago. I found downcache[1] to be absolutely invaluable, particularly as I was debugging my parsing scripts, a I was able to rerun them relatively quickly over the cached responses.

However, when the network was no longer a bottleneck, I found that the speed and single-threaded nature of Node became one. It wasn't really that slow, relatively speaking, but I had a few hundred gigs of HTML to chew through every time I made a correction, so it was important to keep the turnaround as fast as possible.

I eventually managed to manually partition the task so I could launch separate Node scripts to handle different parts of it, but it wasn't a perfect split, and there was a fair bit of duplicated work, where a shared cache would have helped a great deal.

In retrospect, I should have thrown my JS away and started again in something with easy threading like Java or C#. But -- familiar story -- I'd underestimated the complexity of the task to begin with, and by the time I understood, I'd sunk a lot of time into writing my JS parsing code and didn't fancy converting it all to another language, particular when it always seemed like "just one more" correction to the parsing would make everything work right. In the end, what was supposed to take a weekend took about three months of work, off and on, to finish.

[1] https://www.npmjs.com/package/downcache

dchuk 3 ago 2 replies      
Scraping with Selenium in Docker is pretty great, especially because you can use the Docker API itself to spin up/shut down containers at will. So you can spin up a container to hit a specific URL in a second, scrape whatever you're looking for, then kill the container. This can be done via a job queue (sidekiq if you're using Ruby) to do all sorts of fun stuff.

That aside, hitting Insta like this is playing with fire, because you're really dealing with Facebook and their legal team.

mosburger 3 ago 3 replies      
> AngelList even detects PhamtomJS (have not seen other sites do this).

I run a site that aggregates/crawls job boards for remote job postings, and AngelList has been VERY difficult to crawl for various reasons, but you easily get PhantomJS to work (I have). Having said that, I've never felt very good about the fact that I'm defeating their attempts to block me (even though I feel like I'm doing them a favor) and will likely retire that bot soon.

It kinda sucks that I'm just grabbing publicly-available content in a very low-bandwidth way, but I really can't convince myself that what I'm doing is very ethical.

My to-do list includes making my crawler into a more well-behaved bot and that will have to go.

pault 3 ago 0 replies      
I don't know why more people don't use chrome extensions for scraping. Using a boilerplate[1], you can get a scraper up and running in minutes. Start a node server that serves up urls and stores parsed data, and run the scraper in the browser. Best of all, you can watch it running and debug if something goes wrong. I know it doesn't scale well if you're running a SaaS, but for personal projects and research/data normalization it's the lowest barrier to entry, in my opinion.

[1] http://extensionizr.com

franciskim 3 ago 4 replies      
Sorry guys, hit by traffic - just scaling my EC2 at the moment.
jgmmo 3 ago 2 replies      
Good stuff.

I do a good bit of scraping, and made RubyRetriever[1] to make my life easier but it seems like I'm getting roadblocked on occasion, probably due to some of the things you mention in your article.

Is there any way for a site to verify that only their JS and CSS files are linked? Like preventing injection?

[1]: https://github.com/joenorton/rubyretriever

nreece 3 ago 1 reply      
At Feedity (https://feedity.com), we "index" webpages to generate custom feeds. Over the years, we've designed our system to use a mix of technologies like .NET (C#) and node.js, and implemented a bunch of tweaks and optimizations for seamless & scalable access to public content.
IANAD 3 ago 2 replies      
> But if you are automating your exact actions that happen via a browser, can this be blocked?

Yes, by checking times between actions and number of actions in a time period, and blocking atypical activity. I was IP banned from a site once for a few months, after trying to scrape it too much and hitting links on the site that were hidden from humans.

The random wait settings specified in the post are better than nothing, but still too flimsy. You would need to put hours between requests, only request during a certain 15 hour periods, take days off, and eventually you aren't scraping regularly enough to do much good.

Scraping is not an API, and I should know- I used to do it for a living. Its unreliable. It requires constant maintenance. APIs can break too, but they are meant for the sort of consumption you are trying for.

If you scrape for a living, only do it as a side job.

lamby 3 ago 1 reply      
Whilst they mean well, I find this a fundamentally deceptive the arduous parts of "real world" scraping simply aren't in the parsing and extraction of data from the target page, the typical focus of these "scrape the web with X" articles.

The difficulties are invariably in "post-processing"; working around incomplete data on the page, handling errors gracefully and retrying in some (but not all) situations, keeping on top of layout/URL/data changes to the target site, not hitting your target site too often, logging into the target site if necessary and rotating credentials and IP addresses, respecting robots.txt, target site being utterly braindead, keeping users meaningfully informed of scraping progress if they are waiting of it, target site adding and removing data resulting in a null-leaning database schema, sane parallelisation in the presence of prioritisation of important requests, difficulties in monitoring a scraping system due to its implicitly non-deterministic nature, and general problems associated with long-running background processes in web stacks.

Et cetera.

In other words, extracting the right text on the page is the easiest and trivial part by far, with little practical difference between an admittedly cute jQuery-esque parsing library or even just using a blunt regular expression.

It would be quixotic to simply retort that sites should provide "proper" APIs but I would love to see more attempts at solutions that go beyond the superficial.

headmelted 3 ago 0 replies      
I actually love Selenium for this purpose, for much the same reasons the author mentions here.

It's almost impossible for a website to reliably detect that a client web browser is being automated, and I find I can make Selenium scripts much more adaptable to breaking changes in websites when they occur than I can when hooking up my code directly.

I actually disagree with the contention that Selenium is slower than directly scraping though. The Firefox driver has always been lightning fast for me and the bottleneck is almost always server requests that would have been necessary either way.

Twisell 3 ago 1 reply      
What bother me the most is that recently I wanted to extract and archive of all the threads I participated in from an Internet forum. The webmaster told me that the BBS he use don't provide such a function and that I just had to download each thread manually... (300+ thread in my case).

He then say that it don't bother him if I scrape theses thread. And I'm currently figuring out how to manage his site's cookie protected search feature, so that my painstaking effort (I'm not a dev, more a DB guy) could be reproducible more easily by other users of this service.

But this shouldn't appen in the first place because all post of this service are stored in a cleanly organized MySQl DB. Yet as no method is provided the only way to get back structured data is by scrapping (as the webmaster told me that no, he won't run custom SQL because "he don't want to mess his DB").

So even if all the data is publicly available through the internet forum only a geek can download a personal archive... or google because google scrape and store everything.

KennyCason 3 ago 1 reply      
As someone who does a lot of scraping, I was happy to learn about Antigate :)
kingkool68 3 ago 0 replies      
It's trivial to scrape public Instagram URLs...


etatoby 3 ago 1 reply      
Does anybody know what the author means by "lead" (noun)?

I don't think it's any of the regular meanings: http://www.ldoceonline.com/search/?q=Lead

But it doesn't seem to be any of these slang terms either: http://www.urbandictionary.com/define.php?term=lead

frostymarvelous 2 ago 0 replies      
While everyone is busy debating whether scraping is bad or legal, I just can't stop thinking a out Antigate.

Of the sweatshops that must have been setup to deliver this service. That, is to me the true horror of this story.

unixhero 2 ago 0 replies      
And from the trenches:

- rails application

- scraping with nokogiri gem on Ruby

- simple models doing the scraping in rails app

- some scraping is parsed with CSS selectors - nokogiri

- some scraping is parsed with regex - nokogiri

- persisting to DB, Text, even Google docs

- presentation on web, text, pdf, xls


kevindeasis 3 ago 0 replies      
Does cheerio account for single page apps? In any case thanks for the tutorial!

Anyways I added your stuff here along with other data mining resource:


writeslowly 3 ago 4 replies      
Have you run into any issues from running all of your scrapers off of AWS, or just from sites detecting that you're accessing large numbers of pages in some sort of obvious pattern? I guess I was hoping there would be sites with more interesting ways to screw with web scrapers (rearranging certain page elements or something) than just throwing up a CAPTCHA.
zzzcpan 3 ago 0 replies      
> But if you are automating your exact actions that happen via a browser, can this be blocked?

Of course it can! You won't be able to defeat even the simplest attempt on anti-scraping based on statistical data. Like even keeping a list of individual rate-limits for /16 subnets of actual visiting users and you are in trouble.

skeletonjelly 3 ago 1 reply      
Hooray Melbourne! Would be interested seeing this at a meetup group if you were thinking of presenting.
elchief 3 ago 4 replies      
To fight scrapers, we show some values as images that look like text (but not all the time)

And we insert random (non-visible) html and css classes in our site to screw with em, and use randomized css classnames. This fucks with xpaths and css selectors.

You can't stop them, but you can make their lives painful.

slig 3 ago 3 replies      
I wonder how effective the CloudFlare anti-scrapper protection is against this approach of breaking captchas.

Also, I find it interesting that big websites don't just block all traffic from AWS IPs as they do with Tor.

ge95 3 ago 1 reply      
How do you push a button like hit next on a paginated page?
rch 3 ago 1 reply      
There is so much that's missing from this. What about gathering tokens from customers vs. paying for social data feeds? How about canned services like 80legs?
ben_jones 3 ago 2 replies      
Currently getting 502 Gateway. Guessing this post is also trending on reddit and we hugged it to death :(.
rezashirazian 3 ago 0 replies      
When I was building liisted.com I scraped using Selenium and it worked great.
Less stress, more productivity: working fewer hours is good for you and your boss codewithoutrules.com
602 points by itamarst  8 ago   296 comments top 42
pc86 8 ago 16 replies      
> Long hours: "It's 5 o'clock and I should be done with work, but I just need to finish this problem, just one more try," you tell yourself. But being tired it actually takes you another three hours to solve. The next day you go to work tired and unfocused.

> Shorter hours: "It's 5 o'clock and I wish I had this fixed, but I guess I'll try tomorrow morning." The next morning, refreshed, you solve the problem in 10 minutes.

I have too much experience with this. I remember when I was self-employed I humble-brag tweeted about my production deployment ~15 minutes after starting work. A friend of mine asked why I hadn't just deployed last night and the quote above was my first thought (in addition to the fact that you should never deploy to prod then go to bed). I was done with work, so I stopped. It avoids hitting that "grind it out" phase where you're lucky just not to do more harm than good.

bcrescimanno 7 ago 7 replies      
Just this week, I've been collecting feedback from my team. Some of the most powerful I received was, "Since you took over, I feel like I've gotten more done with so much less stress and extra hours. Until you came in, I didn't believe that we could really have work-life-balance and be MORE productive and now I know we can."

Having been in companies where my approach has been called everything from "lazy" to outright "damaging," this feedback was both rewarding and validating.

Edit: For those who are in jobs where you're putting in those long hours, I can tell you there are managers out there who believe in the message here. I have three of them working for me right now and I constantly reinforce the philosophy of working sustainably. Don't settle for less than a manager who respects you the person--not just you the engineer!

jhummel 7 ago 6 replies      
I started a consulting/services company a few years ago, and this year was finally in a position to start hiring. I'm a big proponent of this type of thinking, so while I couldn't offer the best salary, or the best benefits - I could offer a better work/life balance. I knew it was a win-win for us and the new employee.

We landed on a 36 hour work week. 8-5 Mon-Thurs, then 8-12 on Friday. We had more applicants than I ever dreamed of, and scored a great hire that was coming from the 70-hr-a-week startup life. Everyone's been extremely happy over the past six months, and I don't feel like I'm missing out on any productivity what-so-ever.

gthtjtkt 7 ago 6 replies      
> One programmer I know made clear when she started a job at a startup that she worked 40-45 hours a week and that's it. Everyone else worked much longer hours, but that was her personal limit. Personally I have negotiated a 35-hour work week.

Wow, what a depressing time to be employed. The old standard (35 hours) is now considered a "short" workweek, and only the most desirable employees have the leverage to request it. Not to mention stagnant wages, rapidly rising costs of living, and off-hour availability expectations.

How did we end up here?

imh 7 ago 7 replies      
>There are companies where this won't fly, of course, where management is so bad or norms are so out of whack that even a 40-hour work week by a productive team member won't be acceptable. In those cases you need to look for a new job, and as part of the interview figure out the work culture and project management practices of prospective employers. Do people work short hours or long hours? Is everything always on fire or do projects get delivered on time?

I have never figured out how to ask this. It always feels like I'm asking, "I don't like to work much, is that ok with you?"

dasil003 7 ago 2 replies      
Having spent a lot of time both as a manager and as an individual contributor, here's the rub (at least for programmers): software developers are a classic market for lemons. The bigger the org and the more operational complexity the more space there is for poor developers to hide out and draw a pay check. But even at a small scale it's remarkably difficult to measure productivity actually.

Therefore, what you need is intrinsic motivation. You need engineers who really care about solving the right problem at the right time in the right way. 35 hours or 70 doesn't matter if they can't get that right, and it's very hard to have management that is qualified to make that judgement on people (especially as an org grows and management becomes a full time job). So the problem is that given this opacity, finding someone who works 70 hours a week is a better proxy than someone who seems excited about a 35-hour work weekthe former are people who are clearly driven whereas the latter are literally everyone. It's far from a good metric, but perhaps the best one available to the pointy hairs of the world.

Speaking from personal experience, it wasn't until I had a kid that I found out what I could do in a 40-hour week. Once I had that time constraint, it forced me to be more efficient in a very deep and fundamental way from the core of my being. When I was 25 I would hit 5pm and think to myself: I still have another 8 hours to solve this problem before bed. Perhaps this is post-hoc justification, but I think that makes sense when you are just starting out and not really competent yet. Once you've passed the magic 10k hours or whatever, I think turning over problems in ones subconscious can provide a lot more of the value. So sleep / exercise / meditation can all help elevate your performance far more than extra hoursbut only if your brain knows what it's doing.

tonyjstark 7 ago 3 replies      
Of course every personal experience is just anecdotal but since I only work 24 hours a week I feel able to educate myself, do sports, feel more healthy. Before that I worked one week full and one week only 4 days repeatedly and I really got the feeling that coming in on Mondays after a short week was so much easier. I felt more relaxed and satisfied because I could do other stuff on the weekend than trying to catch up with buying things and all that organizational junk you have to do as an adult.

Now I work 6 hours a day, 4 days a week and I don't take long breaks anymore. I stopped to read blogs or news at work after lunch, I just work. Would be really interesting if this is just me or if it could be scientifically proven that working less is more.

codingdave 7 ago 6 replies      
Even better, drop hours as a metric of work. Offer the flexibility to work whatever hours are most productive... and find a boss who doesn't care whether that is 40 or 20, as long as he gets the value he expects from your time.

After all, employment is a business contract. You are given money in exchange for the value you add to the organization. Not hours... value.

rabino 8 ago 2 replies      
I was expecting the list of sources at the bottom of the article to be a mile long. Not a single one. Hot damn, so much broscience and red herrings.
shams93 7 ago 0 replies      
Its the Theater of Productivity, similar to the TSA. You can see Meyer's approach to productivity was ultimately just as successful as the TSA on secutity. Yeah doing an early stage startup it could be on the side of a day job so yes you are going to inevtiably put in a lot of hours for $0 doing a startup. But for regular businesses that are well established such as Yahoo this isn't productivity its Productivity Theater in the same way the TSA are Security Theater.
anupshinde 7 ago 1 reply      
I think the key is less-stress and not shorter hours. Meetings, especially long ones, induce stress.

I've tried reducing hours (as a freelance remote developer). I cut my working hours to 25 per week from somewhere around 60ish per week. (And I could afford losing out money)

At first, I felt I was missing something. I thought it was the money. After being completely off work for a month, I realized I was missing the people (Slack chats, meetings, fire-fights)

When I started back with another client, working just 25 hours per week was much much much harder. None of these worked: 8x3days, 4x6days, 6x4days, 5x5days. Few weeks I could not even complete 15 hours, and other weeks I was over-working. I was still stressed. Finally, what worked was odd - (10-12 hours)x2.5 days. So I ended up working a bit more hours than I wanted to. It was proven again that it takes time and focus to pickup momentum and costs a lot more to loose it frequently. And I still work on my other stuff totalling to about 45-50 hour work-weeks and still feels much less stressful.

jupiter90000 7 ago 1 reply      
There are certain jobs where it may not work, but from what I've seen places I've worked, 40hr workweeks seem like a holdover from some, in my opinion, truly bizarre cultural expectations. I mean, at some offices I've worked people are basically shooting the breeze for at least 10 hours at the office a week with coworkers. Some of them don't like being at home with their families and so seem to think work is a way to get away from their responsibilities outside of work. I'd rather have that 'time bullshitting with coworkers' to do things I like to do not at work.

Is this some holdover from previous generations of just wanting to pressure everyone to do what they had to do? Sure, sometimes things need to get done, but alot of jobs aren't saving lives or doing much amazing, but "hit that deadline or we're going to die" is almost the implication made in business in the US in my experience...

Hopefully we'll stop this work addiction some day and realize lots of things can get done without making people be behind a desk an arbitrary number of hours per day...

dsugarman 7 ago 0 replies      
Henry Ford originally found that 40 hour work weeks were optimal for productivity for his factory employees. Before this, they only had one day of rest and were generally overworked and inefficient. I think the work of a factory worker is vastly different than the work we do today behind a computer, especially programmers where your brain power is everything. It would be interesting if a big enough company were to do some proper research on the most efficient working patterns of a programmer today.
Apreche 8 ago 3 replies      
Recruiters. Find me a company that believes this, and I'll actually reply to you.
fizixer 7 ago 0 replies      
In my experience, in order to maximize the productivity in creative work and still work 40 hours, you have to pick the best 40 hours out of a 168 hour time frame:

- If it means you work '13 hours 20 minutes' on Monday Wednesday Friday, so be it.

- If it means you work '5 hours 45 minutes' Monday thru Sunday, so be it.

- If it means you work one way one week, the other way the next week, so be it.

- If you work best in the middle of the night, so be it.

- If it means you work 10 hours everyday for 52 days straight, and take the rest of the quarter off (39 days!), so be it.

The problem is, we have to do this thing called 'work/life balance' in which we are at work 9 to 5, whether we feel fresh or tired, and have to forget about work for the remaining 128 hours.

Combine that with the need for in-person interaction with the team, the boss, and what not, it gets worse.

IMO you can't have it both ways.

zxcvvcxz 7 ago 3 replies      
Yeah.... not so much for me. I'm going to give an alternate point of view.

To become world class at something, you need to be obsessed. You need to put in as many hours as possible and just love what you're doing.

"Oh but you won't be as effective after 8 hours of work!"

So what. Let's say I'm only 20% as effective after 8 hours. I'll take it -- 20% of the last 5-6 hours of the day towards my craft over 0% doing 'normal' things (drinks on the patio? Pointless travelling? Whatever my fellow annoying millenials like to do).

This is the kind of mindset you need to reach the top of a field. And it should develop naturally, you should really want it. Whether it means outdoing everyone at your company, getting that prototype done two months earlier, closing more leads, getting that tricky piano passage, whatever.

If you love what you're doing, more is more. Because you probably can't help yourself. If you feel like working more, just do it. Don't let normal social expectations hold you back, especially if you're young, because you only get so long to become great at something.

I'll finish with one final caveat. Figure out what level of sleep, exercise, and nutrition your body needs to sustain your desired work habits. Get those right ASAP, keep trying modifications, and realize it's different for everyone's unique biology. I like 8 hours of sleep a night, weightlifting 3x/week at 45-60mins each, and a certain amount of protein and certain vitamins (and coffee of course, ha). You'd be surprised how these 3 lifestyle factors can make such a huge difference in your energy levels and hormones -- one can literally become a different person!

sakian 7 ago 4 replies      
How does a company like SpaceX fit into this? They seem to promote the opposite and are seeing some pretty amazing success. Do they succeed because of or despite working long hours?
yodsanklai 7 ago 2 replies      
The most (professionally) successful persons I know worked very hard at least at some point in their life. For instance, I know tons of professors that pretty much work all the time. Same thing with former schoolmates working in finance now.

I'm not saying it's the road to follow, and obviously, not all careers are comparable, but I think that in order to reach their full potential, people have to (and can) work a lot.

SonicSoul 7 ago 2 replies      
There is no science here what so-ever. Just a bunch of feel good rah rah BS about working less hours and getting more done. Some fodder to go to your manager with I guess.

Who said that 35-40 hours is optimal? How can you even quantify it considering every job is different. I've had jobs where 50% was brainless support so i'd work 10-12 hr days and my brain was totally fine to keep going. Some jobs come with inevitable social BS that will take 50% of your time anyway.

The problem is that management and developers have different incentives. Managers [in more cases then not] are pushed by executives to justify their worth and deliver results.

Developers want to create, but also don't care as much about the bottom line. They want to have a life outside of work.

It's that simple. Let's not go pretending like anything beyond 40 hours takes 10x time to get done because it simply doesn't. It's just an empty thing to say like "you should give me a raise because of inflation".

There are plenty of people working their own startups doing 80-100 hr weeks and getting shit ton done. Does it mean that you should do this at a corporation? Maybe not, maybe yes. It all depends on the trade offs either side is willing to make. Managers will of course be incentivized to push for more productivity per employee.

Places such as Basecamp take a stance against this incentive gap. They're actively working at making employees take more vacation, sleep better, and have a better work life balance. This must improve employee morale and retention. No idea what it does to bottom line or employee paychecks.

As a manger I've had people that worked 40- hrs and delivered amazing results. I'd never even think about caring how many hours they've put in. Then there were others that worked more but delivered a mound of technical debt or nothing at all. They were usually the ones most vocal about working too many hours. There are very few people out there that can come in, get straight to work, and deliver quality. Simply making a blank statement that they will do more by working less is silly.

nathan_f77 7 ago 1 reply      
Right now I'm working 20 hours per week as a contractor. It's beautiful.

My hours are also pretty flexible. If I'm really in the zone, I can work a 10 hour day. That's what I did today. I started at 3pm, had a break around 7pm, and I just finished what I was working on at 2am (30 minutes ago).

Yesterday, I wasn't feeling motivated at all, so I just ended up taking the day off and doing something else.

I'm living in Thailand, so normally I'll work from 10pm until 2am, so I have overlap with clients in the US. I wake up at 10am, and it feels like I have the whole day to do whatever I like.

I'm earning less money, but I'm somehow saving much more than I did in San Francisco.

atom-morgan 7 ago 0 replies      
Imagine trying to convince a slave master that freedom, in the long run, will lead to more productivity on a scale that they can't even imagine. I believe working fewer hours is a modern day version of that.
ivanhoe 7 ago 0 replies      
I've noticed this when playing chess, which is I guess a pretty good way to measure one's analytical capabilities and concentration. When playing against a computer in the evening after 8+ hours of work my games are significantly weaker; I can't anymore beat the same game level that I easily win in the morning. By looking into game stats I've noticed that in the evenings I make significantly more mistakes, don't see good moves, make hasty decisions more often, etc.
dumbfounder 8 ago 2 replies      
Sounds nice, but maybe some evidence would be good?
namenotrequired 7 ago 1 reply      
I thought this was about working fewer hours than full time, but apparently it's about working no more than full time.
ErikAugust 7 ago 0 replies      
Tired is the worst time to code. There are other things you can do to be productive - but try not writing mission critical code when you are exhausted.
afarrell 7 ago 1 reply      
This also applies if you are in university. Your ability to learn is directly related to the quality of sleep you have gotten. If your school believes "sleep is for the weak" and makes labs due at 6am, please realise that this is bullshit and find tools/habits to enforce a different discipline.
Abdizriel 7 ago 0 replies      
I agree with that. Last ~1-2hours in job I feel bored and everything is not so productive than it should.

I would like to work in/create place where engineers work for ~30h a week including lunch/coffee time.

I think with that kind of work everyone would be happy and could easly balance work/life

tzakrajs 7 ago 0 replies      
The key to this article at a macroscopic level is the willingness to quit when the manager or culture are unwilling to adapt to your fewer hours proposition. We can positively change the industry for everyone by uniformly following this guidanace.
aledalgrande 7 ago 0 replies      
I find that, for boring tasks or everyday work, this is true. I cannot use "turbo mode" for too long.

But if there is something really challenging in which I plunged in, and at the same time I am getting results, it will break the rule. I think much of what written here makes sense: https://en.wikipedia.org/wiki/Flow_(psychology)#Conditions_f...

Also, if you really want to be productive, it's not about sleep, but about having very defined and organized tasks, so you limit the number of open decisions you have to take in a day.

Noseshine 7 ago 0 replies      

Does anybody have time to comment on the article submitted by OP? :)

mindfulgeek 7 ago 0 replies      
The best "full time" work environment I experienced was a 9-5 week with morning/afternoon 3-hour coding blocks (no interruptions tolerated) and an hour lunch. The other hour was spent in the standup, email, one-off conversations and blogging/community work. We were a highly production and creative team.
ShakataGaNai 7 ago 0 replies      
This is one of the reasons why more telecommuting/remote work is "better". Since you've got no boss looking (literally) over your shoulder, you can't be judged on hours. Then things like output or project completion actually bubble to the top as most important.
us7892 7 ago 0 replies      
Bottom of the article has a link...to learn from your mistakes (perhaps because you worked too many long hours.) We can get emailed a list of his mistakes. https://softwareclown.com/
jsprogrammer 7 ago 0 replies      
Good article, but I don't see where it addresses how working fewer hours is good for your boss (though, you can try to read in to it). The article even says that it (working less hours) won't work at some companies.
franciscop 7 ago 0 replies      
I thought this was about reducing it from 40h to 30h or so, but it was about fighting for the 40h themselves. I am in Japan now and things are really crazy here for my friends.
Waterluvian 7 ago 0 replies      
I've been fortunate enough that my employer encourages people to find the patterns that work for them. Now that I've been there a few years, I've been taking greater liberties on when I come and go. Some days I just wander home at 3pm because it's just not happening. Other days I'm there 8-8 and skip lunch because I'm so excited to see a new feature work. I've never been so productive or happy.
rdlecler1 6 ago 0 replies      
For much of the article the author is saying that with less time to complete a project you now use your time more wisely and you'll get the same amount of work done in a shorter window. If that was the case then managers should instead add 20% more to every project. Now you'd be delivering 20% more with the same resources as before. Not buying it.
rdlecler1 6 ago 0 replies      
For much of the article the author is saying that with less time to complete a project you now use your time more wisely and you'll get the same amount of work done in a shorter window. If that was the case then managers should instead add 20% more to every project.
zooko4 7 ago 0 replies      
Hey, I know the guy who wrote this! As a boss, and in fact as the former boss of itamarst, I agree with this strategy. I want employees to produce high-quality work, to be predictable in their pace, to not get burnt out, and to be happy and pleasant to work with.
stevenwiles 7 ago 2 replies      
known 7 ago 0 replies      
Not for Non-IT
mastratton3 7 ago 2 replies      
I agree with the fact about cutting out meetings but I don't know if I agree with everything here.

While its easy to say "Just leave at 5pm and come back refreshed" is easy to say, I found personally to have learned to most on the days when I've worked until 2am really digging into a problem and getting it fixed. I also have found that the best people I've worked with have a similar get it done attitude. Obviously this isn't sustainable but I think required from time to time.

I also think this lacks the notion of "how much do you want to move up vs stay in your current role". If you don't have any desire for more responsibilities than this applies, but I think if you want to move up/learn new skills, reducing the hours works against you.

I guess I follow the mantra of, "The first 40 should be productive and for the employer in the role I was hired for but I should spend additional time on top of that learning new skills"

Node.js is one of the worst things to happen to the software industry (2012) cat-v.org
620 points by behnamoh  4 ago   548 comments top 61
Jemaclus 4 ago 19 replies      
Disclaimer: I worked with Node.js full-time for about 14 months, ending about 9 months ago. I'm willing to admit that the following complaints may or may not be out of date, especially with ES6 and newer Node versions. Proceed with grains of salt at the ready.

There are a lot of things to like about Node.js, but the primary thing that bothers me about it is the obsession with async. It's a language/framework designed to make pretty much everything async.

In the real world, almost everything is synchronous, and only occasionally do you really want async behavior. By that I mean, you almost always want A, B, C, D, E, F, G in that order, and only occasionally would you say that you want async(H,I). But with Node, it's the other way around. You assume things are async, and then specify synchronous behavior.

Instead of assuming A; B; C; D; E; F; G;, you wind up with a ton of code like A.then(B).then(C).then(D).then(E).then(F).then(G);

...I know that's a contrived example, and I know you don't really need to do it that way, but so many people do, and it really illustrates the point. In Node.js, you are explicitly synchronous / implicitly async. Most other coding paradigms (including Go) better match what I consider reality, which is that everything is implicitly synchronous, and you specify async behavior when you need it.

Basically, I think it's backward. But perhaps like the OP, I just can't wrap my head around it.

The NPM stuff... well, I think all ecosystems have their pros and cons. I'm not a huge fan of NPM, but it does the job for the most part, and I'm curious as to how people would actually improve it, rather than just complain about it all the time. I don't really have any good ideas (knowing nothing about how package management actually works under the hood).

rdtsc 4 ago 4 replies      
I usually ask people to explain why they picked or like (or dislike) a particular technology and that surprisingly tells quite a bit about their proficiency level.

At least in interviews I found it tells a lot more about their proficiency than say knowing how to invert binary tree in under 20min or solve a digital circuit diagram in with object oriented principles.

Node.js is a technology that raises red flags when someone advocates it. I've heard stuff like "it's async so faster", "it makes things non-blocking so you get more performance not like with threads", "you just have to learn one language and you're done", "...isomorphic something..." When digging in to discover if they knew how event dispatching works or how these callbacks end up called data comes in on a TCP socket, and there is usually nothing.

The other red flag is the community. Somehow Node.js community managed to accumulate the most immature and childish people. I don't know what it is / was about it. But there it was.

Also maybe I am not the only one, but I've seen vocal advocates of Node.js steam-roll and sell their technology, often convincing managers to adopt, with later disastrous consequences. As article mentions -- callback hell, immature libraries, somehow the promised fast performance guarantees vanish when faced with larger amount of concurrent connections and so on. I've seen that hype happen with Go recently as well. Not as bad, but there is some element.

Now you'd think I am 100% hater and irrational. But one can still convince me that picking Node.js was a good choice. One good thing about Node.js is it is Javascript. If there is a team of developers that just know Javascript and nothing else. Then perhaps it makes sense to have a Node.js project. Keep it small and internal. Also npm does have a lot of packages and they are easy to install. A lot of them are un-maintained and crap but many are fine. Python packaging for example used to be worse, so convincing someone with an "npm install <blah>" wasn't hard.

leonroy 4 ago 5 replies      
I've been a software dev for 10 years. When I started I saw the transition from Perl to PHP and a lot of snobbishness from the former towards the latter. Seeing the changing of the guard in web languages was pretty instructional and it's something I see again and again.

I think basic CompSci courses should really have a course or two on managing software projects and handling the problems of what framework do I use to build my new software app? Because fundamental language or framework decisions have both a technical and a business component and even as a front line programmer it helps to be aware of both.

Node.js is a great environment for getting a server side app going fast and it has very good tooling thanks to the rest of the JS community with additions like npm, gulp, bower, express etc. There's obvious benefit in having shareable libraries between client and server side and most importantly (to software companies) hiring coders who can work with it is far, far easier than say finding that rarest of unicorns - an experienced Haskell developer.

If (and it's a damn big if) you outgrow Node.js you're doing well. Then (and only then) look at the alternatives like Play Framework, Spring Boot, Vert.x or whatever else floats your boat.

Rants can be useful in giving a kick up the asses of the relevant community to go address certain bug bears. This rant though is so damn generic it reminds me of those Perl developers at college pouring cold water over the idea of using PHP because they felt threatened by it.

JDiculous 3 ago 6 replies      
Anybody can criticize a language or platform, but it doesn't mean much if there aren't any better alternatives.

This article presents an extreme conclusion without much supporting evidence, so it's pretty pathetic that this made the front page. Nobody even uses callbacks anymore now that we have Promise and async/await.

Yes, Javascript isn't the best language (though ES6 improves tremendously on ES5). But right now it's the only language you can use in the browser (aside from languages like Clojurescript that compile to Javascript). The biggest advantage of Node.js is that you can reuse the same code on the client and server, and thus it's ideal for creating universal single-page web apps. Being able to reuse the same code on the client and server is a massive advantage that can't be understated.

Also, Node.js with Nginx is more scalable out of the box than Ruby on Rails, Python/Django, PHP, etc. Hell it's comparable to Java, which is incredible for a dynamic language. The difference is, you can write a Node.js web application 10x faster than the equivalent application in Java, and with a drastically smaller codebase (less code = less code to maintain). These days developer time is the biggest cost.

These rants come off as coming from either (1) back-end developers who never touch UI code or anything on the client-side (not where Node.js thrives) (2) armchair commentators who don't actually have to get shit done in terms of building and deploying web apps on deadlines, and thus have the luxury of criticizing everything without presenting realistic alternatives.

> "There are only two kinds of languages: the ones people complain about and the ones nobody uses." -Bjarne Stroustrup

imagist 4 ago 5 replies      
It's telling that none of the posts defending Node.js are talking about it's technical merits. They're all saying:

1. Attacks on people--you're being too negative, you're saying this to feel superior.

2. Choosing Node.js is a tradeoff! We can't really say what you get in exchange for using this crappy ecosystem, but "tradeoff" sounds good even if you're trading using a reasonable ecosystem for nothing.

If you really think Node.js isn't a flaming pile of crap, I challenge you to come up with something it does that isn't done far better in another ecosystem.

sebringj 4 ago 3 replies      
The number one issue I've found with Node.js is when developers make things overly complicated for no apparent reason. The bar may be too low to get in so possibly you'll get a higher degree of poor design decisions.

The second would be the overuse of build scripts in that the build seems more complicated than the app in both time to get the thing up and complicated chaining steps. I've not had much fun debugging grunt, gulp or webpack in some of these fortune 1000 projects and I have a hard time wanting to give a shit about knowing them in great detail as the app should be the focus.

The parts of node that I most like are the core libraries that come with the install. When I try to stick with those as much as possible rather than using some half-baked npm module for every whim, I have a very pleasurable experience.

The async/wait and promises, etc along with piping streams are quite elegant in how modules can be snapped together like lego pieces but I find that people fuck it up terribly when they half know it as I initially did and it becomes akin to the messiness of es5 callbacks.

It does take some time to really utilize async well so I would recommend to read up on those concepts in great detail prior to jumping in.

Please npm responsibly.

spion 4 ago 1 reply      
The pot is calling the kettle black.

Its always funny to read something like this coming from proponents of Go. Now thats a language that threatens to single-handedly bring us back 30 years in the past. A typed language invented in 2009 lacking generics and algebraic data types, and being proud of it (or at least its users being proud) - thats pretty much at the exact same level as JS "reinventing" callback-based async programming in 2009. And its users were similarly proud of this in 2010-2011.

At least Brendan Eich is modest enough to apologise for JavaScript's flaws and push as much as possible for fixes without backward compatibility breakage of the web.

lootsauce 4 ago 17 replies      
There is an easy sense of superiority that comes with derision of "X" and the authoritative sounding romanticization of idealized "Y" seemingly adds weight to the argument. Clearly these "Y" people are smarter and better than all those "X" people.

Look at all this horrible code! So sad that all these people are not as smart as me. Look at this horrible language! So sad the people that created it are not as intelligent as me!

Really? There is no more "problem" with Node.js than there is a "problem" with any other platform. There is no more problem with JavaScript/ES-(name your flavor) than there is with any programming language. Different languages are different. Different platforms are different. Of course every system has its own problems. Sometimes people who appreciate them call these "tradeoffs" or the superior types call them idiotic.

The cliche of hacker news haters is really really really getting old. So here are some things that are actually good.

As much of a pile of hot steaming code as it is, Babel as an idea (AKA transpiling one language to another) is pretty cool. Of course you can do this in other places but its featuring prominently in the JS community leading to an interesting result. The language and its features become configurable, easy to adapt and change and evolve over time and suit to your liking. This is interesting!

Finally as opposed to what others may have said about the community being childish I have found the opposite. I find it to be very welcoming and supportive, friendly and honestly creative. Of course there are lots of negatives, lots of horrible code, lots of mistakes happening. But what is missed in all of this? Theres A LOT of stuff happening that is good even great! It's beautiful chaos! So go on hating, but I see lots of great stuff out there. As one great systems and iOS developer told me the other day "Have you tried Express? Its awesome!" HA yeah. But he just tried it, and loves it!

Oh but look at that callback YUCK! Cmon

jondubois 4 ago 2 replies      
I don't see anything wrong with Node's approach to concurrency. It uses IPC which is much more scalable than threads and mutexes. Also, you don't have to use callbacks anymore, now we have Promises and there are tons of libraries that allow you to do reactive programming so you can just wire-up streams of data together in complex sequences.

This article is just inflammatory and illogical.

I think Node.js is one of the best backend engines which was ever created - And I've programmed in everything including AVR Assembly Python, PHP, C#, C/C++, Java and many others. I like Node.js the most.

And yes, you're right, the Node.js community isn't a 'proud' community - We're more interested in constantly improving than sitting there being satisfied with ourselves whilst bashing other tools.

There is no perfect tool/stack; they all have pros and cons. It's all about personal preferences.

wyqydsyq 4 ago 1 reply      
All the points in the first (Uriel) rant are attacking JavaScript itself, and most if not all those points are remedied by using ES2015 and FRP.

All the points in the second (Ryan) rant are attacking bloated/poorly-scoped software and abstractions, and none of the points (except mentioning $NODE_PATH) have any relevance at all to Node.js.

So really none of this is relevant to Node.js (or JavaScript) being inherently bad, it's just pointing out that people can use it for bad things, which is no different to any other language or runtime.

ralusek 4 ago 1 reply      
"The async-style of programming is almost entirely inaccessible to my brain"

Proceeds to write article shitting on language which relies on this.

bryanrasmussen 4 ago 1 reply      
I am certainly willing to entertain the notion that async models do not map naturally to most peoples' brains and as a consequence the callback style of concurrency might just be really difficult to handle. On the other hand the writer admits it may just be them not being able to understand it, which I think is also reasonable given that many people have managed to do more complicated stuff in much less than 2 months.

This post is a land of contrasts, in short.

moonshinefe 4 ago 0 replies      
"processes orchestrated with C. Its a beautiful idea."

That's all well and good until programmers don't know how to manually manage their memory properly or use pointers and it creates massive security holes and bugs in general. Which is unfortunately very, very often.

Apparently this guy disagrees and that's fine, but in my experience he's wrong to bash complexity and praise C. I've read many large C projects' code, and it's about as complicated / messy as it gets.

Some good points in the article other than that.

yoshuaw 4 ago 1 reply      
Here's the post by Ry that was supposedly deleted: http://tinyclouds.org/rant.html

The point I took away from the post is that all abstractions carry a cost, no matter how elegant. Not "Ryan Dahl hates Node.js". But sure.

Seems the author is intent on complaining and that's cool with me, but if you're doing Node then yeah don't let people like this get you down. Node's fast, Node's flexible and has so many users that virtually any abstraction you like is available through npm. Callbacks are part of core because it's the lowest common abstraction for Async. Node is not Python.

Tired devs can complain all they like, but that doesn't make them right.

partycoder 4 ago 1 reply      
JavaScript in Web browsers has a lot of sandboxing (i.e: babyproofing). Node on the other side doesn't. If you are an experienced developer, chances are you won't do something lousy like coding by trial and error and playing with concepts you don't understand and copying and pasting from StackOverflow.

If you follow the golden rule of not touching anything you don't understand, you should be in a safe spot. But that's not the culture in the node community. The node community is all about sharing lousy snippets of code without error handling, without input validation, and without any regard for any non-functional requirement in poorly written blogs and npm. A community of excessive optimism and irrational risk taking.

Floating point numbers, for instance. Every number in JavaScript is a floating point number, and 99% of node developers don't understand a basic thing like how to compare 2 floating point numbers. And that's just the basics... let's not even discuss concurrency, parallelism, how to deal with files, memory, I/O...

jtolmar 4 ago 8 replies      
So what do people use as a server these days?

All I want is to annotate some function calls with paths and have something handle http requests and mangle the input and output into json for me. Preferably in a boring safe language. Preferably in Java, the boringest safest language.

If I have to call a method on a provided object instead of returning in the name of async, great, that's fine too.

soupdujour 3 ago 0 replies      
I don't understand why people think node doesn't scale or that you need to use something else when it's time to "get serious". I can't get into specifics, but I recently worked on an app for a rather large company that had 10s of millions of daily users. The app written in node needed 3x fewer cpus than a java app running in the same system and they were performing similar tasks with almost identical usage. The team maintaining the node app was also 4x smaller than the team maintaining the java app.

The main difference was that the node app was thoughtfully architected and written and the java app was not. It didn't have anything to do with language or tooling. It had to do with good software engineering.

coldtea 4 ago 0 replies      
What those repeating "async is good" don't seem to get is that we have had better ways to do async than what Node.js offers for decades.

Better as in, all of: faster, easier to reason about, better implemented, more robust.

Erlang is but one example.

agentultra 4 ago 0 replies      
The worst thing is a gross exaggeration. I've been guilty of these at times. It seems common. You have to be a very passionate person to care deeply enough to master programming. These are infuriating machines operating in the domain of discrete maths... one error and the entire building comes down.

That being said sometimes, when there's no one else you can kvetch with, a good rant is just what you need. Computers are stupid. Programming is horrible. Everything is terrible.

But Node.js is hardly the worst thing. Some perspective is required.

Ericson2314 4 ago 3 replies      
I dislike shitty languages and successfully avoid using them myself, but I do have to be a bit happy with NPM ecosystem teaching people to reuse more code.
zumu 4 ago 0 replies      
> As soon as I wanted to write any non-trivial code to read stuff from a database and do something with it, I got stumped - I didnt know how to proceed. I could write some code, but it would turn out very ugly. I couldnt write code that was pleasing to read (and it certainly wasnt pleasant to write).

When this was written, he had point, but with es6 I don't think this is a problem anymore.

Accordingly, I don't think this stands

> You lose expressiveness

Arrow functions and destructuring cut down on boiler plate significantly.

k__ 4 ago 1 reply      
What I find interesting is, that everyone has rather hard feelings for any language but Python.

There seem only 2 things people complain about Python, performance and the two major versions out there.

Overall I have the feeling most devs thing it's pretty okay.

andrewmcwatters 4 ago 1 reply      
As much as I love Node.js, I also love this post. It screams the ever-repeated viewpoints of the experienced and tired developer.

There isn't enough of this on HN.

tgarma1234 4 ago 1 reply      
Well I am by no means a great computer programmer or anything so I can certain accept other people saying node is crap but as someone who has used node my reasoning was simple: if you are going to use javascript in the browser and pretty much everything about the "app" depends on what happens in the browser then why use some other language on the server for models and controllers with javascript in the views? Why not just make it javascript all the way down?

Sure, smarter people than me might know why I shouldn't use javascript for everything but for me I do not find it useful to try to figure out how to do so much work in Rails or Django AND THEN ALSO end up using javascript in the views every time anyway. I would rather skip using Rails or Django and just get to the fun part of figuring out what happens next when this button is clicked on the website. I don't think I am alone in that. Rails and Django were a means to an end but they set the bar too high in terms of "programming" and require a lot of programming knowledge that I don't actually need to know to do what I want to do now with node. Sure, my code might suck but unless you are working in some super professional environment with highly skilled managers, pretty much everyone's code sucks. It's about getting the functionality you want with the least amount of effort. Node makes that possible.

aaronpk 4 ago 4 replies      
Great line: "The only thing that matters in software is the experience of the user."
Anagmate 3 ago 0 replies      
I feel like a large part of the negativity around Node.js comes from the callback hell and almost everything being async.

Solution to these two problems (an ingenious one I think) is async/await (and there is an awesome polyfill on npm) - if you want, you can enclose your whole code in one async block and effectively make everything synchronous. Or you use it to its full potential and remain synchronous in all parts of the code that need it while retaining the benefits of async (other parts of code can run while the current one is waiting for a result.

The only caveat I've encourtered (a manageable one, at least for me) is that you must keep in mind that the underlying code IS async, so you can only guarantee synchronous behaviour in the single asynced block od code - so sometimes, race conditions could happen between some parts of your program, but with proper scoping (and ideally, using pure functions), you can avoid it all together.

nilved 4 ago 0 replies      
I try to understand why people would write JavaScript but I just can't. The only explanation to me is that they're not aware of what they're missing out on, or the deficiencies of their own platform, in a Dunning-Kruger sort of way.

I would be interested in hearing from some JavaScript developers about why they use JavaScript, but I have probably heard and refuted it before. Especially after Ryan Dahl disavowed Node, I think it's time to reconsider your viewpoint.

tim9009 4 ago 0 replies      
"The only thing that matters in software is the experience of the user.""if you add unnecessary hierarchies in your code directories, if you are doing anything beyond just solving the problem - you dont understand how fucked the whole thing is."

Oh really? So making the code easily maintainable and easy to comprehend is just a waste of time and simply ads complexity? Comments like that makes this hard to take seriously.

kabes 4 ago 0 replies      
The article is not very relevant in 2016. Much of the arguments are not valid anymore.
mcs_ 4 ago 0 replies      
There will come a point where the accumulated complexity of our existing systems is greater than the complexity of creating a new one. When that happens all of this shit will be trashed.

I think it applies to bank system, politics, and more in general to anything that can be improved and where there are enough money to think about re-design. At that point yes, another life cycle will starts.

aaronbrethorst 4 ago 1 reply      
Quasi-related, one of my favorite programming language essays of all time: https://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/

From what I've observed, I feel like Node is where hipsters who decided that PHP was "icky" ended up.

everydaypanos 4 ago 1 reply      
Node.js solves the problem of "live webpages" w/ Websockets and Polling and chat apps and notifications etcetc.

And that counts. A lot.

lllorddino 4 ago 0 replies      
> better alternatives around with much more sound models and environments, Erlang and Go

Node.js makes it extremely fast to build robust and fully featured web apps quickly. I'm using Go currently for the backend and it's a pain since everything has to be written from scratch. I still enjoy it since the language is simple.

punnerud 4 ago 1 reply      
NASA is using Node.js for its Mission Control Frameworkhttps://nasa.github.io/openmct/https://github.com/nasa/openmct
tflinton 4 ago 0 replies      
I've done a fair amount of messing around with javascript.

1. Recompiled webkit into javascript using emscripten/asm.js (webkit.js)2. Created a desktop version of node (tint2) using FFI calls for OSX/Windows3. Reconstructed much of the oracle o5login/tns/tti protocol for giggles (and if you want to see a very head scratching way of packing 64 bit signed ints, look no further than 11g).

What I can say is it isn't perfect. But it has one gigantic advantage is you don't need to build or compile to run, but still get nearly the same performance from compile time languages.

It's a near perfect prototyping language, and for most peoples use (e.g., 99% of crud/general business humdrum logic) it's perfectly fine.

Just stay away from relying on too many npm modules.

rv11 4 ago 0 replies      
I see a lot of desktop apps made from electron/CEF using Node. examples are Brackets and Steam.
api 4 ago 0 replies      
Meh. It works. Npm is okay if you exercise quality control over what packages you use and keep dependencies down. The language is decently productive for scripting and everyone knows it so it's easy to find maintainers. It performs well and can be multithreaded just fine by running many concurrent node workers and using a good database and cache.

Go may be better but has a less rich ecosystem of turn key solutions and easy integrations. Java is a decent language but has an outdated ecosystem. .NET going OSS is interesting.

Ultimately what you do with languages and tools is more important than which ones you use.

_c_ 3 ago 0 replies      
From Rob Pike quote at the end, after the Ryan Dahl quote:"... the UNIX/POSIX/Linux systems of today are messier, clumsier and more complex than the systems the original UNIX was designed to replace."

"Messier, clumsier and more complex" are adjectives that could describe almost all of today's software vis a vis software from the 1970's. This is not a criticism of today's software it is just the evolution (or devolution) as it happened, an objective observation.

By and large, programmers do not attempt to make software more clean, more efficient or less complex. Most do not spen all their time cleaning up messy code, fixing bugs, or sacraficing inefficiency at the expense of usability or abundance of "features". And almost none spend time removing code and reducing complexity.

They do the opposite: add features, pursue "ease of use" at the expense of common sense and incessantly generate and commit code believing that any decline in commits or "software updates" signals a project is "dead", not "modern" and probably in need of a "replacement". Again, I'm not critiquing this, I am just stating the facts. This is what they do.

Not sure about Pike, but the reason I think some older software is higher quality than most newer software is not because it was or is high quality in an objective sense. It's because today's software is so low in quality and in too many cases worse than yesterday's. Indeed, it's "messier, clumsier and more complex."

In an objective sense, 1970's UNIX is nothing to celebrate. But compared to the then-alternatives, what came afterwards, and what we use today, it can be held in high regard. It's only good in a relative sense. Everything else was and still is so bad. (Why is anyone's guess.)

Avoiding the hassles Dahl alludes to[1] brings me a certain feeling of satisfaction. My language of choice is Bourne shell. And if I am just working with text, such as reading and writing, I do not use a graphics layer - no X11.

The question is: Does Pike's comment apply to Plan 9?

1. Not to mention avoiding the needless complexity and obscurity of Microsoft Windows.

diegoperini 4 ago 2 replies      
Facts are nouns, adjectives are subjective. A subjective analysis is hard to trust. I hate Java, because I choose to do so. Java being a bad language cannot be proven by my personal opinion. People do amazing things with it, Minecraft was initially Java, Android apps run on JVM etc etc.

This article has more than 10 adjectives (many have adverbs attached to them or written like "... much more sane and reasonable language like Python ...") in its first five sentences. Thanks for the effort but no, I cannot trust any of it.

clifanatic 4 ago 0 replies      
Well... all the things like DBus and /usr/lib and Boost and ioctls and SMF and signals and volatile variables and prototypal inheritance and C99FEATURES_ and dpkg and autoconf that he's complaining about were originally conceived as a way to simplify something that their respective authors considered too complex in the first place. I'm not holding my breath that the next attempt to simplify all of those doesn't just add yet more complexity.

On the plus side, job security.

cdevs 4 ago 1 reply      
I used "callbacks" when needed in objective-C in my app days and got sh#t done just fine. I think the best thing to come from node is the ability for a younger crowd to jump into proton/electron desktop apps without needing to get into sdl, qt, or some platform specific language like objective c or the .net C# whatever family. Nose might not be the finally language to take us to that area but it's a start and a ton of awesome open source apps came out of the scene quickly.
Roboprog 3 ago 0 replies      
Async I/O -

CON: I hate having to set up callbacks (even with promise syntactic sugar), whether I need them or not. But of course I do need them when I am coding a U/I.

PRO: the "promise-all" trick is pretty useful if you have several queries that all need to be completed before you can continue with their results. I have some Java code that might benefit from this since it does several queries to generate a PDF. (then again, caching might make this moot - YMMV)

arisAlexis 4 ago 1 reply      
This whole thread would not exist when async await gets traction
supergetting 4 ago 0 replies      
I embrace change in life in general, such a view translates over to the complex ecosystems in technologies, I embrace and accept the complexity of every single technology, honestly it is quite amazing. It's good that we try abstracting away many things and make them more simple and less complex, but sometimes inevitably leads to more complexity and more shit, and I like it and it's fun to be part of such journey!
progrocks9 4 ago 0 replies      
NodeJS works great as a simple web or network container, if you're trying to create a complex enterprise app in Node, basically you're doing it wrong. Simply it's difficult to design complex nested logic or algorithms in a clean manner. It's possible but it's harder than other languages.
annasaru 4 ago 0 replies      
J2EE is up there.. We have a generation of corporate Java programmers who love boilerplate code and obfuscation.

High-School teachers inflicting Java on teens - cringe-worthy. Many public schools only offered Java if you elected programming/CS (in High School). Only recently has Python made inroads.

yAnonymous 4 ago 0 replies      
Yeah. I hate all these great text editors and other cross-platform software that Node brought about.
SimeVidas 4 ago 0 replies      
4 years old?
tim9009 4 ago 0 replies      
"The only thing that matters in software is the experience of the user.""if you add unnecessary hierarchies in your code directories, if you are doing anything beyond just solving the problem - you dont understand how fucked the whole thing is."

Oh really? So making the code easily maintainable and easy to understand is just a waste of time? Comments like that makes it hard to take the rest of what he says seriously.

andrewstuart 4 ago 0 replies      
I rather like JavaScript. ES2015 anyway.
Jean-Philipe 4 ago 1 reply      
Uriel didn't quite explain why PHP is any better than nodejs. I'd be even interested to hear!
narendraj9 4 ago 0 replies      
Cannot access the site.
neelkadia 4 ago 0 replies      
Can you make it good?
phazelift 4 ago 0 replies      
Unbelievable, so many up votes for this complete idiot statement..
Wei-1 4 ago 0 replies      
cannot agree more, but we still use it.
ComodoHacker 4 ago 0 replies      

>I hate almost all software.

YeGoblynQueenne 4 ago 7 replies      
So, this is totally an ad hominem but the OP webpage has a post category titled "Political Correctness" [1] with one post titled "Claims that sexisim drives girls away from Computer Science are feminist bullshit." [2] and one link to an external site titled "Sex Differences in Mathematical Aptitude - By La Griffe du Lion." (a.k.a. "Lion's Claw") [3] starting off with this bit, styled like a research paper abstract (you know, to make it look seintifikal):

Mathematics is a man's game. A gender gap appears early in life, blossoms with the onset of puberty and reaches full bloom by mid-adolescence. It indelibly shapes women's prospects for doing significant mathematics. In this account of cognitive sex differences, Prodigy shows how sex-differentiated ability in 15 year-olds accounts for the exiguous female representation at the highest levels of mathematical research. A female Fields Medalist is predicted to surface once every 103 years.

I stress again that this is essentially an ad homimem: I'm absolutely flagging up this dude as a sexist baboon with brains the size of a small, frozen pea. And because I have much better things to do with my poor feminine brain (say, finish off my MSc dissertation, on a novel grammar induction algorithm) I'm not reading a single word of the OP.


[1] http://harmful.cat-v.org/political-correctness/

[2] http://harmful.cat-v.org/political-correctness/girls-in-CS

[3] http://www.lagriffedulion.f2s.com/math.htm

dang 4 ago 1 reply      
This post is baity but the thread turned out to be rather good, so let's try turning off the flags on it. If it ends up a flamewar we'll have to put them back on, so if you comment here, please keep the discussion thoughtful!

Btw, the top bit of the article is an HN comment by uriel from 2012: https://news.ycombinator.com/item?id=4495305, and the other part had a major thread in 2011: https://news.ycombinator.com/item?id=3055154.

sownkun 4 ago 0 replies      
bit of an overstatement if you ask me
greendestiny_re 4 ago 0 replies      
I'd just like to point out the linguistic paradox of the "one of the worst" syntagm, as only one thing can be the worst, and "one of" implies a multitude of such things; if 20 things are the worst, then none of them are. In short, this is a lazy way for the writer to express his opinion but label it as a fact.
myf01d 4 ago 2 replies      
I am interested. What is Node.js people opinions about Django in 2016? I am a newbie and I chose Django for a somewhat serious personal project. Is it really bad to use django for a new project in 2016?
Browsing your website does not mean I want your spam medium.com
615 points by davezatch  4 ago   239 comments top 53
trjordan 4 ago 11 replies      
> This transaction breaks a core promise using the internet: just because I visit a website doesnt mean I consent to getting spam from it.

No it doesn't. There is no core privacy premise of the internet, and certainly not one that everybody used it signed up for.

I'm not condoning this behavior, but we're in territory that we don't have prior art for. It used to be totally fine for one shopkeeper to mention to another that he saw a customer looking for a particular item. When you do it at scale, the old rules don't apply.

If you think it's spam, hit the spam button in gmail and get rid of it. Use an adblocker. Talk to your congressman about data privacy and sharing laws, because we don't have anything that's effective. Frankly, continue to write Medium posts, because it raises awareness :) But, I disagree with the notion that this is a solved problem with bad actors, because we're in unknown waters.

davb 4 ago 6 replies      
I once had something similar, if not worse, happen.

I was researching some network equipment, looking at lots of websites and comparing products.

Then my desk phone rings. A call being passed from the switchboard - someone asking for the person responsible for IT purchasing.

It was a sales rep from a network equipment distributor, saying they noticed I was browsing their website and wanted to help me through the purchasing process.

I had never used their website in the past. No-one from my company had. I never signed up. I didn't login. I was bewildered.

I asked how they got my details. The rep said they pay a third party remarketing agency for contact details of people who visit their website.

We were a really small company, with no DNS PTR on our main (NAT'd) public IP. We did have an A-record for our mail domain pointing to this IP.

As the sales rep didn't know my name, all I can assume is that their remarketing agency was looking up our public IP addresses in some IP-to-business database, populated by email headers or sign ups at other user sites.

In any case, I wasn't pleased and was pretty surprised at the rather aggressive sales technique.

JohnTHaller 4 ago 3 replies      
I've been getting more spam lately from "legitimate" companies. One of my email addresses leaked from a major open source project I corresponded with. Harvesters found it and now sell it to every small business and entrepreneur marketer you can think of. I get spam from CDNs, off-shoring companies, SEO/SEM, marketing, you name it.

Lots of them use sketchy services like reply.io to make it seem like a real person sent the email. And then another that looks like a reply to the first when you don't respond. And then another and another. Like Katie Malone at HawkSEM.com who 'personally' spammed me another 'reply' today. Essentially, folks like reply.io and similar automate the process of repeat spamming. Even their tag line is "Send Cold Emails That Feel Warm".

Here's a reality check for you: sending "cold emails" to a list of email addresses you bought makes you a spammer. Even if you try to make them appear personal. The giveaway is the tracking image (usually hidden or 1px by 1px white of course) and tracking links in every email so they can track whether you opened it and whether you clicked anything along with the unsubscribe link at the bottom. Except they don't label it as unsubscribe. It says "If you don't want to get any more emails from me, just let me know." with 'just let me know' as a link.

Be sure to mark every email like this you receive as spam so you don't get any more and so their reputation decreases enough to route all of this spam to everyone's spam folders.

r1ch 4 ago 3 replies      
I've had several companies ("data partners" they call themselves) approach us to add these scripts to our websites. All of the ones I've seen use MD5(email) for the "anonymous hashing". I mentioned our privacy policy doesn't allow us to give out user emails, and their marketing guys never seem to understand that MD5(email) is basically the same thing. I even made a video example https://www.youtube.com/watch?v=ViCjzJpEaJw that failed to convince them.
cyberferret 4 ago 4 replies      
I am really beginning to hate browsing the web these days... Especially poop up dialogs asking for my email as soon as the mouse cursor leaves the active browser screen. With an average of 20 browser tabs open, while one is loading I often go to click on another to check on something, and this instantly triggers a flurry of popups begging me to stay/subscribe.

Also the retargeted ads that follow me everywhere now. MOST of them are for companies where I have ALREADY bought something, so they are wasting their ad spend on chasing an existing customer, not a likely prospect.

This has made me resolve to try and make the web a less shitty place, one web site at a time - and I have ensured that my web projects absolutely DO NOT have any popups or cross site tracking in there (aside from normal analytics that is only used in house).

[I accidentally mis-typed 'pop up' above but LOVE the Freudian slip so will leave it as-is].

arnaudlaudwein 4 ago 1 reply      
This is legal[1] in Europe if you consented to receive marketing emails from "partners" of a website you subscribed to (through an opt-in, not an opt-out checkbox).

You subscribe to website X, you opt-in to offers from third-parties, and this allows X to share your e-mail address with Criteo. Then Criteo sends you marketing e-mails for the account of Sears (but they surely don't share any PII with Sears - the e-mail is sent by Criteo).

The logic isn't that "browsing Sears is considered as having a preexisting business relationship with them". It's because users opted-in to third-party communications from a website they may have signed up with, back in 2008.

Other similar use cases include sending you an e-mail for website X when you browse website Y because they know you are in front of a computer/phone and this increases chances of opening e-mails.

Doesn't make it more or less "right" though and it's surely very surprising for users, myself included.

(On a tangent, what still looks like a legal gray area to me are the Data Management Platforms (DMP) - everyone shares user data in a big bucket/database provided by a common partner, all users are identified with IDs but not directly with PII, how much data can companies push/pull legally?)

[1] Not a lawyer but worked with legal teams on these topics. Laws still differ slightly depending on the European country you're talking about, but the GDPR will soon be unifying data privacy regulations. Right now the French and German Data Privacy regulations are some of the most restrictive ones.

kazinator 4 ago 3 replies      
I wrote myself a web application called Tamarind that runs on my web server for managing throwaway mail aliases.

"Tamarind" == "Throw-Away Mail Alias Randomization Is Not Defeatable"



I log in with my IMAP4 user name and password, and then get a simple UI with a table of my aliases, and attached memo strings (which can contain URL's that get converted to links). I can edit these, change their order (select multiple, move to top or bottom, etc) create new ones and delete. When I create an alias, it goes "live" instantly, and when I delete one, it goes dead. Dead means that the address is "unroutable" at the SMTP level; it bounces.

I keep a few aliases from Tamarind in my wallet, in case I have to hand out an e-mail address in "3D life" to some untrustworthy outfit to be eligible for some promo or whatever.

gwbas1c 4 ago 3 replies      
This is why I own my own domain and have a catch-all email address. When I give a company my email address, I use (companyname)@domain.com.

They all forward to gmail; where it is very easy to filter out (companyname)@domain.com once shenanigans like this happen. It's also easy to track down and shame companies for doing this, too.

chime 4 ago 5 replies      
I feel less paranoid now for my browsing process. Almost everything I search is in an incognito window, from shopping and research to programming and how-tos. And when I'm done with looking for a new dog leash or Python module, I close that window. Only things in my main browser are the regular sites I visit and am logged into (email, HN, reddit etc.)

I started this after learning about the filter bubble but I've noticed how helpful it is when searching on Amazon, Wayfair, or Sears. I get non-machine-learned results every time while my wife using her primary browser with cookies often cannot see the same results I do. If I find something on Amazon, I copy-paste the URL without the ?query-string and replace 'www' with 'smile'. It seems like a hassle but it's no different from cleaning your feet before stepping inside the house after playing in the park.

This post just highlights that my practice to avoid unpermitted-profile-building-and-linking is for a good reason. I also have my own @example.com domain that I use and have certainly caught companies selling my info. However, even without being emailed, I don't want algorithms the determine what is best for me based on criteria I choose not to share.

idlewords 4 ago 1 reply      
It's a little rich to write this complaint on Medium, a site that has been uniquely aggressive about tracking its readers' behavior (it has a script that phones home with your position on the page, and its URLs abuse the fragment identifier to track who you got the link from).

If you dislike surveillance capitalism enough to write an essay about it, think about where you're publishing it.

FussyZeus 4 ago 1 reply      
I just love the amazing "Terms of Service" that all of these ad companies have, letting you know that by virtue of loading an HTML page you've consented to have your personal information of ANY caliber spread all over their ad network, their "partners" networks, and to anyone else with a buck and a server, and immediately absolve themselves of any responsibility for what that might mean in terms of information falling into the wrong hands.

I can't think of another business that has this kind of insane amount of easy-to-start interaction that results in so much activity and yet can claim zero culpability for any consequences. It's as if you purchased an airline ticket and the ticket came with a 17 page document attached where they spell out that by flying on this aircraft you agree to have tickets pre-planned in your name for 24 other flights, the plane may or may not make a stop off in 6 airports en route to your destination, the pilot occasionally likes to do barrel rolls and loops but he's real good at it so don't worry, and by the way occasionally the engines fall off but you don't get to sue us if anything goes wrong. ENJOY YOUR FLIGHT

Animats 4 ago 1 reply      
You still have third-party cookies enabled?

Go to Options in Firefox under Privacy, and set "Accept Third Party Cookies" to "Never".

gwbas1c 4 ago 2 replies      
This is why gmail has a big fat "REPORT SPAM" button. Shenanigans like this are SPAM, and should be reported accordingly.
sklivvz1971 4 ago 1 reply      
I just mark all this stuff as spam, including stuff from legit companies that might have tricked me into subscribing to some list.

The thing is, I am never, ever interested in receiving marketing emails. Every single time, without doubt, I opt out of marketing emails. So if I receive one it means that one of these things holds true:

1. It's just spam

2. The website used some dark pattern to trick me into subscribing to something I did not want to

3. The website assumed consent and didn't bother asking

Guess what -- I'm perfectly fine burning all of this crap with a spam filter. It's a waste of time, and time is my most precious asset.

tempestn 4 ago 0 replies      
> Only when we craft the email on behalf of our advertisers, we receive your name, surname and email address from our partners, should you have consented to receive their emails marketing.

> Lets ignore the fact that they assume Sears had my consent (they didnt).

Just a note: I think what Criteo is saying here is that you gave permission to some third party to use your email for marketing purposes and to share it with their "partners", not that you gave Sears permission to use it. But they shared it with Criteo and Criteo shared it with Sears (or sent the email on their behalf) so technically there is "consent". (Of course in practice it's often possible to supposedly give such consent without ever realizing what you're opting into.)

jdavis703 4 ago 3 replies      
This is why I have the username part of my email address tailored to each site/service I register with. So I have a hackernews@example.org, amazon@example.org, etc. Human beings get my real email though (because it would be weird if I told John Smith to email me at johnsmith@example.org). If people start abusing this (politicians do this a lot), I can just block say timkaine@example.org, and never hear from them or people they've sold and traded my email to.
r721 4 ago 2 replies      
This is the reason I keep "Block third-party cookies and site data" option checked in Chrome.
rootlocus 4 ago 1 reply      
I was thinking whether or not sending these emails actually helps companies like Sears by bringing in customers, and whether or not (to an extreme) they might depend on them to survive as a profitable enterprise. What came to me as a revelation is that it's irrelevant. If their income relies on bothering everyone who comes across their website, tricking them into clickbaits or spamming them with (possibly malicious) ads, it might mean their services are not enough to justify their existence. As such, I decide not to pity them, and happily continue loving my adblocker.
joesmo 4 ago 1 reply      
I'm not going to wait for legislation to fix problems I can fix myself. You don't want this to happen? Make sure you have ad-blocking and third party tracker blocking on. I go a step further and use 'Quick JS Switcher' for chrome. By default JS is off and I only turn it on for sites I want. The percentage of sites that I turn it on for is minuscule. I'm seriously starting to question why this isn't the default setup for any freshly downloaded browser.
rapht 4 ago 0 replies      
I personally switched to the following policy a year or two ago to avoid all this crap:1) NoScript extension filtering everything except the base domain => no third party scripts are allowed except when I explicitly allow them2) Cookie Whitelist extension to allow cookies only from domains I choose, only when I need => no third party cookies allowed, ever3) Block incase the webpage tries to load iframe ads4) a unique email address per service (like amazon.[5 random chars]@mydomain.com) so if all else fail and your address gets in the hands of somebody who should not have it, you know where it came from and can expose them
sandworm101 4 ago 0 replies      
>>> But until legislation catches up to regulating the negative consequences of retargeting, there may not be much you can do about this besides blocking cookies, ads, and opting out of Criteos entire system by submitting your email address here.

No no no. Handing over your email address to an online advertiser is a horrible idea. Do not engage them. Blacklist their content, their cookies, via whatever means you want (I use adblock) and be done with them.

An article that discusses tracking via online advertising but doesnt discuss blocking is very suspicious. The most powerful tool against the problem isn't worth even a mention?

davidgerard 4 ago 0 replies      
"Dear Criteo: You opted in to this box of dead rats we just sent you because you once visited a site that partners with our dead rat promotion service."
TeMPOraL 4 ago 0 replies      
Yesterday, after many years, my curiosity finally got better of me - I started playing World of Warcraft. Since my head is now full of thoughts about MMO, excuse me for saying this:

There should be a new class - or race - added to fantasy worlds. The Marketers. More evil than demons, undeader than the Lich King. Their gameplay mechanics would be based around earning gold by draining their own souls, as well as the souls of characters around them. Their primary combat role would be casting annoying debuff spells at everyone around, friend and foe alike.

Seriously though, this article basically says that someone out there has reached another level in insidiousness. If it was an MMO, we could at least form a raiding party and get rid of the problem once and for all.

dnh44 4 ago 1 reply      
jamiesonbecker 4 ago 1 reply      
tl;dr: related: Amazon sold (or gave) my secret Amazon email address to third parties without my express consent rather than using their remailers.

I have exactly one email address that I use for Amazon, and I've never used it elsewhere for anything else.

I occasionally receive emails from vendors (through the vendors' mail servers themselves, not remailed through Amazon per mail headers) at Amazon that I have bought things from (via one-click) as gifts and I am 100% sure I never gave them my email address or replied to any email from them.

An example vendor is a large outdoor clothing store that I bought a North Face jacket for a relative from. I'm now on their mailing list. In the ultimate irony, I could just click unsubscribe but it's actually good stuff ;)

Thanks, Amazon.

jimsug 4 ago 0 replies      
And this is why I use uMatrix, despite the little bits of extra hassle I go through when visiting sites for the first time.
nashashmi 4 ago 0 replies      
I don't understand why everybody does not block third-party cookies by default. I took a stab at my cookie list and found 300 cookies from advertisers and intel gatherers. I deleted them selectively, but I did not want go through that again, so I blocked the third-party ones.

Some have been explicitly aloud because I trust them, like google analytics. But other google cookies are prohibited, like plus.google.com. Facebook is explicitly blocked. Doubleclick is blocked. some websites will not work if certain third parties are blocked, so i have to explicitly allow them once i realize the problem.

loup-vaillant 4 ago 0 replies      
How fitting. I have just received a mail from Medium with no "unsubscribe" button because I commented on it.
dredmorbius 4 ago 0 replies      
I've mentioned putting the Winhelp2002 hosts file on my dd-wrt router a few times. I just checked to see if I need to add any specific entries.

 root@router:/tmp# grep criteo hosts0 cas.criteo.com dis.criteo.com dis.eu.criteo.com dis.ny.us.criteo.com dis.sv.us.criteo.com dis.us.criteo.com ld2.criteo.com rta.criteo.com rtax.criteo.com sapatoru.widget.criteo.com sslwidget.criteo.com static.criteo.net static.eu.criteo.net widget.criteo.com www.criteo.com
Apparently not.

Deets: https://ello.co/dredmorbius/post/v9l7zvlyynvl1pskbwssmq


Jaruzel 4 ago 0 replies      
From the Article:

 I am signed up to some platform which is a Criteo partner. Its entirely unclear who this partner is. While Criteo boasts a close partnership with Facebook, Facebook claims that they do not share personally identifying information such as your email address with ad partners. Regardless, a platform with my email address gave it to Criteo.
This issue is exactly why I use specific email addresses for each website. I tend to follow the pattern <websitename>@mydomain.com. That way if a site leaks my email address to spammers (either intentionally or accidently) I know which site it was, and immediately boycott them in future and move that email address into a blacklist.

For big sites I cannot boycott, I simply register a new email address with them (i.e. <website><number>@mydomain.com), and move the original into the blacklist.

As I run my own on-premises email system, I can't benefit from crowd-managed spam systems, so keeping a lid on the incoming spam is very much a pro-active action for me.

upofadown 4 ago 0 replies      
>The CAN SPAM act actually allows direct marketing email messages to be sent to anyone, without permission, until the recipient explicitly requests that they cease (opt-out).

Isn't this the root problem here? It is hard to see how you could even start to fix this sort of thing without fixing the spam law first.

cptskippy 4 ago 0 replies      
I've been using a catch-all email domain for years where anytime I give out an email address, the local part is a description of the party receiving the address (e.g. bestbuy.com@mydomain.com).

If I receive spam at a particular address, it's easily blocked and I know who leaked it.

An interesting side effect of receiving email from so many different addresses to the same inbox is that I often receive the same spam to multiple addresses simultaneously. This is easily caught by spam filters and so I never have Spam in my inbox. It also makes identifying false positives in my Spam box easy because they usually stand out against the repeated subject lines so it's a simple game of which one of these is not like the others.

DoubleGlazing 4 ago 1 reply      
My wife's cousin had something like this happen to her two years ago when she was planning her wedding.

She browsed a few specialist wedding sites for inspiration and when she went to to some well known retail sites to start pricing things they seemed to know she was getting married and promoted wedding goods and services on their front page to her.

It freaked her out no end. I suggested a few plugins that seemed to put a stop to it. But a few weeks later she did start getting wedding related snail mail spam.

Its very creepy, especially after the whole Target teenage pregnancy thing.

bogomipz 3 ago 0 replies      
On a somewhat related note and what I thought the article was going to be about, what is going on with the phenomenon of a HTML 5 light boxes loading when you are barely a few seconds into reading a page asking you to "sign up for the newsletter." This trend is out of control. If you were browsing shelves in a grocery and someone came and stood between you and the book you would want to punch them.

Does annoying people into something actually work? I feel like it must since its so prolific.

I wish there was a way to block these.

sneak 4 ago 0 replies      
uBlock origin plugin. Globally disable 3p resources for all pages. Manually greylist CDNs only for sites.

Browsing the web any other way is for schnooks.

andrewaylett 4 ago 0 replies      
Privacy Badger is pretty good at blocking things like this -- it watches out for domains that are third-party for more than one site, and blocks requests to them. Does require some tweaking for genuine CDNs (and indeed comes with a yellow-list of common domains that will receive requests but not cookies) but generally very useful.


chadgeidel 4 ago 1 reply      
Am I paranoid in assuming their "opt out" system is basically probably an "opt in"?

Related: I know that it's possible to "opt out" via the Direct Marketing Association communications (https://dmachoice.thedma.org/), but have thus far not done this as I assume I'll just get more junk mail.

anilgulecha 4 ago 0 replies      
More directly - if you want your precious content/resources to make you money, make sure you send the bits over an authenticated & paid account.

Not behind an overlay, or with a adblock redirector or when the user-agent has 'googlebot' in it.

If you send the bits over, then I may consume them with no additional payment, whether via ads or mailing-list or account signups.

justrossthings 4 ago 0 replies      
I talk a lot about this stuff with a friend doing sales operations at a hyper-growth startup in SF. With Criteo, tools like Reply.io and others he thinks we're going to see an event horizon where recipients of spam say enough is enough and online privacy finally becomes 'cool'.
a_imho 4 ago 0 replies      
I know there are a couple of solutions out there, but what exactly stopping the main email providers to offer on demand proxy addresses for one's main account? I think there is a legitimate demand for it, but not enough to actually sign up for yet another service.
__jal 4 ago 1 reply      
I think it is about time to build a one-click opt-out to preemptively opt-out of all of these scumbags' systems.
singold 4 ago 0 replies      
He talks about tge legality of sending the spam, but what about the legality of the partner he is really subscribed to that shared his information with a 3rd party? IANAL but AFAIK that wouldn't be legal in most countries
DavideNL 4 ago 0 replies      
what i usually do is reply to their spam e-mail on a support mail address and ask them to stop sending me spam: waste their time the same way they waste my time... if everyone would do that the problem would be solved.
walrus01 4 ago 0 replies      
... and I just created a new spamassassin rule for criteo. Done and done.
imron 4 ago 0 replies      
And people wonder why adblockers are so popular...
ChuckMcM 4 ago 0 replies      
Set privacy badger to block all Criteo cookies.
ahm750 4 ago 0 replies      
Faced a similar situation recently. It was both surprising and frustrating.
astdb 4 ago 1 reply      
Would disabling third party cookies have prevented this?
LoSboccacc 4 ago 1 reply      
4 days ago 4 ago 3 replies      
not_a_codfish 4 ago 2 replies      
saltyhiker 4 ago 1 reply      
The real problem here is not the chain of marketing tech that allowed this, the issue is that the marketing message itself sucked. If the message was valuable, many people wouldn't have been bothered by receiving it.

As for the message itself, if their intent is to sell you that specific item you searched for, they should say so. Of course, they need to avoid the creepy-factor, which, along with laziness are the two reasons they may have ended up with the junky message you received.

malchow 4 ago 1 reply      
Please. There is no core privacy premise of the internet. The core premise of the internet is one protocol to deliver meshed knowledge to any computer. And the commercial possibilities of the internet are what have underwritten the growth of the network.

Reaction like this one make me think: entitled.

But they also make me think: unrealistic. How much should hypertargeted ads really bother us? Call me when they are using my bank account and medical records to show me ads. Not my browsing history, over whose exposure I have complete control, and which doesn't really expose very much about me or my family.

How do I choose not to share my account information with Facebook? whatsapp.com
524 points by SnaKeZ  1 ago   303 comments top 42
dang 1 ago 0 replies      
There are three active threads about this, the present one and these other two:

"WhatsApp is going to share your phone number with Facebook": https://news.ycombinator.com/item?id=12358751

"Looking ahead for WhatsApp": https://news.ycombinator.com/item?id=12358205

Normally we'd merge the threads, but the discussions are large and come from three different perspectives, so that might not be best. HN practice is to have one front-page thread at a time about a story, but if you're concerned about the topic you might want to check in on the other discussions.

kristianc 1 ago 8 replies      
This is an incredibly ugly dark pattern.

The 'share information with Facebook' nugget is hidden behind a toggle at the bottom of the screen, and will be guaranteed to be missed by the 99% of users who just want to talk to their friends.

Then, once you've agreed to the terms and conditions, you've got a completely arbitrary 30 days to read an online article which tells you what you've signed up for before WhatsApp is irrevocably sharing your data with Facebook.

You can build an incredibly accurate picture of people's lives from metadata alone - WhatsApp know it and Facebook know it.

Not only that - when WhatsApp start building out these 'brand' relationships which will look a lot like helpful information at first - you'll be loading your data into that brand's custom FB audience too. And you won't have an opt out because, y'know, reasons.

This is very obviously not the WhatsApp that promised not to fuck with its users when Facebook bought it out.

AceJohnny2 1 ago 5 replies      
Man, remember the late 90s and early 2000s, when the IM scene was a mess of incompatible networks like ICQ and AIM and MSN Messenger, and people were like "fuck this" and came up with this protocol that was interoperable with all the others through gateways and also it was extendable, and we could all be happy together?...


The worst part is that WhatsApp is actually based on Jabber >:(

Cozumel 1 ago 2 replies      
It's kind of pointless, Facebook owns Whatsapp so they have all that info anyway.

'The Facebook family of companies will still receive and use this information for other purposes such as improving infrastructure and delivery systems, understanding how our services or theirs are used, securing systems, and fighting spam, abuse, or infringement activities.'

skrowl 1 ago 9 replies      
Or... you know... just use something like Telegram that isn't owned by Facebook. Their data-sharing / privacy policy is pretty simple https://telegram.org/privacy

1. Sharing data

We never share your data with anyone. No.

nicolas_t 1 ago 5 replies      
My problem with whatsapp is that there's no way to use it without giving access to my entire contact list to the app. And that's out of the question for me.
Aissen 1 ago 0 replies      
The Facebook family of companies will still receive and use this information for other purposes such as improving infrastructure and delivery systems, understanding how our services or theirs are used, securing systems, and fighting spam, abuse, or infringement activities.
omginternets 1 ago 4 replies      
I don't have the Share my Account option on Android.
jamisteven 1 ago 0 replies      
Linkability at its best. Been saying for years that FB is trying to help establish the single-sign-on for all to access the internet. Thats why every major email and social outlet are demanding you input your phone number and secondary email alongside your existing account info. Eventually this will help the government know who is behind each IP address. No need to keep avoiding the phone number security prompt at the top of FB anymore, they already have it through their acquisiton of whatsApp, what a shame.
hointytointy 1 ago 0 replies      
How much do you want to bet they will make sharing mandatory in 6-12 months? Opting-out is a fig leaf to head off bad press in the short term.
xanadohnt 1 ago 1 reply      
Are we supposed to congratulate WhatsApp for doing something that should be expected with 1.0? Furthermore, your default is incorrect; sharing info is opt-in.
vthallam 1 ago 1 reply      
So this was quite expected anyway. Though initially the acquisition seemed to be a defensive one where FB prevents Wechat kind of features coming from Whatsapp, the amount of data whatsapp has, is a treasure trove for all the advertisers and as a public company, FB just did what is expected by the shareholders.

But it's kinda scary that emperor Zuck has so much power over the people, like FB/Messenger/Whatsapp/Instagram are the top apps everyone uses. I am glad Snapchat didn't sell out.

CodeMichael 1 ago 2 replies      
I'm trying to find more info about https://wire.com

They would seem to be doing all the right things, such as OTR and not rolling their own protocols, but I've only been able to find a couple of opinions and nothing concrete.

The fact that they've made effort to open source it and are letting people write their own clients for it is encouraging, but not proof that it's a solid system.

nashashmi 1 ago 1 reply      
I am reading the opt out agreement as far too specific. It says do not share info with Facebook for improved ads experience. MEANING SHARE info WITH FACEBOOK anyways, even if you are not interested in ads.

I don't want any of my info shared. Yet there is no way to opt out of it.

plg 1 ago 2 replies      
If only there existed a peer to peer solution that doesn't require trusting a central host
nanospeck 1 ago 0 replies      
I wonder, what are the odds of success if someone starts a paid social network say for 5$/month. Ad-free, no-snopping, secure social network. E.g. Diaspora pod with good UI and more features. Would you be willing to pay for it?
peatmoss 1 ago 0 replies      
I had few enough people who I've communicated with via Whatsapp that I was able to simply delete my account. I hope that means they'll actually purge my data rather than sharing it with Facebook.

This is one of the problems of digital identity--privacy has a latent value. The company you choose to share data with today, may choose choose to share with / merge with a third party in the future.

slantedview 1 ago 0 replies      
Having removed the app a while back but not formally deleted my account, seems like I have to re-install, dig up my account again, then opt out.

What a hassle.

nanospeck 1 ago 1 reply      
I thought an SSL encryption would not let 'anyone' else read your messages. So that means even if whatsapp promises SSL encryptin, they can read our messages? Is it technically possible? Forgive my ignorance.
ffggvv 1 ago 1 reply      
Guys, there's only one solution. Delete your accounts and uninstall the apps. That's it!

Here's the link to delete your account.https://www.facebook.com/help/delete_account

gyosko 1 ago 2 replies      
What if I don't have that option in Whatsapp?Does it mean you can't opt-out after 30 days?
fx85ms 1 ago 0 replies      
This seems like a potentially stupid question, but the link only shows how to do it on an android device. The terms sprung up on me in the iOS app (I have yet to update it) and thankfully I found the hidden toggle button. There does not seem to be an equivalent way to turn off the sharing of data on my iOS device, so if anyone has found a way, please tell me about it.
Zhenya 1 ago 0 replies      
I just got the prompt to accept the new terms.

1) you did not have to scroll to opt out. 2) opting out brought up a toast saying "when you tap 'agree', your account info will be used to improve your Facebook ads and product experiences"3) there was a " X not now" option in the top right corner.

I chose that.

Edit: no app update was required. I'm guessing in the future, the could push ads the same way

StanislavPetrov 1 ago 0 replies      
You can't. According to them, even if you opt out:

>The Facebook family of companies will still receive and use this information for other purposes such as improving infrastructure and delivery systems, understanding how our services or theirs are used, securing systems, and fighting spam, abuse, or infringement activities.

chinathrow 1 ago 1 reply      
So when you remove that tickbox "Share my account info" - your data still gets shared:

"The Facebook family of companies will still receive and use this information for other purposes such as improving infrastructure and delivery systems, understanding how our services or theirs are used, securing systems, and fighting spam, abuse, or infringement activities."

Never does it say, you won't get ads. But if you share, you will get "improved ads".

Am I reading this right?

Qantourisc 1 ago 0 replies      
I cannot wait on Matrix protocol, I just hope it will be used.
tailrecursion 1 ago 0 replies      
If people want to use snazzy services but don't want to pay, and then outlaw collecting private information, presumably the owner of the snazzy service will find some more complicated or secret way to monetize its users?
kevincox 1 ago 0 replies      
It's interesting that Facebook Messenger still doesn't have ads. It even supports using it without a Facebook account (although you do need a phone number and it appears to be slightly limited)
ViProvoft 12 ago 0 replies      
My phone does not show 'share my account info' option...Please can you help me
titomc 1 ago 0 replies      
Ironically, I just whatsapped all my "non-IT" friends on how to "not agree" by finding the hidden data sharing checkbox.
prplhaz4 1 ago 0 replies      
sooo, apparently there's only a limited time period (30d) where you are allowed to opt out of the data sharing...

>> After you agree to our updated Terms of Service and Privacy Policy, you will have an additional 30 days to make this choice by going to Settings > Account > Share my account info in the app. If you do not want your account information shared with Facebook to improve your Facebook ads and products experiences, you can uncheck the box or toggle the control.

harry8 1 ago 0 replies      
I wonder how Moxie feels about being used in what looks to be a pretty spectacular privacy bait-and-switch.
et-al 1 ago 0 replies      
There's already two threads on the front page about the WhatsApp TOS change, do we really need a third when this link is already in one of the top-voted comments?



Duplicate threads like this just dilute the conversation especially when the comments made here are already opinions voiced elsewhere.

Zhenya 1 ago 0 replies      
One question no-one is answering. Will this be pushed in the future, or have we accepted the terms earlier?!
ygra 1 ago 0 replies      
So what does this change mean if you're using WhatsApp while not having a Facebook account?
mbloom1915 1 ago 0 replies      
that's not a serious question right? why you are even using anything Fb touches/owns/operates is the real issue
DavideNL 1 ago 0 replies      
in Dutch we have a word for this, called "poppenkast".

Translated literally it means "puppet-show"...

chinathrow 1 ago 0 replies      
I don't see the data sharing opt-out thingie with FB as the big news today.

The big news is that your eye-balls are officially for sale now. Marketing is specifically mentioned as a new form of communication towards the users. No banner ads (yet), but other formats.

It was good, while it lasted.

PS to WhatsApp: If you manage to launch a no-data-sale privacy option for x$/y, I signup in a heartbeat.

wonkaWonka 1 ago 0 replies      

 The Facebook family of companies will still receive and use this information for other purposes...


amaks 1 ago 1 reply      
Simple, just don't use WhatsApp.
et-al 1 ago 1 reply      
> the discussions are large and come from three different perspectives

The discussions have ballooned, but having read the other two in the morning, this one just feels like a rehash of the same privacy concerns folks have raised. And for anyone reading about possible solutions to their concerns, they'd have to jump between 2-3 different threads now.

I'm glad there's at least WhatsApp thread is on front page for everyone's sake, but personally I felt like the Looking ahead for WhatsApp discussion covered much of what's being said here. The difference is that poster didn't have as good of a title and submitted it when much of the West Coast was still asleep.

Also, while this site doesn't have the resources to comb and diff each new thread to make sure it's not a dupe, I wish more users would point this out when it does happen. At the very least, thanks for pointing out the two existing threads, but we shouldn't be afraid of merging threads to make it easier for future users to reference (if they do).

chad_strategic 1 ago 1 reply      
-->How do I choose not to share my account information with Facebook?

Answer: Don't use Facebook.

Show HN: Carbide A New Programming Environment trycarbide.com
614 points by antimatter15  2 ago   134 comments top 48
antimatter15 2 ago 16 replies      
Hey HN!

I'm one of the creators of Carbide, and I'm really excited to share it with you all.

We're thinking of releasing Carbide as open source in the coming weeks if there's a community interested in building stuff on top of it.

One of the areas we'd appreciate help with is adding support for different languages Python, Scala, Haskell, Rust, etc.

Other than that, general feedback / questions welcome.

We've scrabling to turn off jet-engine mode on the website :)

kbart 2 ago 4 replies      
The page is surprisingly uninformative. It took me good 5 minutes of looking for "download" button before realizing that it's an in-browser IDE. A big "Try it now" button on top of the page would do wonders. A short, one sentence description of what it is would also be helpful, because "new kind of programming environment" reminds something from managers' meeting slides. Anyway, good job and thanks for making it available to us, I was looking for something like this few months back, I'll sure give it a try.
chriswarbo 2 ago 3 replies      
Looks like a cool project, but I can't scroll very far down the site before my browser crashes. I've reproduced this several times, here's the terminal output if it helps:

 $ conkeror https://alpha.trycarbide.com ...... JavaScript strict warning: https://alpha.trycarbide.com/, line 603: SyntaxError: test for equality (==) mistyped as assignment (=)? JavaScript strict warning: https://alpha.trycarbide.com/, line 604: SyntaxError: test for equality (==) mistyped as assignment (=)? JavaScript strict warning: https://alpha.trycarbide.com/, line 605: SyntaxError: test for equality (==) mistyped as assignment (=)? JavaScript strict warning: https://alpha.trycarbide.com/, line 606: SyntaxError: test for equality (==) mistyped as assignment (=)? JavaScript strict warning: https://eponymous-labs.github.io/carbide-splash/static/main.js, line 162: SyntaxError: in strict mode code, functions may be declared only at top level or immediately within another function JavaScript strict warning: https://eponymous-labs.github.io/carbide-splash/static/main.js, line 201: ReferenceError: assignment to undeclared variable diff Console error: [JavaScript Warning: "window.controllers is deprecated. Do not use it for UA detection." {file: "chrome://conkeror/content/window.js" line: 331}] Category: DOM Core Segmentation fault

nostrademons 2 ago 4 replies      
I think that something like this will probably be the future of programming, but Carbide itself needs to dial it back and focus on which data visualizations give the biggest bang for the least obtrusiveness.

Apple's been moving in a similar direction with Swift playgrounds, and recent Java IDEs (IntelliJ, and I think Eclipse) will display the values of variables next to the line of code when you pause in the debugger. These are both useful features. They get cluttered really quickly, though, and in the playground case take you out of your normal development flow.

If you want this to be impactful, focus on delivering information at your fingertips without delivering information overload. The core idea of being able to inspect & manipulate run-time values alongside the code that generates them is sound. The implementation - with lots of fancy gadgets that overshadow the code itself - needs some design love.

gnuvince 2 ago 1 reply      
> Requires no installation or setup

This is not something I want; if a product I use is available only via a web site and the company goes under, the product goes away with it. If it's installed on my hard drive, I can keep using the product.

jc4p 2 ago 1 reply      
This looks like it could be cool. I'm mostly a plaintext editor kind of programmer but a IDE that helps me get my job done better would obviously be the better solution.

There's... a lot happening here though. What does:

> Comments live in a Rich Text sidebar #LiterateProgramming

mean? I played around with some samples and it seems that there's a method for displaying text or something?

I was hoping it'd be an inline `// yeah I know doing +1 looks wrong but it's because` --> automatic transcribing over to a sidebar but I don't know what's actually happening.

What does:

> Imports modules automatically from NPM or GitHub

mean? Does it mean "import" in the sense that you don't have to write the import statement, or in the `npm install` sense? What happens if I misspell a package name and there is a malicious package under that name? Will it auto install and auto run the post install scripts??

wodenokoto 2 ago 2 replies      
Looks interesting, but the website is so ressource intensive that I practically can't scroll down on it. I gave up reading what it is.
prance 2 ago 0 replies      
I was pretty excited about LightTable (which has some similar ideas) when it started out, but then Chris Granger went on to another project and LightTable was left hanging. They open-sourced it, but developer uptake was slow initially and there wasn't much progress. Commits/pull requests seem to be better now (2 years on), but e.g. the blog is still very much inactive, and the last release is more than half a year ago.

So for me, I'd only try yet another IDE if I have enough confidence that it will live long enough. The commitment of the original team is of course a big factor, but as was seen with LightTable and other projects, that can change quickly. So I would require that either - it is commercial and has enough investment/backing,- it is open-sourced and gets enough dev uptake quickly, or- I believe so much in the concept that I'd try it even if the other two points are not satisfied.

Speaking of functionality, however, there isn't any mention of refactoring capacities. To me, that's probably the #1 feature why I'd use an IDE instead of just an editor in the first place. I'd consider the in-place partial debugging display only as a nice add-on, but nothing I would throw out a mature IDE with proper refactoring, search (for definitions, references) and other typical IDE functionality for.

monkmartinez 2 ago 1 reply      
Cool looking site/project... However, the web page kicked the fans on my laptop into jet mode.

Is this like Jupyter or built with Jupyter tech?

rasjani 2 ago 1 reply      
There is, or was IDE with the same name: https://en.m.wikipedia.org/wiki/Carbide.c%2B%2B
NarcolepticFrog 2 ago 1 reply      
I'm curious about how the backwards computations work - how do you detect when the backwards computation can be easily calculated? For example, if we hash some input string and then change the value of the hash, it should be very difficult to do the backwards computation.

Does it literally do gradient descent on the input to try to match the output (as is suggested by the backpropagation terminology?) Can it handle discrete valued outputs?

jomamaxx 2 ago 0 replies      
First - super cool.

Second - I'm at a loss for exactly what it does. Sorry. I get the 1000 mile-high view, and I get some of the super specific examples. But I'm not sure how it applies to most js progamming which is much more mundane than the very specific examples given.

Maybe you could add a section that takes a much more 'basic' introductory approach?

Again, great work.

elliotec 2 ago 3 replies      
But, why?

Like whats the purpose of using this as opposed to vim and a browser?

And what do they mean that it requires no installation or setup? Is it not a native program?

How does one use this to give it a try?

None of this was clear after a few reads through that page.

MadWombat 2 ago 0 replies      
This seems like a really cool idea. Sort of like Jupyter notebooks the way they should be done.

I agree with the sentiment that the website could be a bit more comprehensive. I saw "requires no installation" immediately, but then I started looking for a link to actually try it out and couldn't find it. And only then I realized that you can only try it on example notebooks. Or can I write my own code and I just didn't find the way to do it?

I tried a couple of example notebooks and unfortunately it was painfully slow to do anything. XOR network cell took some 5 seconds to process. Typing code was laggy, sliders took really long time to update etc. The C-Enter shortcut didn't work, I had to click the button with my mouse. So it seems like you have an awesome project, but it still needs some work.

pjmlp 2 ago 0 replies      
Carbide was the name of the latest IDE version that Nokia produced for Symbian.

Have you researched the name?

rajington 2 ago 0 replies      
https://alpha.trycarbide.com/new is the link to try it out. Not sure why it's getting so much hate, the widgets are just icing on the cake, it's like tonic.dev on steroids. You only need to look at the Python community to see how powerful notebooks can be.
ergest 2 ago 2 replies      
Very cool! Seems similar to what Brett Victor demos here: https://vimeo.com/36579366
nikki93 1 ago 0 replies      
This is awesome!

One of the things I've been thinking about lately is what are the languages that lend themselves to "true" livecoding. Like what if you want to change the class hierachy while you're already in a game and the objects are moving around? What if you wanna move a door in the in game editor mid-level but merge this edit back at the start of the level?

Turns out livecoding and live property editing interfaces are an entry into thinking about the next question: what are the abstraction and logic storage models that allow for very radical live coding changes as your app is still running and help you explore?

So far I have liked the idea of prototype inheritance but with the objects stored in a reduxy immutable situation. It seems to get at both the functional and OO side of the "expression problem."

fake-name 2 ago 2 replies      
"A new programming environment" for javascript.

Welp, there goes my interest.

taneq 2 ago 0 replies      
Wow, the back-propagation thing is very cool! Although it would take me a long time to trust something like that enough to use it.

Is there any way to use it offline? (I guess if it's open sourced then at the least, you could run it on a local web server.)

RBerenguel 2 ago 1 reply      
Every time I see a project in this space I wonder why I never got to finish a similar (in spirit) one I had. Multi-language REPL (with "data translation" across languages for specific data types, so you could pass a javascript array to APL, or load a CSV file as a function in APL and send a column as a Javascript array) and built in visualisations for standard types (a matrix could show a heatmap, for instance, vectors barplots or sparklines, specific arrangements of JSON were chord plots).

The goal was just to make a tool where small-scale data munching was easy (so you could get a plot of a small timeseries right now, no matplotlib, no ggplot2, no gnuplot: just having the variable on the screen meant the graph was there automatically) and as painless as possible. Of course, I didn't advance much because I needed to rewrite everything that I had working so it was easy to extend, and I found another shinier thing to work on... But that still sits in my "someday" code folder.

For now, Apache Zeppelin is almost there in terms of what I want, and has spark in it so...

rezashirazian 2 ago 0 replies      
This is feels somewhat similar to Swift's playground.

I think every language should have something like this. Now that we've been compiling and highlighting errors in real time for such a long time, it only makes sense to do the same at run time and provide real time visualization of the program in action.

hibbelig 2 ago 1 reply      
How does this compare to LightTable?
sebastianconcpt 2 ago 0 replies      
I love instant feedback, it really changes the way you can create code because your cognitive system is unloaded of many assumptions to rely on real results (in-place evaluated feedback). For someone that used to work in Smalltalk this is a move in the right direction in IDE design.
scottmf 2 ago 1 reply      
I must be missing something. Link to download or Github?
pavlov 2 ago 1 reply      
Nokia used to make an IDE for the Symbian operating system called Carbide:


The memory of Symbian C++ makes me shudder.

electriclove 2 ago 0 replies      
Very cool! Some of this reminds me of the talk that Bret Victor gave at CUSEC 2012:https://vimeo.com/36579366
rkho 2 ago 0 replies      
On a mobile webview I noticed that the auto play video took over the screen and opened several instances that I had to keep closing one after another. I didn't get much further after that.
zubat 2 ago 0 replies      
OK. I don't know if Carbide was designed for this or not but I suddenly had a vision of a data viewer widget that is just an entire notebook system with a custom initial state. That would be pretty handy and I could imagine folks chucking it into a web page without extensive work.

For applications I have no expectations that these kinds of systems scale up or give the touted wins. But they are great for easing the exploration process.

stockkid 2 ago 0 replies      
Feedback: I did not want to read all the texts (it's too much) and just wanted to try it out. But I couldn't easily find out a way to try. After scrolling down a lot, I finally found some example notebooks.

Maybe 'try out an example' button at the top?

hugozap 2 ago 0 replies      
This is great. Tools like this are useful when exploring a problem, it's very handy to have the instant feedback. Using the widgets to tweak things is also very cool, I could definitely integrate this into my workflow ( if I can install it locally ).
Waterluvian 2 ago 0 replies      
This looks perfect for live demos and interactive gists of relatively straightforward code.
outworlder 2 ago 0 replies      
The terminology is interesting. "Notebooks", "Kernels"...

So, Jupyter-like?

pmontra 2 ago 0 replies      
The most interesting feature is how it computes the inverse of the program to yield the inputs that produce a given output. I'm not sure if it's useful but it's cool.

But saving to public gists, no thanks.

peternicky 2 ago 0 replies      
Awesome work!! Please release the source!
polskibus 2 ago 0 replies      
Would love to see an example on how to use it with D3. Live refresh is very useful when working on d3 visualizations scripts.
rch 2 ago 0 replies      
Seems like this would be a great foundation for building a CMS backend interface. I'd try it in that context before calling it a programming environment.
fiatjaf 2 ago 0 replies      
Seems awesome. I will probably never use it because it is not flexible enough to run anywhere and on VPSes without friction, but I sincerely hope you succeed.
donretag 2 ago 0 replies      
Am I the only that thinks of Union Carbide (and the disaster) when they see the word carbide?
partycoder 2 ago 0 replies      
Reminds me of the "manipulate" package for R. This takes the concept really further though.
didibus 2 ago 0 replies      
Is it only a notebook style environment? Or can it be used as a modern IDE for real programming?
pbreit 2 ago 0 replies      
Looks interesting.

What sorts of use cases were you envisioning?

Would it work or make sense for ad hoc mongo querying?

kiliancs 2 ago 0 replies      
The site skyrocketed Firefox's CPU usage, unfortunately.
_prototype_ 2 ago 0 replies      
Their neural network example is broken
skorkmaz 1 ago 0 replies      
sorry for pointless comment but looks really cool
cercatrova 2 ago 0 replies      
Open source?
kasajian 2 ago 0 replies      
antar 2 ago 0 replies      
Looks limiting.
Amazon Vehicles amazon.com
466 points by kjhughes  1 ago   244 comments top 51
simonsarris 1 ago 17 replies      
Before you laugh that you'd never use this to buy a car, it may be worth taking the time to find your make/model and add it to your "Garage" on amazon. Amazon will then try to confirm or not if the cabin air filter you are about to buy is a correct fit for your car. Amazon tries to do the same thing with camera gear.

At least, that's the theory. In practice you still need to do 100% of the due diligence or you'll end up with spark plug wires that are for the non-California version of the engine of your girlfriend's 2004 Outback, and then you have to build an awkward bracket over the motor because you're too stubborn to return them...

ThePhysicist 1 ago 4 replies      
What's really interesting about this business model is not the fact that you can buy a car online, it's the price transparency that you create with this: Car dealers often have 10-20 % margin on a car sale, and they make most of their money by providing services to you after you bought the car, so they are usually able to give you some big discounts. Also, as they usually receive a bonus by the car maker for selling a given number of cars, they have a strong incentive to sell more. The money that they make for themselves depends therefore on the margin that they can achieve, which in turn depends on the customer's knowledge of the "fair" price. Having all the transaction prices out in the open would therefore be quite a nightmare for most of them, as it will allow customers to push the price down quite a bit. Especially for small dealerships (that have higher fixed cost in comparison to their sales and receive less bonus as they sell fewer cars than big dealers) this could be highly problematic, while it could be beneficial for the larger dealerships. Truecar has been doing this model for a while, and I guess Amazon finally wanted to have a share of this market too (and the data associated with it).

On the technical side it looks like they're using the JATO database for their search/configuration tool (http://www.jato.com/). A few years ago I also implemented a car configuration website/app for a German startup (which doesn't exist anymore) using this data. The most challenging aspect was the huge amount of possible configurations/options that exist for some models (especially BMW and Mercedes), which would also have very complex inter-dependencies (e.g. choosing package A + B means you can't choose package C except if you also choose D). To resolve this I had to write a dynamic constraint solver: Finally some abstract computer science that was usable in practice :D

As far as I understand, in the US there are usually less options to choose from, which makes extensive configurators unnecessary. Still, it was an interesting challenge.

hardwaresofton 1 ago 9 replies      
Yes please.

Anything that will get us closer to direct-to-consumer sales of cars. I'm ready for car dealerships to be a relic of the past.

ejcx 1 ago 5 replies      
I think it would be great to buy a car from Amazon. The WYSISYG pricing is something missing from car buying. Negotiating a car price can be a lot of fun if you're a good negotiator, but people who aren't might love this.

It also gives you a tool for negotiating if you DO plan to take the offer in to a local dealership.

htedatsu 1 ago 2 replies      
This will be a... clarifying moment for eBay. I make my living on eBay and gross about 10x the average USA wage. Used stuff is involved. eBay once loveed people who bought and sold used stuff. They are one of the biggest used car venues in the world. They want to sell new stuff and become Amazon. Now Amazon is beating eBay on its own turf. Maybe eBay will have to remember their roots and start taking care of both its sellers and its end users in the used stuff sector.
deegles 1 ago 2 replies      
Where's the one-click button? :)

Seriously though, if I were in the market for buying a car I would love to use Amazon. I wonder what the restrictions are for them to register as a dealership in each state...

rudedogg 1 ago 7 replies      
Has anyone used Costco to buy a new vehicle? Their program isn't direct or anything, but you're supposed to get better prices than working with a dealership directly.

I've been curious how much below invoice they are. The closest one is 4 hours away from me, so I'm not a member (and can't see prices).

paulrosenzweig 1 ago 3 replies      
I wonder if once Tesla fights back all the local monopoly regulation around dealerships, this will naturally transition to an online car dealership. They'll have to figure out test drives though.
GBond 1 ago 0 replies      
Serious question. What exactly is new here? I've used the My Garage to buy part from amzn before. Did they add a new homepage for My Garage? Some of the comments are suggesting this is a one-click direct car shopping but doesn't seem to be. Just wondering if I missed something.
dchuk 1 ago 3 replies      
How do they already have so many reviews and content for this? Private Beta? Pulled from other sources?
webwanderings 1 ago 4 replies      
I was just wondering yesterday, if there is a service out there, which would let me see the type and model of cars which are popular, and most-bought? I assume there must be such a dataset available somewhere through government sources? Or perhaps not. Such a data could be a valuable service based on crowd's wisdom. Amazon here seems to provide public comments, which is generally available via Google.
qntty 1 ago 3 replies      
How do I buy one? I don't see an "add to cart" button anywhere.
mattiemass 1 ago 1 reply      
"like hell Tesla is going to be the only one selling cars online"
Loic 1 ago 0 replies      
In Germany you have a very nice service if you want price transparency before buying a car: http://www.meinauto.de/

Basically, car dealers register their offer on the portal and you can shop without knowing about the dealer. Just at the end, you sign a middle man agreement and the website is forwarding your request to the car dealer.

At the end, you need to travel maybe 300km to get your car, but you get it from a normal dealer with standard guarantees. If you still want to buy at your home car dealer, this gives you a good idea of the discount you could try to negotiate.

santiagobasulto 1 ago 0 replies      
There'll be a lot of people saying "I'd never buy a car on Amazon"...

The same people that said "I'd never buy a book on the Internet" back in 2000.

MasterYoda 1 ago 0 replies      
I was hoping for Volvo Amazon vehicles :)


jsight 1 ago 1 reply      
Didn't they actually have a car sales service at one time? It seems like I remember them doing something with this about 15 years ago.
erickhill 1 ago 1 reply      
Now, if they delivered one of these via Prime ...
daljeetv 1 ago 3 replies      
Soon we'll rent cars, like we rent books. :)
grecy 1 ago 0 replies      
Does anyone know if they'll do Amazon Associates on a car sales? 5% of the purchase price would be very nice indeed.
camtarn 1 ago 0 replies      
Hah! When I worked at Amazon, we occasionally used to joke that the only thing you couldn't buy in the Automotive and Industrial department was actual cars. That discrepancy has now been corrected :)

[edit] Aw. No, it hasn't: this is just a way to "See specs, read reviews, and ask owners" about vehicles. Pity.

CodeSheikh 1 ago 0 replies      
Boy, good thing Amazon does not need to ship them with USPS. "Pime Drivers" will drive them to your door steps.
ameliaquining 1 ago 0 replies      
Was anyone else expecting this to be an announcement of an Amazon self-driving car project?
soheil 1 ago 0 replies      
You can't actually buy a car, but could this be a prelude to a near future when you can?
bdrool 1 ago 1 reply      
Oh awesome, I can't wait to buy a knock off truck from "Ford-Motors" that has a handful of fake 5-star reviews and doesn't meet safety standards. Thanks, Amazon!

(See here for explanation: https://news.ycombinator.com/item?id=11939829)

kyyd 1 ago 1 reply      
So many 5 star reviews.
Overtonwindow 1 ago 0 replies      
I just added two Bentley's and a Porsche to my online garage, since I'll never have these in my real garage.
bluedino 1 ago 0 replies      
I don't really get what this is - if I want to find parts for my car, I can just type something like '2012 Honda Accord Wipers'

I already have my car in my 'garage', they've had that functionality for years.

robbiemitchell 1 ago 1 reply      
It will surprise many that this is already a thing. e.g., vroom.com
grandalf 1 ago 1 reply      
Ever since CarWoo bit the dust I've been hoping for some actual innovation in this space. Let's hope this is it!
mikeevans 1 ago 0 replies      
Waiting for the time that I can referral link a car purchase.
codecamper 1 ago 0 replies      
I couldn't find any Trabants for sale. Lame.
madengr 1 ago 2 replies      
I don't believe those ratings. I was looking at Ram truck ratings the other day from (from an unbiased site) and they were 1.5/5.
paxunix 1 ago 0 replies      
A number of people have commented about the quality (or its lack) of Amazon's "this fits/doesn't fit your car" data. Let me explain a little about how that works:

In North America, this fitment data conforms to a popular (but not the only) schema called ACES (Aftermarket Catalog Exchange Standard). It's a long and pretty well-defined XML schema that specifies things like year, make, model, and all kinds of attributes like engine configuration, fuel type, wheelbase, etc., as well as brand name, part number, quantity, and so on. Fitment data providers create an XML document according to this schema and populate it based on the products they say fit particular vehicles.

For example, say you are the manufacturer of a FRAM oil filter (FRAM is a common aftermarket filter brand in the US). It has a part number A123. You know it fits the 2000-2010 Honda Accord with engine XYZ. You add to your fitment XML document an entry for this filter that specifies brand=FRAM, partnumber=A123, parttype=oil filter, years=2000-2010, make=Honda, model=Accord, engine=XYZ. Now, you (or some 3rd party you designate that specializes in turning your catalog of products into fitment data), sends this fitment document on to companies that care about fitment information because they are selling your FRAM oil filter and want to be sure their customers can tell if it will fit their car or not (Amazon, Ebay, Rock Auto, etc.).

They take that fitment data and join it against their product database and out pops the yes/no fitment data you see on their website.

Now scale it up: there is no requirement that only one company produce these fitment records. Anyone else can produce a fitment record that says all the above, but for the 2000-2010 Honda Civic. Maybe that's a mistake, but as a receiver of the fitment data, Amazon or Ebay can't know it's a mistake--they can only presume the fitment data they are given is valid.

Now, complicate it further by adding the human element: e.g. some fitment data providers have fitment data in Excel spreadsheets. Some poor human fat-fingered that data from the spreadsheet into XML and maybe they left off the leading 0 on all the part numbers (because that's Excel's default for number cells). Oops. Now none of those fitment records will match to any parts in the database of the companies that sell them. Or they're entering this data off a piece of paper and can't tell if that's a 0 (zero) or an O (oh). Oops.

Or, worse: the fitment provider gets the wrong vehicle ID (because the schema is all based on IDs, not human-readable names) and submits fitment data that says that FRAM filter fits a 2000-2010 Tesla Model S. Well, that's extremely unlikely, but the receiver of the fitment data is a machine and the machine doesn't know that is completely ridiculous.

Or equally bad: the fitment provider says it fits a 2000-2010 Honda Accord, but doesn't specify the engine type at all. Now, Amazon's machines see that and think "the customer only needs to tell me their year, make, and model". A smart human knows it also needs the engine configuration, but the machine can't easily know that because none of its data specifies an engine configuration is needed for fitment. So Amazon sends out the filter because its data says "it fits!" and the customer is unhappy.

So, in the end, the customer is displeased because Amazon shipped them an oil filter that can't possibly fit their car, even though it told them it would because they can only go on what the data tells them.

A closely related problem is that sometimes a seller will have a product that they know should require fitment data (like an oil filter), but there has been no fitment data submitted for it. In that case, the company can neither say it fits, nor it doesn't fit--it doesn't have enough data to make the determination. This is common as the new model-years start to hit the marketplace: the car exists; you can buy it; you can buy parts for it; however, the aftermarket fitment data hasn't caught up with the car's attributes yet. It also happens when Amazon has fitment data for some models but not for others, even if the part will fit those other models--without data saying so, there's no way to say "yes that will fit your new 2017".

To complicate it even further, if you're talking about this stuff outside of North America, there are other schemas and data providers with very little overlap of the NA offerings. This is visible in the very different fitment experience you see in most EU countries on Amazon (for example, try https://www.amazon.co.uk/auto and select a car).

The takeaway? Unhappy customers result when you have complex data quality problems, and sometimes it's no fault of the implementation at all.

Negative1 1 ago 0 replies      
Getting through "The Everything Store" right now, and, yep. Wow.
bowmessage 1 ago 1 reply      
no motorcycles? :(
rm_-rf_slash 1 ago 4 replies      
I wonder how long it will take until you can buy a house on Amazon.
MicroBerto 1 ago 0 replies      
Will the Associates program pay commissions for this??? If so.....
praveenster 1 ago 0 replies      
Where does this leave the likes of Edmunds, AutoTrader etc.?
mikehines 1 ago 0 replies      
Self-driving car market research under the cover?
briansugar 1 ago 0 replies      
tapeman 1 ago 0 replies      
Psh, no Prime delivery?
ruffrey 1 ago 0 replies      
Where are the Teslas?
known 1 ago 0 replies      
Can we buy car engines?
avisser 1 ago 0 replies      
Killer URL. /s Would it kill them to add a route?
_audakel 1 ago 0 replies      
I would like an Amazon Prime Air to deliver my new car plz.
partycoder 1 ago 0 replies      
They need VR
bedros 1 ago 0 replies      
where's the lease offer?
codecamper 1 ago 0 replies      
So if I buy the car with my amazon card, I get 3% back? woot!
niciliketo 1 ago 0 replies      
For high value items like vehicles reverse auctions can work, as long as you can specify what you want and are not too emotionally attached to the model that you want http://blog.marketdojo.com/2016/03/buying-a-car-via-auction....
A few HTML tips mozilla.org
495 points by nachtigall  3 ago   173 comments top 17
adregan 3 ago 4 replies      
One thing I remembered while reading the section on telephone inputswhat happened to all the HTML5 date and time input types?


All of the other cool new input types appear to be on the way in the modern browsers (http://caniuse.com/#search=input%20type), but Safari and Firefox don't appear to be budging at all.

I would prefer to use the native date pickers across devices, but I feel like I have been waiting on those elements to land for years now. Did I miss something?

megous 2 ago 2 replies      
I wish for icon fonts to die.

I don't mind website designers defining what fonts to use. That choice is almost always something I don't want, unless it's serif or sans-serif. I can override that choice with my own, which is better for me. Having the same font on every website increases readability and pleasure of using the web. Browsers allow for this in the user visible configuration, perhaps for this reason.

Also, security. I don't trust random fonts from the internet, to not be malicious, so I disable them all.

If I choose readability and security, use a normal browser feature to do this, by disabling website enforced font choice, the website should not break.

Icon fonts are definitely a hack. There's no fallback to anything sensible when the font can't be used. It just breaks, and the website is unusable. At least use a title attribute, so that when hovering over some random character I can see what the supposed icon I never saw stands for. Especially when used for icon only navigational controls.

And this is not even related to accessibility.

rimantas 2 ago 1 reply      
Anyone remembers Dan Cederholm (http://www.simplebits.com/) and his excellent books "Web Standards Solutions" and "Bulletproof Web Design"? He even had a series of exercises like this on his blog.Oh, the gone days of the web which was a web and not some misfit wannabe app engine :(
wtbob 3 ago 6 replies      
Web Fonts for images are a true anti-pattern. It's actually almost funny how many pages are full of and so forth, because they assume that folks are using a graphical browser running JavaScript.
mattmanser 3 ago 7 replies      
Is it common these days to nest inputs in labels? As a comparison, bootstrap doesn't, http://getbootstrap.com/css/#forms.

To me that's conceptually as well as semantically broken, the two are separate things why is the input inside the label? He's saying make everything semantic, and then semantically breaking forms.

Also if you have any inputs that don't have a label for whatever reason you either have to wrap them in them any way to get CSS to work or make sure you never rely on the input being nested in the label in your CSS.

And having them apart also makes it easier to design responsively if you want a side-by-side design for desktop vs a stacked design for mobile.

Saves having to add a for tag I guess.

I guess a lot of that article is opinion though, so you're always going to get people objecting to one point or another, though a lot of it seems sensible advice.

msl09 3 ago 5 replies      
I can see the value of almost all of those recommendations but I don't understand the value of using the correct header sequence or tagging subheaders. I don't see anything in the linked W3C article that explains why I should use one idiom over another.
jonespen 2 ago 0 replies      
A note regarding alt attributes on img: they should almost always be empty. Please don't just fill it with the related title or heading, as it just creates a lot of unnecessary text for screen readers. More: http://www.456bereastreet.com/archive/200811/writing_good_al...
ravenstine 2 ago 2 replies      
Funny how they answered a question about paragraph tags that I was wondering yesterday. Most answers to the "p vs br" question seem to be "do whatever you want", but the recommendation in the article seems most correct to me, since p tags are for the parser(in case it needs to know what the paragraphs of the body content are) while breaks are presentational.
DanielBMarkham 3 ago 8 replies      
Can anybody tell me how legit the placeholders recommendation is? I understand it's not accessible, but on a limited viewport, having the text tell the user what you want seems like a no-brainer. And I see it a lot. Isn't there some other way to provide accessibility?
sirtastic 2 ago 3 replies      
Question: Can anyone comment on how important HTML taxonomy in terms of SEO (disregarding accessibility). Will the use of <header> and <article> make any difference in how a page is indexed and ranked directly or indirectly?

One indirect way I can see SEO benefiting is via sharing. Facebook crawling will have an easy time grabbing the <h1>Title</h1> along with whatever comes first in <p>for an excerpt</p>.

Is there any no-brainer reasons or supporting evidence proper taxonomy helps rankings?

donjh 2 ago 0 replies      
I found the section on SVG sprites particularly interesting. Here's a Webpack plugin to generate SVG spritesheets from individual assets: https://github.com/mrsum/webpack-svgstore-plugin.
pjungwir 2 ago 2 replies      
I'm curious if there is any good solution to use sprite sheets with background-image, possibly with a hover effect, and still get the accessibility benefits of alt tags? For instance:

 <a class="fb sprite" href="..."> </a> .sprite { background-image: url("/sprites.png"); background-repeat: no-repeat; } .sprite.fb { background-position: -4px, -4px; width: 42px; height: 42px; } .sprite.fb:hover { background-position: -4px, -54px; }
What would you do to make that accessible? I did some Googling about this a few weeks ago, and it seemed to be a topic of "ongoing research." None of the solutions I could find were very appealing. So, any suggestions?

nailer 2 ago 1 reply      
Slightly off-topic, but:

> Couldnt be much more from the hearth<br>

Couldnt be much more from the heart.

lllorddino 3 ago 3 replies      
That feeling when the first paragraph of an article points out something you've been doing wrong all along :D. I use <br> in between paragraphs and it looks horrible, what should I be using instead?
perlwle 2 ago 0 replies      
I wish safari supports currency keypad with decimal in HTML5. It's working for android but not for iOS ;-(
intrasight 2 ago 2 replies      
> There are some CSS limitations though: when using SVG this way, with <use> linking to a <symbol>, the image gets injected in Shadow DOM and we lose some CSS capabilities.

Which is why SVG "symbol" and "use" are best avoided. Your client will inevitably ask you to style an icon and you won't be able to. Avoid the shadow DOM and you'll have full CSS capabilities.

56k 3 ago 3 replies      
Everyone know these things. What's the big deal?
Almost 80% of Private Day Traders Lose Money curiousgnu.com
402 points by kabouseng  5 ago   268 comments top 47
habosa 4 ago 5 replies      
Excuse my meta-comment, but I notice that HN seems to have an obsession with proving that active trading is bad and that we should all buy low-cost indexes. And before anyone jumps on me, I 100% agree with that investment strategy and use it myself. I'd just like to comment on the intersection of that belief and tech enthusiasts.

It seems to generally come from a place of binary/analytical thinking which we're all so good at. Studies show that active investing is a losing battle for the average investor, so we assert that as fact and invest our money that way.

But I think deep down in myself (and probably others) it comes from a place of wanting to justify risk aversion. It seems that many people that we all know do become quite rich as investors, and the secret to their good fortune is not apparent. Studies promising us "they'll all lose in the long run!" makes us feel good about our decision not to participate in the game.

It's similar to the derision of startup equity, etc. I know many people who have become very rich at a very young age by latching on to the right startup, but I like my comfortable BigCorp job. The HN comments are a place where I can feel smart about my decision to not take risks even as we celebrate those who took risks and won.

Anyway I don't know why I chose to write this comment or explore this topic at this moment, I hope someone else reads it and understands my sentiment.

nicholas73 5 ago 8 replies      
This blog post did not answer it's own question because it's conditions were not day trading (over 3 trades in 12 months). That condition selects for people choosing individual stocks hoping for a moonshot, for which people tend to choose riskier stocks rather than stocks actually likely to make them money. So no wonder 80% lost money. On the other hand, notice that the 20% who do make money have a large power distribution curve.

A better way would have been to compare performance versus frequency of trading (I don't expect this to prove one way or another though). The fact is, there are many strategies one could take, and it's how well you execute them that counts.

Personally, I don't find daytrading riskier than holding stocks. By far my biggest losses come from holding the wrong stocks for a long period. It just seems riskier because you have to confront yourself with the possibility of loss each day, rather than hold "long term" and deny that you are wrong.

I now take the Doyle Brunson approach. The poker champion loved to pick up small pots and felt it was critical to do by aggressively playing small hands. That way, these little wins pay for the risk of playing bigger hands over time. I've had the same experience - my daytrading tends to be small money but it stems from work done for holding long term stocks. So why not put it to use?

But, to actually make good money daytrading is still really difficult. Commissions alone can make you have to be 55/45 correct, but there is also the steamroller affect where people tend to hold on to losses and double down further. So it's also about mastering yourself in addition to your market. Otherwise, there is not reason why you can't be better: you are putting in more work than others to make good decisions, and that's how you profit. Trouble is, when are you still outgunned informationally?

crucialfelix 5 ago 2 replies      
I did it for a while in the very exciting year of 2008. I see-sawed back and forth and was profitable for months at a time, but in the end I was down. I strictly kept my daily losses limited at $50/day. So really small risk. At the end of that year my day trading losses were about $1500 and otherwise I was in cash just watching the world burn. My long term funds lost more than that in a day.

I've lost out on more money by not being in the market long term going up.

Of course I thought I might get good at it (everybody dreams), but the main reason I did it was to learn to confront fear and to keep logical and strict to the trading plan in the face of it. I am still risk averse though, so I didn't really learn thoroughly.

The other main thing I learned is that if you aren't a robot then you don't have any business out on that killing floor. The age of human day traders is over.

inputcoffee 5 ago 3 replies      
Isn't is possible that the sample is biased because you only looked at the subset of traders who were not sophisticated enough to turn off the feature that allows others to observe their trades?
Animats 4 ago 0 replies      
Even worse than day trading are "binary options". This allows betting on, for example, whether the price of gold will go up or down in the next minute. Most of those are total scams. There are about 100 binary trading firms based in Tel Aviv, Israel, and most of them are telemarketing operations for a software platform called SpotOption.[1] Features of the scam include 1) they're not brokers, they're the other party, so you're betting against the house, 2) the house controls the price info and may tweak it slightly to make you lose, 3) there's a "margin trading" system which prevents you from withdrawing until you've done a certain amount of trading volume, and 4) even if you win, they make it really hard to withdraw money. Most "investors" lose 100% of their money.

Read "The Wolves of Tel Aviv" in the Times of Israel.[2] All these outfits are prohibited from operating in the US (the CFTC catches some of them trying; "Banc de Binary" was forced to return all US customer losses) but they've been able to operate in the European Union by registering in Cyprus and using that as a passport to the entire EU.

[1] http://www.spotoption.com/[2] http://www.timesofisrael.com/the-wolves-of-tel-aviv-israels-...

brongondwana 5 ago 8 replies      
It's just slot machines with a more respectable coat of paint.
joosters 5 ago 1 reply      
The Financial Times often refers to spread betting companies as 'debt collection agencies with a side interest in trading'. Emptying the accounts of users and then collecting the additional money owed is what these companies do. Then they use some of the profits to advertise for new customers to continue the cycle...
chollida1 5 ago 4 replies      
I re-read Michael Lewis' Liar's Poker once every few years. In new reprints he's added a prologue where he says something along the lines of "I tried to write a book about the horrors of wall street and instead it became a recruiting tool for wall street"

In a similar vein whenever something like this comes up I write something that says don't try and do this and inevitable I get 10+ emails from people saying "Ok I understand but I still want to do it anyway."

So if you really want to try and day trade, here is my 5 points of advice

1) You haven't done anything until you trade with real money. Ever played poker with friends that wasn't for money? You always get some jerk bluffing and going all in on Ace-7 because why not, it doesn't matter.

If you want to trade you need to learn your own emotional limits. Get invested as soon as possible. Learn what kind of stress you can deal with.

2) The average successful individual algo trade looks like John Turturro charter Joey Knish in rounders. You are always grinding out a living $100 to $1000 at a time. If that isn't' appealing then stop before you start.

3) You need money to begin. It takes as much effort to trade 2000 as it does 200,000. Above a million or so things change. A minimum amount I'd use would be 50,000. If you can't set that aside to trade then don't start.

I mean if you have a great year trading 5,000 and make 20% you have made $1,000. That's not worth the effort. But at say 100,000 your 20% is starting to be real money!!

Be clear to yourself that you are doing this for the money. Any other reason just gives you an easy out.

4) The most important number to look at is draw down, not sharp ratio. I see lost of people on quantopian show great backtested algos that return 200% over 5 years, but have a 70% draw down at one point.

Most people don't have the psychology to loose money. At 10% down you start to question yourself. At 20% down you get really grumpy in all aspects of your life. Most people don't have the ability to let their positions fall to 30% down. If your backtest has a draw down of more than 30% you don't have a workable algo.....period.

Sharpe ratio is great if you are an HFT firm trying to decide how to divvy up your money across 30 successful algos. You don't have 30 successful aglo's. You will be lucky to have one.

5) Don't use leverage....... you can break the other rules, this one you can't. I've seen people go bankrupt. Don't use leverage.

akrymski 5 ago 1 reply      
eToro wouldn't be used by any remotely professional day trader. Most of those accounts are most likely idle and judging by eToro's target audience - highly amateur. Whilst majority of course loose money, so do majority of businesses in any industry, from startups to restaurants to hedge funds. The nature of human economics is such that there's more money chasing opportunities than the value of those opportunities.

Suggested reading: "Economics in One Lesson"

mholmes680 5 ago 0 replies      
I have a serious problem with the "conclusion" of this post. This data isn't normalized against some kind of control group to show that maybe day-trading during Brexit is a different beast than index-tracking during Brexit. Or maybe its the same beast, we can't conclude anything...

I'm beginning to realize the difference between "data scientist" and just looking at numbers to put them on a chart.

songzme 4 ago 3 replies      
Here's a story of my mom's journey as a day trader: My mom got laid off in 2008. The job market was brutal and she had a hard time looking for a new job. She had to find ways to make money to support my sister and I through school so she turned to the only other place where she could earn some money - the stock market.

She started with around 100k, read up about stock options, and decided that shorting calls and puts would work well. The time value decay seems to work in the investor's favor. Every time she made enough money, she would take them out and buy houses (all cash) and start from 100k again. Now, 8 years later, she has acquired a few properties in West LA and Oakland and makes more rental income per month than my salary as a senior software engineer. She still trades today.

hal9000xp 4 ago 1 reply      
Since 2012, I'm interested in financial markets. I read some university books. I trade from time to time (mostly index ETF and futures/options for experimental reasons).

I would argue that ~80% is quite optimistic estimation. I think in long-term more than 99% of retail day traders lose money.

It's NOT because making money on financial markets is ridiculously difficult.It's because if you trade frequently, you get feedback from market pretty quickly.The feedback in terms of profit and loss of your money.

If you become day trader, I would guess that you want beat the market (otherwise you just buy and hold index ETF).It means that you as a day trader is willing to take risks to lose money in case you are wrong that you think you are able to beat markets.

What does it mean "to beat the market" in long-term? It means that you are smarter or faster or have some insider information. In other words it means that you have competitive edge.

Well, obviously, average person doesn't have competitive edge against hedge funds and other institutional investors who hire people with outstanding intellectual capabilities. Just read hedge fund interview questions to see how they filter people.

Other way to estimate your situation as a retail day trader: if hedge funds don't want to hire you, why do you think you are still capable to outsmart hedge funds while you can't answer their interview questions for trader role?

I don't say that it's inherently impossible to beat the market as a retail investor. It's fairly possible if you are REALLY smarter than others. And in this case, you most probably are capable to impress professional traders and investors while talking about markets. If so, they will bring you their money, and incorporate hedge fund, and you are not retail investor anymore. Just like Michael Burry did.

In other areas, ~99% of failed attempts is also true. For example, ~99% startups fail to make money. So it's normal state of affairs.

0xmohit 5 ago 2 replies      
And the primary reason is that one loses more in a single trade than what one probably earned in 5 or more trades.

Humans are inherently risk-averse when it comes to protecting their profits and risk-taking otherwise.

The possibility to make money increases if one acts more like a robot, i.e. acting strictly upon rules decided a priori.

koolba 5 ago 2 replies      
If you think that's bad, check out forex trading accounts. I don't have the exact stats on hand but an insane percentage of them get completely wiped out.

I'm not talking about losing some of your money, I mean 100% of the money in your account and possibly owing even more.

igf 5 ago 3 replies      
Restricted to the publically-available data from one particular and rather odd site, but probably not crazy.

At first it seems counterintuitive, shouldn't it be roughly 50% gaining and 50% losing? But I guess that people typically put in a modest amount of play money, and then tend to trade until either they run out or get discouraged from losses.

thedevil 5 ago 1 reply      
The way the math works out with that distribution: only 80% of them lose money over one year, but as time continues over multiple years, the percent losing money approaches 100%.
NDizzle 5 ago 0 replies      
... of course, 50% will end up in the bottom half and 70% will end up in the bottom 70%.

- Charlie Munger.


samfisher83 4 ago 9 replies      
If someone is consistently good at losing money 60% of the time. Why not just do the exact opposite of whatever their initial hunch is. Then they should make money. Based on loss of 36% from the article that is more than just transaction and trade fees which are 1 to 2%of trade therefore if they just do the opposite of what they are doing they should make money.
83457 4 ago 0 replies      
I looked into trading a while back after my dad lost a bunch in the market. I read a lot of info and in the end finally found someone who stated information like this in the link. In the end I learned that the vast majority of retail traders lose money, I think 85 or 90% was listed, the majority of people who get into professional day trading quit within a couple years, day traders make profit by leveraging their money many times over just to make a living, and day traders actually take an income instead of letting their money ride and losing it all in a few trades but this also keeps them from progressively getting richer. Day traders expect to lose on at least 40% of their trades which is apparently one of the biggest issues mentally with trading. None of the retail platforms explain these facts nor many other sources of info online.
glippiglop 5 ago 0 replies      
It's an interesting stat, but the chart doesn't speak as to the levels of experience of the traders. All beginners lose money, but it's entirely possible for an amateur to become a profitable day trader with time.

Speaking from experience, the real issue for me was that once I started to get good at it (after 2 years) I realised just how awfully boring it was. It's not intellectually demanding and for the investment of your time, stress and the risk involved, the rewards even if you reach profitability aren't worth it.

Now that I can reflect on it, I couldn't be happier that I stuck with software development instead.

lordnacho 4 ago 0 replies      
Here's a quote from a guy who runs one of these shops:

"90% of the customers lose 90% of their money in 90 days."

A friend of mine was chatting to him about doing some business.

I would think it's more about leverage than commission. Looking at FX, you can get tiny spreads, almost comparable to what I saw in a hedge fund. It takes a while to eat up a whole account on such small percentages. Leverage, on the other hand is something you can use to demolish capital over an afternoon. Ratios like 1/200 are a formula to go broke if you haven't had a look at something like Kelly Criterion.

grondilu 4 ago 2 replies      
Isn't day trading a zero-sum game? If so, it's not very surprising that the majority of day traders lose money. Every dollar you make comes from someone who lost it. So unless you have a competitive advantage, such as lower brokerage fees, you should stay away.

Me, I believe in https://en.wikipedia.org/wiki/Buy_and_hold. The money you'll make will come from the actual profits of the companies.

gedrap 5 ago 0 replies      
I think the analysis could be improved by excluding people who made a couple of trades worth of a few dollars just as an experiment, etc. This way, leaving only the people who took it somewhat seriously. Even then, the data would be heavily biased - it includes only traders who decided to use eToro AND keep the history public. This is far from a random sample and very biased.

However, I would not be surprised if the actual results (across all brokerages, etc) are close to that, or even more extreme.

DrNuke 5 ago 1 reply      
Was all in into Barclays shares the day before it was going bust in 2008 but got out before 5pm fearing the night and the wiping out. Instead, Gordon Brown bailed them before the market reopened the next morning ("too big to fail") and I missed my once-in-a-lifetime opportunity to make the big money as an indie trader. Done.
Paul-ish 4 ago 0 replies      
>On the 1st of August, 2016, I downloaded the publicly available data through their ranking API. I selected all users who were active during the past twelve months, traded with real money, and had at least three trades. The results consist of 83.3k traders who fulfill these conditions.

What may be interesting is to see a graph where the number of trades a person executes is on the x axis and on the y axis is median return. Do people who make more trades make more or less money? It would also be interesting to look at a window larger than 12 months. Do people with trades going back years do better than people with trades going back months? Is this gambling, or can some people build some skill in this?

unknown_apostle 5 ago 0 replies      
And then there are these terrible products they promote to retailers (at least in Europe). Like CFDs and such. Orders in these things don't even go to the market, like in those bucket shops from old days.
anonu 4 ago 0 replies      
The smartest thing an individual investor or trader can do is put their money in extremely low-cost index funds and leave them there. I have worked as a trader at various large financial services firms for almost a decade now. Even though I believe I have more insight into the markets in general than the lay person - the moment I decide to trade for myself, I will not get the same quality of execution and breadth of services I would sitting at my work desk...
bedhead 5 ago 0 replies      
Only 80%? Shocked it's not higher.
JackFr 4 ago 0 replies      
> The data source for this post is eToro, a brokerage company that offers a feature called Social Trading, which is social network for traders. It is enabled by default and allows users to view and copy other users trades. Therefore, everyones trading performance is publicly available who have not disabled Social Trading.

What if the winners are choosing not to make their trades public? That alone could introduce some serious bias.

nstj 5 ago 1 reply      
Ironically, it seems the HFT's have started eating their own lunch too [0]

> Overall, HFT firms revenues in the US have slumped from about USD 7.2 bn in 2009 to USD 1.3 bn in 2014 (see chart 3)

[0]: https://www.dbresearch.com/PROD/DBR_INTERNET_EN-PROD/PROD000...

known 4 ago 1 reply      
Insider trading is not illegal;http://cnbc.com/id/43471561
praptak 4 ago 0 replies      
"Why I don't trade stocks and (probably) neither should you", a very good summary about why it is stupid(#) to try to be "smart" about stock market:https://news.ycombinator.com/item?id=6831461

(#) If it's not your full time job.

drcode 5 ago 1 reply      
I suspect this is a relatively new development- I think 5 years ago I would have been pretty comfortable with my chances against AI trading bots. But in 2016, not so much.

Based on my own anecdotal evidence, I think the large corps building HFT systems with AI have gone a long way towards eating the lunch of "lone wolf" day traders in the last few years.

thomasrossi 4 ago 2 replies      
"80%" it's very unlikely, say it is true, you just do the opposite and you have free money. I suspect the correct overall percentage should be much closer to 50%. Finding a profitable strategy should be much more difficult than "do the opposite of day traders", right?
atemerev 5 ago 0 replies      

Anybody who doesn't understand exactly where money are coming from and why, will be ripped off, and probably deserves it. If you want to gamble, you will be gambled. If you want to reduce your risks, you have to research accordingly.

btbuildem 5 ago 0 replies      
It would be interesting to know whether the gains were concentrated among a small group of same individuals over time, or equally across the large group (ie everyone getting lucky once)
Nihilartikel 4 ago 0 replies      
Did not read the article; but is anyone looking at the kooky square-wave action on SPY and DIA today? I'm totally hopping on to the next 60m cycle! Can't lose.

Or... maybe not

pbreit 4 ago 0 replies      
The headline is probably close but I don't think you can reasonably extrapolate out from only eToro customers.
eddd 4 ago 0 replies      
Afaik 98% of forex day traders loose money. You can't compete with scalpers with 10ms advantage.
sunstone 3 ago 0 replies      
Day trading is the perfect forum for the overly confident.
EvgeniyZh 5 ago 2 replies      
Well, how else someone would earn huge amount of money in this negative sum game?
wehadfun 5 ago 1 reply      
80% are losing who the hell is winning and how? I fell like they are cheating or have some sort of advantage that is not easily available to private day traders and no I don't think that advantage is a PHD and fast data connection.
cloudjacker 4 ago 0 replies      
Almost 80% of Hedge funds lose money too
danieltillett 5 ago 1 reply      
And the other 20% just lie.
paul_milovanov 4 ago 0 replies      
Furthermore, almost 100% of private day traders lose money!

from the your headline is meaningless and you should feel bad dept :P

HashThis 5 ago 1 reply      
I'd love people's opinion on this. From my understanding, HFT (High frequency trading) firms are front running stock orders. NYSE sells them data on stock orders. They buy data center space in order to front run stock orders. That is how the HFT guys make a profit when Day Traders can't make the same kind of profit. Is that true?
Text Summarization with TensorFlow googleblog.com
411 points by runesoerensen  2 ago   68 comments top 18
nl 2 ago 0 replies      
Allegedly (according to HP labs anyway) my 15 year old software[0] was state-of-the-art at this on the CNN dataset[1].

By my thing was very heuristic based, and couldn't generate new sentences like this can. I'm pretty impressed - I'd say some of the machine generated summaries are better than the human ones.

[0] http://classifier4j.sourceforge.net/ (yes, Sourceforge! Shows how old it is!!)

[1] http://dl.acm.org/citation.cfm?id=2797081

fiatjaf 2 ago 3 replies      
This is way too difficult to get right, even for humans.

I dream of a not-so-smart news summarization engine that will not try to rewrite the news, but only pickup all the numbers and quotations, then present them in a table of who-said-what and how-many-what, along with the title.

This would put an end to filler-based journalism.

trengrj 2 ago 3 replies      
$6,000 for the Gigaword dataset they used to train...


8ig8 2 ago 2 replies      
There's SUMMRY (http://smmry.com). I don't recall it being very smart -- that is, it doesn't rewrite sentences. It extracts the most important/relevant ones to basically shorten an article. It's definitely useful.
vtempest 2 ago 2 replies      
Here's my approach when I built my text-summary app with TensorFlow's SyntaxNet.SyntaxNet (Parsey) gives part-of-speech for each word and a parse tree showing which less-important words/phrases describe higher-level words. "She sells sea shells, down by the sea shore" => (down by the sea shore) is tagged by SyntaxNet as lower describing "sells" so it can be removed from the sentence. Removing adjectives and prepositional phrases gives us simpler sentences easily. Next, we find key words (central to sentences) for news article based on n-grams, and then score key sentences in which they appear. Use MIT ConceptNet for common-sense linking of nouns and most likely relations between them and similar words based on vectors. Generate article summary from the grammatically simple sentences.

My question is how well the trained models interpret human meaning in joined sentences. I discovered that by simplifying sentences you lose the original meaning when that grammatically-low-importance word is central to the meaning. "Clinton may be the historically first nominee, who is a woman, from the Dem or GOP party to win presidency" is way different meaning than that if you remove the "who is a woman". I am also interested in how it makes sence to join-up nouns/entities across sentences. This will cause the wrong meaning unless you are building the human meaning structures like in ConceptNet by learning from the article itself, as opposed to pretrained models based on grammar or word vector in Gigaword.

My work for the future, is using tfidf style approach for deciding the key words in a sentence, which I would recommend over relative grammar/vectors. In the example in your blog post ("australian wine exports hit record high in september") you left out that it's 52.1 million liters; but if the article went on to mention or relate importance to that number, by comparing it to past records or giving it the price and so on, you can see this "52.1 million liters" phrase in this one sentence has a higher score relative to the collection of all sentences. As opposed to probabilistic word cherry picking based on prior data, this approach will enable you to extract named entities and phrases and build sentences from phrases in any sentence that grammatically refer to it.

VierScar 2 ago 0 replies      
Does this include the state of the network in its already-trained state? It looks like we need to train it with the $6000 dataset if we want to get good results like those mentioned. Is it possible for its state to be saved to disk and restored (so we don't all need a copy of the dataset)?
wrp 1 ago 0 replies      
We've observed that due to the nature of news headlines, the model can generate good headlines from reading just a few sentences from the beginning of the article.

This illustrates the importance of taking the trouble to understand a domain before trying to model it. I was taught in journalism class that the first paragraph of a newspaper article should summarize the story, the next 3-5 paragraphs summarize again, then the rest of the article fill in the details. Not only do the authors spend time discovering what should have been known from the outset, they reverse cause and effect. The model can generate good headlines due to the nature of newpaper writing, not due to the nature of headlines.

mastazi 2 ago 0 replies      
The Hainan example in the article is especially impressive, the generated summary uses completely different expressions compared to the source text, yet it is spot-on. Of course those are probably cherry-picked results, but still. As a side note, it would be interesting to see how the algorithm performs with longer sources.
Cozumel 2 ago 4 replies      
I wonder what it'd be like on a novel, say something like 'pride and prejudice', would it be able to essentially summarise the plot or would it end up like 'movie plots explained badly'

Either way, this is great research with a ton of real world applications!

ProxCoques 2 ago 1 reply      
That reminds me. Whatever happened to http://summly.com?
greenspot 2 ago 0 replies      
> we started looking at more difficult datasets where reading the entire document is necessary to produce good summaries

Was hoping to get rather some more insights on this.

Because when looking at the examples given, I wonder if we really need machine learning to summarize single sentences? Just by cutting all adjectives, replacing words by abbreviations and multiple words by potential category terms, we should face similar results. Maybe it's just a start or did I miss anything?

paulsutter 2 ago 2 replies      
Next one could turn this into a comment generator, generating comments in the style and personality of any HN commenters with an adequate corpus.
kharms 1 ago 0 replies      
The following appears in the README.md

 # Run the eval. Try to avoid running on the same matchine as training.
Why should one avoid that?


yalogin 2 ago 1 reply      
May be the wrong context here, but if I am starting out learning Machine Learning using the Stanford course or some other one, is Tensorflow a good candidate to look into? Or does it only contain the advanced algorithms?
hyperbovine 2 ago 3 replies      
The table is nice, but I'd like to see examples where it performs poorly as well.
gravypod 2 ago 0 replies      
I remember there was a hackernews comment I read once where the user created an emacs plugin to do just this. It would generate a single sentence summery about whatever text was input into it.

Edit: Hey, found it! https://github.com/mck-/oneliner

Probably not as sophisticated but does the business. Nice bit of work done on this.

deepGem 2 ago 0 replies      
I would love to use this kind of tool and look for gotchas or key hidden elements in legal documents. It's not exactly summarisation, but very useful. Something like bubbling up the important elements of the fine print.
josephcooney 2 ago 1 reply      
I seem to recall someone used Microsoft's AutoSummarize feature to reduce and reduce classical works of literature to a few lines. The results were pretty hilarious, but I can't find it now.
Show HN: Top books mentioned in comments on Hacker News hackernewsbooks.com
429 points by leandot  1 ago   180 comments top 49
leandot 1 ago 16 replies      
Hacker News Books is a service that aggregates all links to books on Amazon, Safaribooks and OReilly found in comments on Hacker News. It does that on a weekly basis, calculates a rank based on how often each book is mentioned and the karma of the users. So books mentioned several times by different people, having high karma will make it to the top. Ive processed about two years worth of comments (you can also see the whole yearly list, which is pretty long - up to 1000 books). I was expecting to see way more tech books but in fact the topics are really diverse like startup, management, parenting, mindfulness, etc. especially when the links dont come from a long discussion about functional programming for example.

Ive added a newsletter in case people want to keep up-to-date and I plan to maintain the service as Ive found some unique books while I was coding the website. There is also search that searches only the submitted books, so this is another good way to browse the collections. Currently there is a newsletter popup, which may be annoying to some users, please click away if that is the case. I will rework it to be less intrusive.

Any recommendations and feedback are welcome. Disclaimer: For amazon links there are referrals, the other links don't have those.

espitia 1 ago 1 reply      
Would love an all-time/yearly list! Also, I see a few comments criticizing the affiliate links. That's the reward you get for building cool stuff. No reason not to!
swalsh 1 ago 2 replies      
There are a lot of books rated 5 stars. They're good reads most of them, but there's another class of books. The kind where even a year later you remember them. They truely changed how you think about a topic. I'm always searching for those top class books, the kind that feel like religious experiences to read. The issue is there's no way to distinglish them from regular 5 star books, the kind where you put them down and say "wow good book", but a year from now you really can't recall much about it..
samwyse 1 ago 0 replies      
Visited the site, dismissed the pop-up, scrolled to the bottom, saw that every book had just one mention. Wound up spending more time reading comments and looking at the list, don't think I'll return as the biggest problems do not seem fixable. You are using Amazon links as a substitute for actual linguistic processing to parse out book names. It may be possible to use the links to prime the pump by creating a list of titles, then re-scanning all comments for closer approximations of those titles but I don't know the fesability of that.
pmorici 1 ago 3 replies      
You should remove the numbering. It gives the impressions of some kind of ranking and I really doubt anyone should be looking to purchase "Palm OS Programming Second Edition"
louprado 1 ago 1 reply      
There are quite a few book recommenders based on HN data. This recent one came to mind:

Top 30 books ranked by total number of links to Amazon in Hacker News commentshttp://ramiro.org/vis/hn-most-linked-books/

I am surprised that someone doesn't solve this problem once and for all by allowing decent search parameters (e.g., all time, last month, by karma, by genre, etc.).

This latest submission weights by karma and bins the results weekly. Those restrictions make it personally unappealing but it is a great first step. I wouldn't have thought to consider karma.

Edit: clarity

tedmiston 1 ago 4 replies      
I wanted to take a second to share the contrarian opinion in support of OP's use of affiliate links.

He/she did a lot of work to make this project and to maintain it. I think they are done tastefully and sensibly in a service that adds value.

jamesfe 1 ago 1 reply      
My feature request is to know the frequency with which these books are mentioned! I love the simplicity of the site though - thank you!
tienthanh8490 3 ago 0 replies      
Nice project, but probably worth adding some natural language processing to avoid things like this from being added to the list:

https://news.ycombinator.com/item?id=11704954 --> "I loved this book for being exceptionally clear and terse. I was hooked from the first sentence: "Probability is a mathematical language for quantifying uncertainty." That one sentence makes the concept clear in a way that the entire chapter on probability from "Statistics in a Nutshell" (http://www.amazon.com/Statistics-Nutshell-Sarah-Boslaugh/dp/...) did not."

--> "Statistics in a Nutshell" got added to list 20 of year 2016

lucasnemeth 1 ago 0 replies      
I didn't knew about "Feynman Lectures On Computation" and was reading the reviews. There's only one 2 star reviews, I clicked to see what it was about. And surprise, it is by Guido Van Rossum! (And he's not that impressed by the book)


leandot 1 ago 0 replies      
Btw you can also list a whole year - http://hackernewsbooks.com/year/2016
thesmallestcat 1 ago 2 replies      
This can't be right. There's zero chance that "Palm OS Programming" is the 16th most mentioned book.
leandot 1 ago 1 reply      
Some stats if people are interested :

- People online right now according to GA is about 500 and has been stable like that for the last two hours or so

- ~80% of the traffic comes from HN, rest is FB, Twitter, etc.

- Last two hours - 9,414 users, 16,338 pageviews

Will write a blog post with more info.

tropin 1 ago 2 replies      
It's an honest question, as I can't see anything in the guidelines: are submissions of webs with useful data but with referrals ok with HN? Just asking in case some day I have a cool idea as this.
Unbeliever69 1 ago 1 reply      
"Palm OS Programming, 2nd Edition" curated content at its best! Is there some seminal work in this book I'm not aware of?
core2pro 23 ago 0 replies      
Absolutely fantastic site I do have to say.Thanks for sharing, I think I will - and did already - find some interesting books to read. Thanks for this repository, I am sure I have found the perfect goal for weekend. :)
bballer 1 ago 0 replies      
Highly recommend "Trust Me, I'm Lying: Confessions of a Media Manipulator", number 18 on the list.

Like others have said, for the love of god please get rid of the popup subscription, or at least make it not alter history.

qwertyuiop924 1 ago 1 reply      
Hey, where's SICP? Has nobody mentioned that and gotten enough upvotes? Because I see it all the time...
_lpa_ 23 ago 0 replies      
I made something last winter (http://www.hnreads.com/), though I haven't fixed a couple of bugs or updated it since. I used named entity recognition rather than links to find book titles, perhaps something you might consider if you wanted to expand the site. I like the idea of showing the top books per time period!
yekim 1 ago 0 replies      
Nice. I'm always taking books that are mentioned in various HN comments and adding them to my reading list. Thanks for making it easy!
marai2 23 ago 0 replies      
I'd like to shamelessly plug my own site here. The itch I was trying to scratch was to have an HN like site for books that could lead to discovering and promoting higher quality books and better reviews than the average review at Amazon.


fitzwatermellow 1 ago 2 replies      
What no "Zero to One"? No "Meditations of Marcus Aurelius"? Where's the "Innovator's Dilemma", or "Hard Things", or "Lean Startup", or "Creativity, Inc.", or "Black Swans", etc, etc, etc ;)
kol 1 ago 1 reply      
Great idea! It'd also be great if someone created hackernewspapers.com too. There are so many great papers mentioned on HN. Since most links point to pdf files, it'd be possible to collect the pdfs on the same site (if there's no copyright violation).
iplaw 1 ago 1 reply      
Let us know how the Amazon referral program treats you.
mikecope 19 ago 0 replies      
OP, great site. Curious whether you've already been approved by the amazon affiliate program. (For others not in the know, amazon will only approve your site once you've completed your first sale.)

I ask because I was recently rejected for my application (http://addonbuddy.com/). Their reason was "lack of original content." Here's part of the message I received:

"A part of our criteria is that your site has to be established with enough unique content. We rejected your application due to one or more of the following reasons. - Lack of content which is original to your site and beneficial to your visitors - Pages that are mainly empty when advertisement content is removed"

I like your site and think it offers a lot of value, but Amazon's affiliate program seems to favor blogs that have a lot of content. They can also be real sticklers about including certain text in your website indicating that you're part of the affiliate program.

gtrubetskoy 1 ago 0 replies      
I see one of the books at the top links to a comment of mine, but your service would get the Amazon referral for it. Which I don't care about entirely, it just got me thinking - do we presently have a technology that would make it possible for us to split some sort of a micro-credit/payment - I think if such a thing existed, that would be quite revolutionary. People could finally get paid for all the things they refer to, not to mention ads on their blogs, etc.
samblr 21 ago 0 replies      
Since HN discusses about books time and again - will it be a good idea to have a group for HN in goodreads ?
Jaruzel 22 ago 0 replies      
I quite like this (not that my opinion really matters tbh), but a thought does occur to me - any chance of putting the title of original post on HN that contains the referring comment - i know you can click to it via the comment link, but seeing the post title alongside the book would make for interesting browsing.
kobayashi 1 ago 0 replies      
Would you mind adding TLS compatibility? The free LetsEncrypt CA is, of course, an easy way to do so.

Great site, looking forward to the new reads.

samblr 23 ago 1 reply      
Can anybody comment on #1 in list : Mother Nature: Maternal Instincts and How They Shape the Human Species
kozikow 1 ago 0 replies      
Recommendation engine based on (occurs in same base post) or (suggested by the same HN user) could be interesting as well.
ultramancool 22 ago 0 replies      
This can't be top... there's so few technical books and a book about Palm OS programming is showing up here.

Is there an 'all time' list that would be better? This contains a lot of garbage that got referenced once in an argument (and often a bad one at that).

pieterr 1 ago 1 reply      
Nice site! Maybe add links to reviews of the books too?

Example #17:http://eli.thegreenplace.net/2009/10/07/book-review-c-interf...

hansc 1 ago 1 reply      
Funny that the first book in the list "Mother...", I cannot find when using hacker news search!?
georgiev 1 ago 1 reply      
But there's also a lot more books that are mentioned without the Amazon link. On the top of my head "The cathedral and the bazaar" is probably mentioned zillions of times. Are you thinking of handling that as well?
Aldo_MX 1 ago 2 replies      
The "Hacker News Books" title raised too much my expectations, I was expecting books which talked about entrepreneurship, technical topics and good practices, not "Selfish Reasons to Have More Kids"...
DanBC 1 ago 0 replies      
I love this.

I think it gives too much weight to me.

Look at 2016, and there's a bunch of books I've linked to. I believe these are great books, but it's a bit weird to see them so high on the list.

nojvek 19 ago 0 replies      
Having a top year, month would be a 100x more attractive for me. Weekly is too much churn to figure out what is good.
cholantesh 1 ago 0 replies      
Not sure if it's an anomaly or a repeating issue, but I notice that in week 30, Exploring Expect is listed twice, at positions 11 and 12.
magic_beans 1 ago 0 replies      
This is amazing!!! Great idea, and really useful!
uberneo 1 ago 0 replies      
Nice aggregation -- Would be handy , if the books available will be by Category and Source (Amazon , Safari, O'Reilly)
questionr 22 ago 1 reply      
some (mostly technical) books are mentioned in shorthand (eg. k&r and gof).

how are those accounted for?

tmaly 23 ago 0 replies      
something seems off, Palm OS programming is mentioned as #16. This does not make sense.
aaron695 1 ago 0 replies      
Canonical Lists.

I remember loving them then going to hate.

Guess it's where you are in life.

RodericDay 1 ago 1 reply      
Terrible pop-up subscription panel hijacking my back button.
davb 1 ago 2 replies      
The popup asking me to subscribe is an annoying trend I'm seeing on a lot of sites. It's the first time I've been to the site, why on earth would I sign up for anything without even knowing if the site is any good or of any interest to me?
vortegne 1 ago 3 replies      
Great website, but the popup subscription window is terrible.
jszymborski 23 ago 2 replies      
Please do not ever pull that "Subscribe" pop-up stuff.

Pleeasse don't do it... especially on a website whose target audience is the HackerNews folk, because they will not put up with it.

Those pop-ups are jarring. I think it's great you've got an affiliate link, and I get you want to get as many recurring eyeballs on this as possible, but those pop-ups are everyone's pet peeve... it's the MIDI autoplay, MARQUEE and BLINK tag of websites today.

Edit: Removed capitals for emphasis and adjusted tone to better communicate my intention and comply with HN Guidelines.

July was the hottest month ever recorded, according to Nasa nytimes.com
381 points by MarkEthan  4 ago   501 comments top 35
harryh 4 ago 30 replies      
Personally I think that the idea that these trends can ever be revered by some sort of world wide agreement to lower consumption is a pipe dream. People want to eat meat and drive cars and fly on airplanes. As long as they're rich enough to do so you'll never stop them.

The only solution to this problem is technology. Some combination of green energy sources plus probably very large scale terraforming.

codecamper 4 ago 3 replies      
Solar panels now sell for about 45c a watt out of China. Cost to create is around 36c / watt.

This allows deployments such as in Chile & Dubai where the cost/kwh is 3 cents & below.

This now cheaper than coal, without subsidy.

Remove the subsidies that oil, gas & coal receive, and you will witness a new day.

Only problem for solar right now is over supply caused by a sudden drop in demand. China reduced its subsidy a bit. Also the US extended subsidy for 5 years, which is great, but it has removed the urgency to complete utility scale solar plants before subsidy cutoff, causing the utilities to sit it out & wait.

Better act soon! (250 years to clear CO2 from atmosphere, feedback loops only now being discovered.)

sevenless 4 ago 6 replies      
These maps are based on emission trends continuing, assuming no political action will be taken. But wouldn't a hotter world tend to exacerbate emission trends, with human needs for AC and desalination rising, plus forest fires? Moreover, there might be some tipping point for large scale methane release. The 'tail risk' to global warming has appreciable density a long way out.
spdustin 3 ago 7 replies      
I've come to the (I feel) inevitable conclusion that we are already well and truly fucked. I will still do what I can to slow the inexorable slide into worldwide famine, war and anarchy, in the hopes that the majority of my children's lives will be made incrementally better, should they choose to remain on this planet (and, it is assumed, that they may actually have a choice, but there's no way I can know).

We're on the wrong side of the story arc here. It's too late. The damage has been done, and it will continue to snowball even if we stop putting carbon (as both CO2 and methane) into the atmosphere today. Anyone who cooks can instinctively understand how "carryover heat" works - the temperature gradient hasn't yet finished equalizing, the latent energy (and, for that matter, the latent GH gasses) stored in the oceans and, indeed, the land itself hasn't yet made its full effects felt, and we've successfully argued and argued and argued about it without actually doing anything substantial enough to make a meaningful difference.

But that sort of hopeless talk makes me sound like a crackpot, and I've typed this sort of message into many a text field on the Internet over the past year before deleting it, or pressing cancel, or closing the browser tab. But today, in a moment of vulnerability, or for another reason I can't hope to know, something is telling me to press "submit".

bjourne 4 ago 1 reply      
Anyone else who feel that the climate-related news are getting very scary? If for every month for the last year, there was a new sprinter breaking the last months world Record in 100 meters you'd know something was fishy. Like some kind of incredibly potent anabolic steroid runners were ingesting. With new and improved versions of that steroid being released every few months...
davidf18 4 ago 0 replies      
There is much we can do in the US to decrease air pollution within a very short time. These reductions also contribute to lowering asthmatic episodes and hospitalizations and various forms of heart disease in the elderly, especially women.

1. As of a few years ago, 10% of the 450 coal powered electric power plants produced half the air pollution by coal powered plants (according to an Environmental Pollution Agency report). Take the top 45 polluters off line. The Obama administration is already to shut down some of these plants.

2. Many buildings in the Northeast and Midwest burn old #6 and #4 fuel oils for heat during the winter which are very polluting. New York City, where I live, has now banned the dirtier #6 oils, but politics has intervened and significantly delayed the banning of #4 oil. But other cities should ban the burning of these dirtier fuel oils.

The shutting down of the top 45 polluting coal powered electric plants and the banning of #6 and #4 fuel oils for heating buildings would not be hard to do and would make a significant difference to reducing air pollution.

triangleman 4 ago 1 reply      
El Nino + urban heat island effect. According to satellites, July 2016 is the 2nd hottest July since the 70's when satellite records began. The hottest is July 1998, also El Nino.

Take a look at weather stations that have been situated in the same place for 100 or more years, far from cities:http://www.john-daly.com/stations/stations.htm

kpwagner 3 ago 2 replies      
Sergey Brin made a statement regarding global warming: something like, "I'm not worried about it. We can put solar-powered carbon scrubbers in the deserts of the world and essentially fix the problem with an investment in the tens of billions (relatively small)."

I forget where I heard this and wasn't able to dig up a source. I think it was a Jason Calacanis podcast or video of an interview. Either way it was secondary information. Bad sourcing, I know--sorry.

I agree with many who suggest reducing consumption is a risky gambit at best. How practical are counter-measures? Can we just geo-engineer our way out of it?

throwanem 4 ago 4 replies      
It never ceases to bewilder me that nuclear power isn't taken seriously as a replacement for fossil-burning baseload generation.
blondie9x 3 ago 1 reply      
We all know by now CO2 and CH4 leads to a warmer planet. We also know what's driving greenhouse gas levels to rise across Earth. Contributors are deforestation, intensive animal farming, and primarily the combustion of carbon fossil fuels like coal, tar sands, oil, natural gas etc. But here is the underlying problem, despite us knowing how bad things are, (97+% of scientists who study this field agree we are causing the planet's climate to shift away from the temperate climate we thrived in) not enough is being done at present to truly solve the problem.

What really is disheartening and what no one in the media and government is talking about is how in 2015 CO2 levels rose by the largest amount in human recorded history. 3.05 PPM http://www.esrl.noaa.gov/gmd/ccgg/trends/gr.html We are being lied to and mislead by our governments that uniform actions are being performed to save the planet for the future of man. Vested interests in the fossil fuel industry continue to drive climate change. Yes, solar and wind energy is starting to become incredibly efficient and cheap but not enough of it is coming online in proportion to fossil fuel burning that persists and is also installed annually. If we do not rally against it, our ability to live on this planet is at stake. The lives of our posterity are also at risk because of the burning. It will not be until we take extreme actions not on a country level but as humanity together that we will slow the burning and save ourselves.

What are these actions you might ask that will actually be effective? These can range from banning fossil fuels entirely, global carbon pricing system, banning deforestation, changing human diets, extreme uniform investment in renewable energy and potentially fourth generation nuclear reactors, more funding for developing nations to install alternative energy sources, and to shift the transportation grid towards sustainability.

bcarrell 3 ago 5 replies      
I own a home built in the 1960s in the northeast US. My home (like many others, I'm sure) uses an oil-fired boiler to provide heating and hot water. I don't have gas lines to my house. The gas company won't run them unless I pay ~$5,000 for them to do so. Even if my home used natural gas instead of oil, I feel like it's not really solving the problem.

What are my options here? Is solar viable? What can _I_ reasonably do to improve consumption? I use only LED lighting, eat meat rarely, use a programmable thermostat, have new windows/doors, etc. Improving the situation seems either not economically viable for most people or an incremental improvement. Just wait?

arethuza 4 ago 0 replies      
Found this link with similar maps for Europe:


NB from 2012.

Interesting that it actually predicts that the UK will have a distinct benefit from warming when it comes to agricultural productivity.

Karellen 4 ago 0 replies      
Phil Plait of Bad Astronomy also has a good (if now depressingly repetetive) take on it[0].

[0] http://www.slate.com/blogs/bad_astronomy/2016/08/16/july_201...

aangjie 4 ago 0 replies      
This graphic could've used a Brett victor style tangle.js based simulation model. Then we could play around with it assuming future emission rates going down or up etc.. http://worrydream.com/Tangle/
cconcepts 4 ago 2 replies      
I initially thought this was a NYTimes self promotion piece. As in "Think NYTimes is hot right now? Just Wait".
AdmiralAsshat 4 ago 0 replies      
Two weeks from now: "August was hottest month ever recorded, according to NASA."
brooklyndavs 3 ago 0 replies      
I'm just a software developer with a side interest in climate science so please excuse any ignorance on this topic. What I find troubling is it seems, from what I have read, our climate models currently have a poor handle on co2 and methane feedbacks, both positive and negative. For example there seems to be a general scientific consensus that NOx from industrial sources (mostly coal power plants) is having some cooling effect. When these are removed, which will happen slowly but steadily as coal is retired, there will be a warming response but it is not clear how much this response will be. Similarly, you have positive feedbacks coming into place like the decline of sea ice in the arctic, less forests and more combustion of those forests, and co2/methane releases from permafrost. These are other feedbacks are known but nothing I have read has convinced me we have a full scientific understanding of the impact that these feedbacks will have.

Thats why like others in this thread I feel rather hopeless about humanity ever getting global warming under control. We of course have the problem of humanity to continue putting greenhouse gases into the atmosphere via our economic activity. This of course is co2 but also methane and HFCs. The challenge of bringing these down to safe levels while keeping not only the standard of living we enjoy currently in the west but also bringing billions more people into the middle class world wide is impossible with existing technology and incredibly hard with emerging technology. To get to a carbon neutral prosperous, middle class society for everyone on the planet will take many decades. If we ever get to that point I'm afraid feedbacks and built in system inertia will be so strong that the planet will keep on warming for 1,000s of years despite our best efforts.

TL;DR I think we as humans really screwed this up and I don't have much hope of us collectively being able to fix it.

artursapek 4 ago 3 replies      
I didn't know if it was just because I now have a 2 year old, but New York summers have definitely seemed harder to deal with each year I've lived here.

Is there a commonly accepted minimum temperature difference threshold that is perceivable by humans? E.g., if summer in 2016 was hotter than 2015 by X, I will notice?

Zigurd 3 ago 0 replies      
The answer will probably be complex and unpleasant: We'll get serious climate disruptions before people get serious. We'll need to choose to deliberately forego energy sources, and energy is tightly linked to economic growth. The nuclear proponents will have to demonstrate that nuclear power can be economical and safe. If it is not economical, as it has not been so far, it is useless.

If we are past the point of preventing large scale calamity, geoengineering will probably have to be part of the solution. It's very risky, not least because it will lead to complacency if it works at all.

novalis78 4 ago 0 replies      
and here we are, still without any thorium reactor anywhere on the planet in operation.
chukye 4 ago 5 replies      
OK, but... any ideas how can we solve that?
misja111 4 ago 4 replies      
So 14 of the 15 hottest years ever have taken place since 2000. I'm wondering, if this is the result of global heating due to emissions, why do we see so many new record highs only now?The industrial revolution has been going on already for over 150 years. Why were there so few records broken in, say, 1980 - 1996, and so many in 2000 - 2016?
tombert 4 ago 4 replies      
The part I find especially frustrating is that, on average, a New Yorker has a lower carbon footprint than the typical American, probably due to the heavy use of the train.

So even when I do the (I think) eco-friendly thing by not having a car and using public transit, I'm still being punished for what the rest of the US (and the rest of the world) does.

return0 4 ago 0 replies      
As someone who does not tolerate heat well, i can confirm. And it isn't over yet.
kelvin0 4 ago 0 replies      
Well clearly it has nothing to do with man-made pollution, it's nature taking it's course and these rampant wildfires. Oh wait, what causes the wildfires you ask? :-)
mfer 4 ago 0 replies      
Not to negate the issue at hand, which is important, there there's an interesting note on perception here.

"ever recorded" is used but how long have we been recording? The last study I checked on that made a claim like this was about 100 years. "ever recorded" sounds more sensational.

In any case, how we treat our planet... our ecosystem... is important. Seeing it go downhill is saddening and motivating. Glad to see things pointing out the change, its direction, and possible impact.

izzydata 3 ago 1 reply      
No wonder my electricity bill was so insane. Maybe this winter will be the coldest winter ever recorded.
elcct 4 ago 0 replies      
I planted palm trees and bananas in my London garden to reflect that :)
p4wnc6 3 ago 0 replies      
I took off all my clothes.
webXL 3 ago 0 replies      
Where, New York Times?
factorialboy 4 ago 4 replies      
bnolsen 4 ago 3 replies      
pedro2 4 ago 0 replies      
staticelf 4 ago 3 replies      
I am lucky I live in the northen parts of Europe. I do not wish to live in America in ~50-100 years from now since there is so much guns and other weapons easily available.

People who care about their childrens future should start planning now. I think when the shit hits the fan it will hit hard and fast.

gadders 4 ago 1 reply      
It looks like the actual increase was minimal and within the margin of error:

NASA and the National Oceanic and Atmospheric Administration (NOAA) announced this week that according to their calculations, July 2015 was the hottest month since instrumental records began in 1880. NOAA says that the record was set by eight one-hundredths of a degree Celsius over that set in July 1998. NASA calculates that July 2015 beat what they assert was the previous warmest month (July 2011) by two one-hundredths of a degree.But government spokespeople rarely mention the inconvenient fact that these records are being set by less than the uncertainty in the statistics. NOAA claims an uncertainty of 14 one-hundredths of a degree in its temperature averages, or near twice the amount by which they say the record was set. NASA says that their data is typically accurate to one tenth of a degree, five times the amount by which their new record was set.


Evidence points to another leaker at the NSA reuters.com
376 points by mudil  4 ago   229 comments top 25
silvestrov 4 ago 3 replies      
Even if it's not Russia, the release of the hacking tools gives weight to the argument that nsa/fbi should not be able to demand companies to create a back door in their products.

> Apple: If we're forced to build a tool to hack iPhones, someone will steal it.

> FBI: Nonsense.

> Russia: We just published NSA's hacking tools


adamc 4 ago 4 replies      
I don't think calling a thief putting up hacking tools for the highest bidder "another Snowden" is particularly accurate. (I realize this was the article title and am not complaining about the post here. I fault whomever crafted the original title.)
tptacek 4 ago 4 replies      
Where by evidence for another NSA leaker, Bamford means literally no evidence. The fulcrum of his op-ed rebuts an argument nobody is making --- that Snowden himself disclosed this cache of exploit tools.

The prevailing narrative, one echoed by Snowden himself, was that this was likely taken from a staging server: a machine somewhere out on the Internet used as a pivot point for attacks. Snowden claims (I don't know with how much authority) that a compromise of one of those staging servers is not without precedent.

Nothing in this entire piece refutes or even engages with that narrative.

John23832 4 ago 5 replies      
> In addition, if Russia had stolen the hacking tools, it would be senseless to publicize the theft, let alone put them up for sale. It would be like a safecracker stealing the combination to a bank vault and putting it on Facebook. Once revealed, companies and governments would patch their firewalls, just as the bank would change its combination.

Why would this be bad for Russians (if this was indeed the Russians)? We can/should assume that Russia has it's own methods of infiltrating systems. The value of this data to them would be knowledge of how it's done, not necessarily hoarding and replicating how the NSA does it. If anything, having vendor's patch exploits that they're not using, but their enemy is, would be a great chess move.

advisedwang 4 ago 0 replies      
It seems likely (https://www.schneier.com/blog/archives/2015/04/counting_the_...) that there are actually quite a few whistleblowers/leakers out there.
redwards510 4 ago 1 reply      
Do people really think this "auction" is legit? That people can seriously bid on these tools? The whole thing plainly sounds like a big joke for maximum attention. There is no real intention to sell the tools, at least not this way. The terms of the auction are ridiculous; no one with enough money to make a serious bid would risk losing it all like that.

Perpetuating the meme that this is a serious auction is dangerous and faulty journalism. It is a publicity stunt to embarrass the NSA. Let's not get hysterical and pretend that some third world terrorist country could obtain the NSA's cyber capability by bidding all of their petro-dollars in this farce.

dredmorbius 4 ago 4 replies      
It's inevitable that there are, or were, or will be, other Edward Snowdens working at NSA. Persons who find that the Agency's mission no longer sits well with them.

The question is: who are they working for?

Snowden was working for the American People, and upholding the US Constitution.

To draw from some relevant if non-US history, Kim Philby's interests did not lie with his nation's subjects, despite his aristocratic pedigree.

okket 4 ago 1 reply      
Edit: Apparently somebody agreed and removed the clickbaity 'Snowden' from the title. Thanks.
lostlogin 4 ago 0 replies      
The article sort of makes fun of the hackers writing, then immediately writes "loosing" instead of "losing".
zmanian 4 ago 1 reply      
Bamford's expertise in espionage is pretty similar.

There seem to be two plausible explanations for the Shadow Brokers release.

1. The doctrine of the US govt in cyberwar is proportionate response. This is either preemption or escalation on behalf of Russia. This assumes the attribution of Russia for Democratic political hacks are accurate.

2. This is further activity by whomever leaked the ANT catalog to Applebaum.

The Shadow Brokers are going to be difficult to attribute technically. Attribution is based more on your theory about what's happening the Russian covert escalation.

grandalf 4 ago 0 replies      
Once a useful zero-day has been discovered by an adversary, it may make sense to give up using it so that one's own side's computers are not vulnerable.

My guess is that the NSA has excellent methods for detecting DNS exfiltration and the recent tools are at least a decade old technology.

What's interesting is the disinformation value of intentionally releasing the tools, but to understand that we'd have to know who the intended adversary was.

bogomipz 4 ago 0 replies      
"In addition, if Russia had stolen the hacking tools, it would be senseless to publicize the theft, let alone put them up for sale."

No not all Russia was recently accused of hacking the DNC in the US, so it would a perfectly logical for one state actor to say to another "you do it too and here is evidence." Is that not so obvious?

PieterH 4 ago 2 replies      
As soon as the author began referencing Applebaum seriously, the article lost most of its credibility.
justcommenting 4 ago 0 replies      
Does anyone else recall Jacob Appelbaum referring to RC6 constants [0] in public talks (e.g. 30c3) back in 2013?

I'm surprised Appelbaum hasn't been directly suggested as a potential source[1,2].

[0] https://securelist.com/blog/incidents/75812/the-equation-giv...

[1] http://crypto.stackexchange.com/questions/12661/what-could-l...

[2] http://www.nakedcapitalism.com/2014/01/jacob-appelbaum-30c3-...

atmosx 4 ago 0 replies      
> A more logical explanation could also be insider theft. If thats the case, its one more reason to question the usefulness of an agency that secretly collects private information on millions of Americans but cant keep its most valuable data from being stolen, or as it appears in this case, being used against us.

If there's anyone still questioning the results of Snowden's move, here you have it. The Reuters opinion is stating that this data is potentially being using against us. If the perception that the NSA doesn't collect for the public good becomes broadly accepted, change can be achieved at a political level.

bronz 4 ago 2 replies      
i dont understand why the highest bid in the shadow brokers auction is less than a thousand dollars
return0 4 ago 0 replies      
Why don't we have widespread use of tools that spoof online activity ?
partycoder 4 ago 0 replies      
Or probably it's information intoxication (leaking fake information)
Aelinsaar 4 ago 0 replies      
...It's like they have no idea what "Evidence" actually means, or frankly what the hell Snowden actually did; it wasn't selling hacking tools on the black market!
progressive_dad 4 ago 4 replies      
sickbeard 4 ago 1 reply      
Isn't this the reason why governments react harshly to people like Snowden and Manning? To discourage even worse breaches?
nxzero 4 ago 1 reply      
Anyone that expects anything related to "intelligence" to have logical explanations might very easily find themselves chasing tails.
the_wumpus 4 ago 0 replies      
One hundred Royal Zorkmids for the title author!
thomasrossi 4 ago 1 reply      
"Its one more reason why NSA may prove to be one of Washingtons greatest liabilities rather than assets." WOW, never though I'd read that on something like Reuters, he put his foot down.. :s
SixSigma 4 ago 2 replies      
This is all smoke and mirrors.

The DNC is claiming a Russian Hack with wonderful support from the media. Reuters, for instance, gave over $1m to the Clinton Foundation.

When the FBI accused the N. Koreans of the Sony hack, at least there was some credible evidence conjoured up. Obama used an executive order to apply more sanctions on NK even though there were voices saying it was still inconclusive.

But now we are expected to believe the DNC has been hacked by the Russians in partnership with Trump.

Seth Rich, supposed DNC leaker gets shot in the back.

And now this.

The DNC stabs Russia in the back - Bill was happy to accept $500,000 for a speech in Moscow and the Clinton Foundation $millions just before Hillary authorized a major Uranium deal.

I sound crazed writing this, like I'm something from InfoWars. This election cycle is standing the world on its head. Be very careful who you believe.

University of Chicago: We Do Not Support So-Called Trigger Warnings time.com
421 points by ourmandave  1 ago   420 comments top 37
basseq 1 ago 10 replies      
This is tough.

With the aspirations of "civility and mutual respect" along with "rigorous debate, discussion, and even disagreement", I would hope any truly egregious edge cases disappear.

In the middle, I think people have the right to remove themselves from discussionscivil or otherwisethat they don't want to listen to or engage. But that right doesn't limit others' ability to have those selfsame discussions in public or private venues, nor require others to give you advance warning about it.

And finally, U. Chicago has the right to set degree requirements. If you are unable or unwilling to meet those requirements (e.g., not participating in discussions), then you have no place at that institution. After all, the "real world" won't hold your hand.


Trigger Warnings ("A trigger warning is advance notice about subject material that may be difficult for certain students to read, hear or see.") U. Chicago says they "don't support" them. I think actively discouraging them in every case seems extreme, but the institution simply can't be required to provide trigger warnings in every case for every individual. In other words, a lack of trigger warning doesn't mean you won't be triggered.

Safe Spaces ("A safe space is a place they can go to avoid those subjects or heal after confronting them.") U. Chicago doesn't condone the creation of safe spaces. Similarly, I think this stance is extreme. Safe spaces may be necessary for individuals. But U. Chicago is under no obligation to provide them for you or excuse you from other discussions as a result.

In other words, U. Chicago isn't a place for people who don't want to discuss issues that make them uncomfortable. Which, on its face, seems reasonable to me.

superuser2 1 ago 5 replies      
It's important to acknowledge that the term "safe space" means different things to different people.

To the Yale protestors (who were widely supported at UChicago) the entirety of university housing at Yale was meant to be a "safe space," so it was incumbent on the university to fire professors who voiced unwelcome opinions in that space.

To many of the people who are angry about this letter, "safe space" means a support-group-like room in a specific place at a specific time, which doesn't interfere with the rest of the university's operations.

I suspect it is the former which Dean Ellison meant to attack. I find it hard to believe that he's trying to shut down, for example, group therapy programs for LGBTQ youth offered by the psychology arm of the university's student health system. But I do appreciate him sending the message, "we're not going to fire professors because you don't like what they said."

jpfed 1 ago 9 replies      
What I don't get about this is why trigger warnings and safe spaces always seem to get lumped together in discussions like this.

It's really hard to actually create a safe space. You have to actively intervene to prevent (or provide redress for) the "wrong" kinds of speech. I can totally see why a university would not think safe spaces were worth it, or even consistent with their goals.

But trigger warnings are a really simple common courtesy. There was a popular meme spread around the Fourth of July to be considerate of veterans who may be distressed by the sound of fireworks. If you think this is a consideration worth extending to a veteran, it seems perfectly consistent that you'd want survivors of rape or abuse to know that a discussion of rape or abuse was coming up.

jordigh 1 ago 5 replies      
One thing that bugs me is that people think trigger warnings are to make sure people don't get offended. Being triggered is not the same thing as being offended. Being triggered is about having a fucking panic attack, hyperventilating, feeling like you're having a heart attack and you're about to die.

Panic/anxiety attacks are not a silly little thing. They were portrayed somewhat accurately in Netflix's recent Jessica Jones (she even uses a common grounding tactic to prevent them, like street names from childhood), and in Love and Mercy, that biopic of the Beach Boys' frontman. Having had close experiences with people who have panic attacks and see them getting triggered by being reminded of certain incidents, I can tell you that panic attacks are not fun, and those suffering from them are not faking, nor are they being whiners. With time, it's possible to get over your triggers, but it's a long healing process. During healing, trigger warnings are the same courtesy as signs saying that people with heart conditions or expectant mothers shouldn't get on certain carnival rides.

But people now on the internet say stupid shit like "triggered much?" for what was once called "u mad bro?" And whether exaggerated or not, it seems that this kind of "triggering" is what UofC seems to be speaking out against. That's not what a trigger warning is for.

mankash666 1 ago 3 replies      
How did universities evolve from spaces meant to evoke controversial and provocative thought (like the birth of the free speech movement at UC Berkeley) to the "safe spaces" of today? If a university is meant to enrich you, one worthy lesson is the ability to "agree to disagree in a cordial fashion". If you disagree or feel threatened by a line of thought, make your voice heard. Receding into a safe space won't prepare you for the real world.
stephenitis 1 ago 2 replies      
Safe spaces and banned trigger words creates the tendency for people to silo themselves in a echo chamber of their own ideas. This manifests itself later on in our population in the form of filtering friends and news who dissent against their opinion.

The rise of these recent techniques of protest are more a failure of our high schools and universities to provide sufficient visibility and importance to debate and discussion.

jackcosgrove 1 ago 2 replies      
There's a subtle point that's getting overshadowed by the lightning rod discussions about trigger warnings and safe spaces.

The U of C is explicitly stating in this letter that if you don't agree with their values, don't come to the school.

This seems uncontroversial to me, but I think it's worth affirming that self-segregation is a valid social strategy. I know segregation is a loaded word, but that's because historically it was caught up in the exercise of power differentials.

Robert Putnam concluded (http://archive.boston.com/news/globe/ideas/articles/2007/08/...) that diverse societies actually see a decline in civic life because there is more friction and distrust between individuals with different values. So there is such a thing as too much diversity, and diversity must be balanced with cohesion.

Moreover, diversity seen as social mixing is self-defeating since after enough social mixing everyone is the same and there is no more diversity. There is a happy medium whereby people can be tolerant and different at the same time. Indeed, tolerance and difference need each other to exist.

Segregation is abused when there is an exercise of a power differential, whether between individuals or groups.

So I'll strike the balance by saying, "You have the right to segregate yourself from others and hold exclusionary values as long as you do not use force to impose those values on others."

If you disagree with this statement, I'd ask you to examine how many ways you segregate yourself from others. Do you befriend anyone? Do you date anyone? Do you work with or hire anyone? Everyone engages in this behavior because we are social animals and shared values are necessary for society.

In the long term perhaps human society will become universally shared, at which point many of these controversies will be moot. However that is not the case now so we should allow communities of like-minded individuals to form and share values that differ from the values of others. Indeed we should always allow this, because there will always be minorities no matter how homogeneous the dominant culture is.

seibelj 1 ago 2 replies      
Some 18 year olds fight in wars and die. Other 18 year olds cry and throw tantrums when someone says a word that they don't like hearing. The world we live in!
rdtsc 1 ago 2 replies      
I remember someone a while back posted a lecture page from CMU. They support Trigger Warnings. And a few papers there had them.

It was a database course, I can only imagine "master/slave" terminology.

So I was wondering, how are Trigger Warning supposed to work? Can the student refuse to do the required reading and claim full credit because might be triggered, seems like it would open loophole.

That was just CS stuff. What about history, that's full of prejudiced offensive language and terminology. Can students avoid doing required work by claiming they'd be triggered?

Is there anyone undertaking an effort to sanitize records to "fix" the triggers. For example take some classic distributed system papers and search and replace master/slave with leader/follower?

I can understand wanting to have safe spaces. But I don't quite follow how trigger warnings resolve themselves in the context of classroom learning. Any recent grads have any experience with how this is handled?

EDIT: I think it was a link like this (even perhaps this exact course, just for last year).



Concurrency Control III Optimistic /!\


/!\ : Trigger warning: The material presented in this lecture uses explicit language and discusses certain situations in database management systems that may be triggering to some students.


Are students allowed to pass without learning and being tested on Optimistic Concurrency Control in databases? If not, what is the purpose of the warning, just to have them mentally prepare for it?

chias 1 ago 0 replies      
> "When you read something that makes you angry, stop for a moment and answer two questions: who wants you to be angry, and why?"

Something my father once told me, that has served me well over the years.

There's a lot of money (read: clicks) to be made by manufacturing outrage, and there's far too much to be angry about in this world to be inventing any more of it. Maybe this is something you should be angry about, maybe it isn't. But unless you like being mad you owe it to yourself to think about those questions before making a decision.

For this thing in particular, it doesn't make any sense without context. I'm not at the University of Chicago, and nor is anybody I know. Are "sjw pc trigger-warningers" a problem there? Or is the university being ridiculous? We can argue back and forth forever whether "safe spaces" are good or not depending on your definition -- which definition is the one that is relevant at the University of Chicago? This letter may make perfect sense in the context of current issues going on at the university. Or it may be a load of bull. But, at least from where I'm sitting, I have no context. And whoever uploaded this didn't provide any. What was their purpose? Who knows. Definitely not enough justification for me to worsen my mood about it.

sergiotapia 1 ago 0 replies      
Safe spaces used to mean "say anything you want, face no judgement."

Now it means, "don't say anything that might offend someone."

mathattack 1 ago 0 replies      
This seems like a good way to define what makes the school different. If you like this, go there. If you don't like it, you can go somewhere that does have trigger warnings and safe spaces.
danso 1 ago 0 replies      
I'm teaching a journalism class this fall in which I've made David Simon's "Homicide" a required book. It's a terrific book overall, but I chose it because of its examination of the system as a whole. But as you can imagine, a book that involves following the Baltimore homicide department around unavoidably gets pretty grisly, even with Simon's treatment of it (that is, in the way The Wire is quite possibly one of the most boring shows ever about police work, while being also the best show on TV ever). There's lots of details about murders and rapes (the central event involves the rape and murder of a child).

That said, it is a journalism class, and students who actually become journalists should expect to report on such traumas on a fairly regular basis. So I don't feel that concerned about using the book for journalism class. I'll probably warn students not to read the chapter about autopsies while eating. If I were in another liberal arts/humanities departments, I might take pause (but I have no idea what it's like to be an instructor in any other departments, so don't take my opinion as having any insight to other departments or their proclivities)

gragas 1 ago 0 replies      
I'm not sure why that video clip was included in this article.

I expected the girl in the video to make a clear argument for or against trigger warnings and safe spaces, or at the very least provide a overview of the situation and how trigger warnings have affected her. In the end, it just seems to be an almost completely unrelated video about a common traumatic event.

I don't, in any way, mean to downplay the atrocities of rape and the unfortunate case of the girl in the video. But it is ultimately an appeal to pity and does not really present an argument. Triggering was hardly mentioned.

pklausler 1 ago 2 replies      
Bad move, Chicago. If the kids don't learn how to use trigger warnings and safe spaces at school, how will they know how to use all of the trigger warnings and safe spaces in the real modern world after they graduate?
aeturnum 1 ago 0 replies      
I have never understood the discomfort around trigger warnings. Society has used them forever in areas that are considered sensitive. When the news warns you they're going to show graphic footage or that a story involves sex - those are trigger warnings. It's simply displaying common minimum level of empathy for the experiences of others.

Safe spaces are more complicated and (inherently) restrictive and so I understand the discomfort around those too. You can allow controversial / awful groups to exist (Nazis, KKK, etc) and provide trigger warnings for their events. Indeed, I would think that having 'opposition' groups and hosts provide their own sets of trigger warnings would be a fantastic introduction to outsiders.

qwertyuiop924 1 ago 0 replies      
As someone who has difficulty listening to many subjects (I am extremely prone to depression) I still think this is a good move: If you have difficulty with a topic, pretending it doesn't exist won't help you.
davidf18 1 ago 2 replies      
At a time when many prestigious private schools had Jewish quotas for students and faculty, the U of C did not.

There should never be bans of free speech consistent with that of the First Amendment (i.e, you can't shout "Fire" in a movie theater).

I hope that there is a law passed that universities that block free speech as many have, are not allowed to receive federal funds in the form government grants for faculty, and financial aid for students.

mox1 1 ago 2 replies      
Am I missing something, or doesn't safe space just = Library / Dorm room ?

Maybe add some headphones so you can't hear anything....

brantg 1 ago 0 replies      
U Chicago's letter is essentially a single, all-encompassing trigger warning to the incoming freshmen.
louprado 1 ago 0 replies      
My concern with all the attention given to college based safe spaces, trigger warnings, rape culture, social justice, weed culture, athletic program scandals, etc. is that practical minded children from working poor families will just assume college is an alcohol-fueled summer camp for rich kids and jocks that doesn't teach real world skills.

That's what I thought at 16 after seeing Animal House and having never visited a college campus. Fortunately someone insisted engineering is a practical degree that I should consider. But had that person not influenced me, I would have just went to Apex Tech which gives you a box of tools and a "diploma" in 6 weeks.

scotty79 1 ago 0 replies      
When an irrational reaction to common stimuli stopped being a concern of only the person afflicted and their doctor?
cpeterso 1 ago 1 reply      
The University also seeks to eliminate so-called 'spoiler warnings'.

54 percent of students said the climate on campus prevents some people from saying what they believe because they are fearful of looking like assholes.

Swizec 1 ago 1 reply      
This is a really hard topic to think about.

On the one hand, I worry that we're moving towards a mind-police driven society where we actively discourage people from saying anything that goes against the grain of society. Be it by activist groups, or algorithms optimizing for clicks and user retention.

On the other hand, it is important to be courteous to each other and not upset people needlessly. Discussions should give people the option to bud out and not participate, if they feel they cannot do so.

On the third hand, maybe we've always lived in a society where we actively discourage participation by anyone who is perceived as upsetting the group and nothing new is happening.

Then again, I think people should be actively encouraged to entertain stances and opinions other than their own. Being upset and offended can often be a good thing.

Waterluvian 1 ago 0 replies      
We just need a large enough group to make a fuss over blockages of discourse being a trigger for them. And that they need safe spaces where they can openly communicate without fear imposed on them of possibly triggering someone else.
jamesash 1 ago 0 replies      
Post WWII, an entire generation of soldiers who had seen some of the most horrific scenes of war entered university on the GI Bill and made it through their education without trigger warnings.

The students at the U of C will be fine.

aethertron 1 ago 0 replies      
Are there any proposed metadata schemes for including trigger-warning-like info in, say, web pages?
PhasmaFelis 1 ago 2 replies      
There seems to be whole lot of slippery-slope happening in this thread. Asking people to respect trigger warnings for the most common types of PTSD (generally, rape/sexual abuse) does not lead inevitably to banning everything that anyone has every been uncomfortable with. This is about reasonable accommodation.

Consider the example of disability laws: you're required to install a wheelchair ramp/lift if an employee needs it; you're not required to let a quadraplegic person be a lifeguard.

joesmo 1 ago 3 replies      
I think this should be a requirement for any university that wants accreditation and certainly all state-run institutions. This is a bold statement not for the Uni. of Chicago, but for every other university that does not have a similar policy in place. Sorry, but you cannot get a proper education with trigger warnings and safe spaces. Therefore, those institutions that do not have the courage to stand up for education should lose their accreditation. Until we start holding institutions and individuals responsible for their actions, however, this will not happen, and holding institutions and individuals responsible for their actions is not something that, at least in the US, we know how to do well or at all.
michaelbuddy 1 ago 0 replies      
Brilliant move on the part of the University. This is actual intelligent PR during the current pathetic climate of what is making headlines at Universities nowadays.

People want to go to a university to grow. Growth is not retreat. Strength is not capitulation. Employees want to know that a graduate has a mental resilience along with the skillsets learned.

PhasmaFelis 1 ago 1 reply      
I'm not clear why people on both sides of the debate seem to be interpreting "trigger warnings" as "no triggers allowed." It's meant to be exactly what it sounds like, a advance warning of upcoming content, so that vulnerable people can take appropriate action. That doesn't mean that such content should be forbidden, or that sexual assault victims should get credit for a course that they didn't attend; just that they can prepare themselves mentally, and decide in advance if that credit is worth the pain and discomfort it may cause.

I've had to help a friend having a sobbing panic attack because someone made a stupid rape joke and triggered her PTSD. It's not a laughing matter. But it's not an excuse for censorship, either.

dang 1 ago 1 reply      
"Please don't use uppercase for emphasis."


zyxley 1 ago 1 reply      
The "so called" part of the announcement feels like it's meant to be particularly patronizing. "You rassafrassin youngsters, with your so-called rap music and trigger warnings!"
cmdrfred 1 ago 6 replies      
What I find rather interesting, is that throughout human history we never had a need for trigger warnings until now. We've had slavery, 2 world wars, famine, countless horrors inflicted on a countless number of people. Nobody asks for it.

Then I look to the third world there we have things like female genital mutilation, are the women who had that horror put upon them concerned with trigger warnings? Nope. Not as far as I can tell.

Why here, why now? Are modern first world citizens mentally weaker somehow? I posit that it these types of policies may do more harm than good long term.

kaonashi 1 ago 0 replies      
Old man yells at cloud.
panglott 1 ago 4 replies      
They don't want students to have any spaces where they feel safe/accepted? That's weird.
ebola1717 1 ago 3 replies      
This statement by the University is willfully ignorant. The University provides an officially sanctioned LGBTQ Safe Space (https://lgbtq.uchicago.edu/page/safe-space). Clearly the University isn't ensuring that people who think gay people are evil are also represented in those safe spaces. Furthermore, some ideas are clearly off limits. The University is not going to allow and promote legitimate arguments about whether the Holocaust was justified, or whether we should return to slavery and colonialism.

The point of contention is and always has been which spaces, why, and for what reasons. The University is clearly not ignorant of the nuances of this debate, and if it wanted to commit itself to rigorous debate, it could do so while respecting that there is a valid rigorous debate to be had about this policy.

Why GNU grep is fast (2010) freebsd.org
414 points by mozumder  3 ago   144 comments top 23
jcbeard 3 ago 4 replies      
If you use grep in OS X, and you use it a lot (even called through other programs/libraries) it pays in time to install GNU grep vs. the default OS X provided one. It's several times faster. Ran into this when benchmarking grep + GNU Parallel on OS X vs. Linux in my thesis (raftlib modeling) work. Results on my small MacPro cluster didn't match those on Linux. The OS X runs were orders of magnitude slower despite almost identical hardware configurations. The culprit as it turned out was largely the implementation of grep. I compiled with the GNU version, re-ran and the rest of the mis-match was simply due to the lack of POSIX features like thread pinning, and NUMA control.

<stepping up to soap box/> So yeah...as usual, algorithm matters. Implementation matters, and well....knowledge of the architecture/data-movement matter. These days the architecture/data-movement knowledge matters almost as much or more than the algorithm. Picking an algorithmically faster implementation that moves data more (e.g., tree traversals usually have poor cache behavior, more bursts required, etc.) is often the case b/c programmers don't really understand how the computer works, but are great at coding O(n logn) algorithms. We shouldn't stop teaching algorithms, but data-movement/access likely needs to be emphasized along with the understanding of algorithms.

chukye 3 ago 4 replies      
I think this was already quoted in somewhere, but, its the best advice;

"The key to making programs fast is to make them do practically nothing."

rusanu 3 ago 1 reply      
Previous discussions:

- https://news.ycombinator.com/item?id=1626305 (115 comments, 2193 days ago)

- https://news.ycombinator.com/item?id=2393587 (68 comments, 1972 days ago)

- https://news.ycombinator.com/item?id=2860759 (43 comments, 1842 days ago)

I'm sure there are more.

ComputerGuru 3 ago 2 replies      
I thought for a second this was one of my all-time favorites, which can also be summarized as "how is gnu grep so fast?" [0].

0: http://ridiculousfish.com/blog/posts/old-age-and-treachery.h...

lubomir 3 ago 0 replies      
For those interested in the Boyer-Moore algorithm, this is an commented demonstration:http://www.cs.utexas.edu/users/moore/best-ideas/string-searc...

Compare to Knuth-Pratt-Morris:http://www.cs.utexas.edu/users/moore/best-ideas/string-searc...

hossbeast 3 ago 1 reply      
primis 3 ago 1 reply      
The key to making programs fast is to make them do practically nothing.

This is the Ultimate Truth of optimizing computer programs, and it seems so few people understand it.

"Why can't you make Python faster!?" Because Python does a lot of stuff without you asking.

userbinator 3 ago 1 reply      
I wonder if the simpler Boyer-Moore-Horspool might be even faster, since in practice BMH beats regular BM except in the pathological cases:


Also, unrolling loops may actually slow things down in modern x86 CPUs since they will automatically decode and cache very small loops. Some interesting discussion on that here:


danso 3 ago 1 reply      
OT, but along the same lines of how GNU has brought considerable performance improvements to what (in my naive layperson's mentality) seem like straightforward tools:

Does anyone have an insight as to why GNU tail is, in at least one situation, considerably faster than FreeBSD's tail (at least the one that comes with OS X)?

I'm referring to the use of: `tail -n +2` to skip the first line, versus using `sed '1d'`. To me, it makes intuitive sense how the latter could be faster, and on FreeBSD, it is, by at least one order of magnitude. However GNU tail is one order faster than sed. Is GNU tail using some heuristic to handle "skip the first line in a huge file" differently?

0xmohit 3 ago 3 replies      
GNU grep is fast, but there are certain alternatives that are pretty fast (probably faster):

 - ack [0] - ag (also known as the silver searcher) [1]
You could make `grep` faster by telling it to interpret the pattern as a fixed string (-F or --fixed-strings) is you're not using a regex. Setting to LANG to C would also probably make it faster.

[0] http://beyondgrep.com/

[1] https://github.com/ggreer/the_silver_searcher

amluto 3 ago 3 replies      
I wonder if all of these optimizations are still as helpful. For example, skipping bytes only saves memory bandwidth if you skip whole cachelines.

Also, for --mmap, there are a couple of conflicting considerations. On newer CPUs, large memory copies are very fast. With mmap, though, unless you use MAP_POPULATE, you can get a page fault every few pages, and on CPUs with exception mechanisms as bad as x86's, page faults are very slow (probably 20 times as bad as a syscall).

burntsushi 3 ago 2 replies      
Some other details on GNU grep's performance:

* The Boyer-Moore algorithm it uses has a skip loop, which is used to quickly search for the last byte in the needle before trying to do more checking. Specifically, the skip loop is implemented with memchr, which can be found in your libc and probably compiles down to SIMD instructions. (The trick here isn't actually skipping bytes, but processing many bytes in a single loop iteration.) This optimization works well in the common case, but can slow things down a bit if your skip byte is really common in the haystack.

* When a pattern consists of an alternation of literals, like, `abc|mno|xyz`, then it uses an algorithm based on Commentz-Walter, which you can think of as a hybrid between Aho-Corasick and Boyer-Moore. That is, you get the byte skipping of Boyer-Moore even though you're searching for multiple patterns. The performance of Commentz-Walter degrades as the number of patterns increases because there's fewer opportunities for meaningful skips. Nevertheless, small/short alternations are probably the common case with GNU grep. (The state of the art has actually improved for this specific case and GNU grep could probably benefit. Specifically, the Hyperscan folks over at Intel have some really cool SIMD algorithms for matching multiple patterns. I implemented one here (the comments include a description with examples): https://github.com/rust-lang-nursery/regex/blob/master/src/s...)

* The actual regex engine is a lazy DFA (similar to RE2) in that its states are computed on-demand as needed. That is, some parts of the DFA that aren't used are never computed at all. If the DFA gets too big in memory, the existing states are dropped and re-computed as needed. In this way, GNU grep gets the performance of a DFA while preserving linear time matching. (At most one new DFA state is computed for each byte of input.) The inner DFA loop is unrolled.

* To re-emphazie a point from the post: avoiding line-by-line matching is critical. This makes it much easier to extract literals from a pattern. For example, if your pattern is `\w+foobar\d+`, then GNU grep can still search for `foobar` because it can make an assumption that its haystack is generally a very small slice of the entire search space (i.e., a single line). This type of optimization is much harder to pull off in a general purpose regex engine. A general purpose regex engine might limit itself to looking for prefix or suffix literals only.

* GNU grep really doesn't do all that well for multi-byte encodings. e.g., The time difference between `LC_ALL=C egrep -c '\w{5}'` and `LC_ALL=en_US.UTF-8 egrep -c '\w{5}'` on a large file is quite dramatic. (~3 seconds vs ~52 seconds for a ~2GB mostly-ASCII file that is warm in my page cache.) There's really no need for this slow down if you can bake UTF-8 decoding into your DFA (RE2 does this). However, you then lose the ability to `grep` over other character encodings without transcoding them to UTF-8 first. (In fact, GNU grep may be doing this transcoding step today anyway, so maybe baking UTF-8 into your DFA would be a strictly better solution since it would be much faster on a much larger number of inputs, but probably not slower than any inputs. However, I'm not terribly familiar with this part of GNU grep, so I could have this wrong.)

asadjb 3 ago 0 replies      
Here's a link to the paper the article talks about ("Fast String Searching", by Andrew Hume and Daniel Sunday):


nachtigall 3 ago 1 reply      
What about concurrency? I have 4 CPU cores.
josuah 3 ago 1 reply      
I believed busybox's grep to be faster as it was very small, with GNU version slower due to the many features added over the time, but it is quite not true! On a slow device as some evoked said, the difference is highly noticeable.
underyx 2 ago 1 reply      
Wait, I always tried to make my pattern as short as possible and I thought it would speed up the searches. So I guess this means I'm actually better off searching for the longest possible match then?
skywhopper 3 ago 0 replies      
Always worth a re-read because it's short, interesting, and teaches a good lesson
SixSigma 3 ago 0 replies      
Try it in UTF8 mode and then you'll see a difference
dubmax123 3 ago 0 replies      
Wisdom: "The key to making programs fast is to make them do practically nothing. ;-)"
SunboX 3 ago 0 replies      
Another rule to make programs fast: break early, break often.
gumby 2 ago 0 replies      
Great bit of wisdom buried at the very end:

> The key to making programs fast is to make them do practically nothing. ;-)

BlackBox blackbox.cool
391 points by gregorymichael  2 ago   124 comments top 36
oftenwrong 2 ago 7 replies      
Yet another HN title that says nothing about what lies on the other side of the link.

"BlackBox: a new shipping company from the creators of Cards Against Humanity"

Isn't that just a bit more descriptive?

scandox 2 ago 4 replies      
"..without publishers or bloodsucking middlemen taking most of the money..."

Obviously publishers and bloodsucking middlemen are more or less synonymous in this sentence. I'm on the board of a publishing company and have also worked as an agent (a true bloodsucker).

Two things:

1. In a practical sense it is easy to see publishers and agents as bloodsucking middlemen. Once you've actually worked with lots of creators and artists what you learn is that there'a very good reason for the middlemen. Artists do not generally wish to have too much to be doing with their public - especially not in a commercial sense. In the end of the day Publishers and Agents give writers the thing they really need most: faith. It sounds like corny bull - hell it is corny bull...but it's also incredibly important.

2. I remember reading a great article about Fintech startups trying to "cut out the middleman" and it had a great observation in it: the middleman is there because the two parties don't trust each other - not because they're stupid. I always think of that when I hear people trying this approach.

qwtel 2 ago 3 replies      
It seems like a cool company and much needed as well, but I'm a bit puzzled by the political rhetoric mixed into the marketing page. It's not even that there's politics involved at all (after all you could make the argument that Tesla/SpaceX are like that as well), it's just how short-sighted it is.

Ignoring for a moment that the "bloodsucking middleman" of today were the hot new shit in their time, what other than innocent looking gifs do we have in way of them not becoming bloodsuckers themselves, once or if they reach a position with that kind of pricing power? It's easy to act all nice when you don't have a choice anyway...

Also, the company shipping the product from the producer to the customer seems pretty "in the middle" to me, no matter their size or political ideas.

gsmethells 2 ago 2 replies      
For a second there I thought the Window Manager Blackbox had risen from the dead. https://en.m.wikipedia.org/wiki/Blackbox
elliotpage 2 ago 4 replies      
Tried to give this a go (It sounds cool and shipping that scales is a massive pain) but the $5 press kit has $33 shipping to the UK!Guess this won't be solving my shipping problems after all.
Animats 2 ago 1 reply      
Oh, it's a fulfillment house. There are lots of companies which do that, including Amazon and UPS. The site seems to be addressed to people who don't know that. Nice art, though.

Not to be confused with "blackbox.com", which sells networking accessories.

yAnonymous 2 ago 0 replies      
"Quality and simplicity are more important than saving a few pennies on shipping. Our price includes the beautiful Blackbox checkout button, our real-time dashboard, and our dedicated customer support team."

"Artists own the relationship with their customers and can talk to them whenever and however they want. That relationship works best without bloodsucking businessmen in the middle."


jseip 2 ago 2 replies      
Immediately hit 'inspect element' on bb-animated-header
magic5227 2 ago 1 reply      
5% retail price? That seems quite steep.

I've personally shipped 1,000+ gifts for Kickstarter backers, I'd suggest using Easypost.com unless you really need a warehouse. It's super cheap.

RobertLong 2 ago 3 replies      
I really thought this was going to be my favorite yet evil puzzle game Blackbox. http://blackboxpuzzles.com/
lelandbatey 2 ago 3 replies      
This seems like an interesting service, but I want too toss out a suggestion for those looking to disseminate info about a product or service:

Tell me a story.

More specifically, use a hypothetical story to provide context and an example of why the hell I'm reading your page. This is the explanatory copy on this page:

Our mission is to help you sell and ship stuff directly to your fans for afraction of the cost and effort of doing it yourself. Blackbox works like aco-op: if we all go in together, we get the cheapest pricing, the fastestshipping, and the best service. The shipping is fast. We pay your sales tax.You can customize the packaging and the inserts. Its pretty great.

At the end of this, I still am confused about what you do. What would help is a hypothetical use case and narative, and to demonstrate that, I'll invent one:

Let's say you're running a Kickstarter. You're a domain expert who's builtsomething awesome, and you've even successfully kickstarted a project tobuild what you love. Now though, after having built all these gizmos youneed to ship those to your backers, and this is something you're NOT anexpert in. Looking in to shipping, you see it's going to involvecomplicated contracts and procedures to ship your gizmo at any scale, andat a cost that FAR exceeds your back of the envelope calculation of priceand effort. You wish there where some simple shipping service where you only worry about the packaging, and they handle picking up from the manufacturer, storage, and shipping; a service withstraightforward pricing, and customer service oriented at helping someonewith your needs and your scale.

That's where BlackBox comes in. Our mission is to help you sell and shipstuff directly to your fans for a fraction of the cost and effort of doingit yourself. Blackbox works like a co-op: if we all go in together, we getthe cheapest pricing, the fastest shipping, and the best service. Theshipping is fast. We pay your sales tax. You can customize the packagingand the inserts. Its pretty great.

I don't know if the above story accurately represents what it is that BlackBox provides, but I hope you can see that having a story helps a reader answer the question of "is this helpful or relevant to my situation or goals?".

kajecounterhack 2 ago 1 reply      
Sounds to me like yet another fulfillment service but maybe they take care of more? Anyone know how it compares to fulfilled by Amazon or what comparable services exist? I'm curious to know if it's cheaper or just easier. (Or both, or neither!)
hartror 2 ago 2 replies      
Who is the pixel artist?
tux1968 2 ago 4 replies      
Presumably you have to pay to ship all your inventory to them in the first place. Didn't see any mention of that in the details.
paulrosenzweig 2 ago 0 replies      
So is this Gumroad but for physical goods? The sales flow is similarly out of your hands, but it's super easy to set up.
empressplay 2 ago 0 replies      
$35 shipping to Canada for a box of fortune cookies. Pass.

Unless you _only_ want to sell stuff inside the US. Then it's okay.

Everlag 2 ago 1 reply      
I'm interested to see how 'We pay your sales tax' works out.

Also, should there be an epilepsy warning for the header?

justrossthings 2 ago 1 reply      
These guys spent more time on animating their team portraits than explaining their product
voltagex_ 2 ago 1 reply      
$50.75 US to ship to Australia.
kdamken 2 ago 0 replies      
I saw the TLD and thought to myself, "What a silly thing to end your url with. This better be one badass website".

Then I opened the page and saw that gif, and they totally redeemed themselves! Very mesmerizing.

brunorsini 2 ago 0 replies      
https://shotput.com is a YC company also doing fulfillment. It seems focused at slightly larger customers though.
aetherspawn 2 ago 0 replies      
This needs an NSFW tag, like, when you open this page and your monitor starts psychedelically flashing, everyone turns around and stares at you to shame.
dutchbrit 2 ago 0 replies      
I actually saw the animated header and clicked the back button thinking it was just a gif and no actual page (until I read the comments).
felipemesquita 1 ago 0 replies      
Shipping the promotional material package to Brazil is quite expensive and deliver date format is a bit confusing. Looks great for orders placed within the US, though.

Regular ShippingDelivers 2/9-9/9$43.25

stevenwiles 1 ago 0 replies      
"BlackBox: The makers of Cards Against Humanity do another stupid thing and get paid lots of money for it"

^ that is a much more accurate title

skeltoac 2 ago 0 replies      
I'm just going to watch that gif all day now. Thanks.
davnn 2 ago 0 replies      
I always wonder about the conversion metrics when I see an artistic website like that. I think the website would perform better without that huge header.
cercatrova 2 ago 1 reply      
So like Massdrop but for any arbitrary product?
dlevine 2 ago 0 replies      
I believe that they were responsible for delivering the Exploding Kittens Kickstarter.
daxfohl 2 ago 2 replies      
Lost me at 1489167 -> 1489280 -.>


1489167 -> 1489280 -.>

At least pretend to get your realtime data right before posting on HN.

brakmic 2 ago 0 replies      
Some clickbaits deserve more than just a single "downvote".
blazespin 2 ago 0 replies      
It's the sharing economy hitting creating companies. it's about freaking time...
homero 2 ago 0 replies      
That would go well with bitcoin
hazelnut 2 ago 1 reply      
seems to be down
jethro_tell 2 ago 1 reply      
That intro. Seems like they are selling headaches not shipping.
Whostasay 2 ago 1 reply      
Wow, what an offensive website! First thing I do when I visit websites like that is to close the tab.

Hope they aren't a business. The horrible pixel art from 1980s doesn't help.

How AI and Machine Learning Work at Apple backchannel.com
326 points by firloop  2 ago   143 comments top 20
Smerity 2 ago 4 replies      
None of this really answers the overlying question that Jerry Kaplan and Oren Etzioni raised. The question raised by most in the field isn't whether Apple use AI/ML internally, the real question is why they avoid the research community so strongly.

For me, the greatest thing about the ML/AI community is how open it is and how strong a sense of camaraderie there is between people across the entire field, regardless of whether they're from industry or academia.

Employees from competing companies will meet at a conference and actually discuss methods.Papers are released to disseminate new ideas and as a way of attracting top tier talent.Code is released as a way of pretraining students in the company's stack before they ever step through the company's doors.Papers are published on arXiv when the authors feel they're ready - entirely free to access - without waiting for a conference for their ideas to be spread.

This entire push of camaraderie has accelerated the speed at which research and implementation have progressed for AI/ML.

... but Apple are not part of that. They publish little and more broadly don't have a good track record. On acquiring FoundationDB, they nixed it, with little respect to the existing customers. Fascinating pieces of technology lost. If they aren't using the exact thing internally, why not open source it? I fear the same is likely to happen to Turi, especially sad given the number of customers they had and the previous contributions that many of Turi's researchers made to the community via their published papers.

Apple may change in the future - they may become part of the community - but a vague article of self congratulation isn't going to sway me either direction.

"We have the biggest and baddest GPU farm cranking all the time" ... Really? \_()_/

throwanem 2 ago 7 replies      
I like how Apple can't win here.

If they publish, they aren't doing anything new and they haven't innovated since Steve died, and they should really just give up because there's obviously no point to anything they do and hasn't been since 1997.

If they don't publish, they're evil secretive bastards who don't contribute to the ML community and probably drown puppies or something because who knows what goes on behind closed doors?

I don't really have a dog in this fight, except inasmuch as I'm a generally satisfied iPhone owner. I just think it would be really neat if people would settle on one narrative or the other, instead of keeping on with both at once.

cromwellian 2 ago 1 reply      
I realize this is going to get voted down, but piece reads like a Trumpism "we have great AI, trust me, you would not believe, if you could see what we're doing".

I don't think you get to claim PR credit for advances in ML unless you publish. In general, for R&D, IMHO, you need to publish. There's product R&D, and there's fundamental R&D. If you make an advancement in something fundamental, but that helps your product, then publish it. If it is specific to your product only and can't be transferred elsewhere, then maybe it's ok to keep it secret.

Apple and Google's competitive advantage now arises from scale and path dependency. I think they need to let go of this idea that somehow they derive a competitive advantage by keeping these things secret. The Open AI community is going to advance at an accelerated rate regardless and IMHO, it's better to be part of it than to be seen as a kind of parasite that consumes public R&D, but doesn't give back improvements.

theinternetman 2 ago 1 reply      
Finding these heavily curated advertorials Apple has been pushing out (This and the recent wired advertorials come to mind) a bit of a sign that not everything is sunny at One Infinite Loop.
mikesf888 2 ago 0 replies      
IMO the most profound quote in the interview: Our practices tend to reinforce a natural selection biasthose who are interested in working as a team to deliver a great product versus those whose primary motivation is publishing, says Federighi.
tahoeskibum 2 ago 1 reply      
Nice article, but in my perception, Apple is way behind in AI vs. Google on mobile. Siri's current speech recognition is still far behind Google on my iPhone. Half of the time it Siri doesn't recognize something but Google recognizes it right way. Apple Maps continues to lag in the following features: biking and public transit and even things like which lane to take on a big interchange. As a result I end up using Google (Now) and Google Maps by default.
vthallam 2 ago 0 replies      
> If youre an iPhone user, youve come across Apples AI, and not just in Siris improved acumen in figuring out what you ask of her. You see it when the phone identifies a caller who isnt in your contact list (but did email you recently). Or when you swipe on your screen to get a shortlist of the apps that you are most likely to open next. Or when you get a reminder of an appointment that you never got around to putting into your calendar. Or when a map location pops up for the hotel youve reserved, before you type it in. Or when the phone points you to where you parked your car, even though you never asked it to. These are all techniques either made possible or greatly enhanced by Apples adoption of deep learning and neural nets.

This whole paragraph must be a joke. Google started doing this since way too long and they don't even publish these as their best features.

KKKKkkkk1 2 ago 1 reply      
Why should Apple researchers contribute back to the AI community on their shareholders' dime? What makes AI special as opposed to any other field of computer science?
soared 2 ago 2 replies      
I've never had a single one these happen to me. Has anyone actually seen these behaviors out in the wild? Is it because I use gmail, chrome, google maps, and shut off most of the siri/recommended apps/etc functions in favor of serious battery life gains?

>You see it when the phone identifies a caller who isnt in your contact list (but did email you recently). Or when you swipe on your screen to get a shortlist of the apps that you are most likely to open next. Or when you get a reminder of an appointment that you never got around to putting into your calendar. Or when a map location pops up for the hotel youve reserved, before you type it in. Or when the phone points you to where you parked your car, even though you never asked it to

LeanderK 2 ago 2 replies      
i am really disappointed by apple. I respect Apples wish to develop products in secrecy and i understand that you can't just open source your secret sauce. I also really like their products.

But not publishing your advancements harms the community greatly. Its like building your product entirely with open-source software (the published work of other researchers) and not contributing back.

msoad 2 ago 1 reply      
Apple sends their employees to ML/AI conferences with fake company names on their badges to avoid leaking a single bit of their knowledge. I don't know how any AI researcher resists working at Apple!
vonnik 2 ago 4 replies      
I have a big problem with articles like this.

Apple's PR is notorious for cracking the whip, which means that the "inside story", if they give it to you, comes with a warning to the journalist to behave and be nice. Levy's piece is generous with flattery and cautious with criticism. He quotes Kaplan and Etzioni high and briefly in the piece, and spends the rest of it refuting them. Apple will give him another inside story down the road.

Apple has a big question to resolve for itself about the tools it's going to use to develop this. It can't go with Tensorflow, because TF is from Google. It's kind of at another turning point, like the one in the early 90s when it needed it's own operating system and Jobs convinced them to buy next and use what would become OSX.[0]

The most pointed question to ask is: What are they doing that's new? The use cases in the Levy story are neat, and I'm sure Apple is executing well, but they don't take my breath away. None of those applications make me think Apple is actually on the cutting edge. There's no mention of reinforcement learning, for example; there is no AlphaGo moment so far where the discipline leaps 10 years ahead. And the deeper question is: Is Apple's AI campaign impelled by the same vision that clearly drives Demis Hassabis and Larry Page?

We see what's new at Google by reading DeepMind and Google Brain papers. Everyone else is letting their AI people publish, which is a huge recruiting draw and leads to stronger teams. Who, among the top researchers, has joined Apple? Did they do it secretly? (This is plausible, and if someone knows the answer, please say...) The Turi team is strong, yes, but can they match DeepMind? If Apple hasn't built that team yet, what are they doing to change their approach?

Another key distinction between Apple and Google, which Levy points out, is their approach to data. Google crowdsources the gathering of data and sells it to advertisers; Apple is so strict about privacy that it doesn't even let itself see your data, let alone anyone else. I support Apple's stance, but I worry that this will have repercussions on the size and accuracy of the models it is able to build.

> We keep some of the most sensitive things where the ML is occurring entirely local to the device, Federighi says.

Apple says it's keeping the important data, and therefore the processing of that data, on the phone. Great, but you need many GPUs to train a large model in a reasonable amount of time, and you simply can't do that on a phone. Not yet. It's done in the cloud and on proprietary racks. So when he says they're keeping it on the phone, does he mean that some other encrypted form of it is shared on the cloud using differential privacy? Curious...

> "How big is this brain, the dynamic cache that enables machine learning on the iPhone? Somewhat to my surprise when I asked Apple, it provided the information: about 200 megabytes.."

Google's building models with billions of parameters that require much more than 200MB, and that are really, really good at scoring data. I have to believe either that a) Apple is not telling us everything, or b) they haven't figured out a way to bring their customers the most powerful AI yet. (And the answer could very well be c) that I don't understand what's going on...)

[0] If they have a JVM stack, they should consider ours: http://deeplearning4j.org/

devy 2 ago 0 replies      
There are a lot of criticisms here about Apple's secretive AI/ML development practices. But it's not unusual to Apple's long culture heritage of secrecy.

From consumer's perspective, I applaud their firm believe of customer privacy as well as pioneering consumer products based on AI/ML development AND with differential privacy in mind.[1][2]

[1] http://highscalability.com/blog/2016/6/20/the-technology-beh...

[2] http://www.imore.com/our-full-transcript-talk-show-wwdc-2016...

quattrofan 2 ago 0 replies      
THis feels so much like a story done as a marketing/PR angle
castell 2 ago 0 replies      
Is there a way to op-out of this on iOS and macOS? That's really scary.

ML and AI on OS-level should run decentral on the device itself, and don't leak data at all. The spirit of the 1990s was that way, and we older desktop software works fine that way (even on Pentium 1 hardware), so it would run like a piece of cake on a modern smartphone.

The "differential privacy" technology may sound good, but without an independent audit who knows how good it works.

plg 2 ago 0 replies      
has anyone considered the possibility that perhaps Apple has made a judgement that the recent resurgence in AI/ML is, (like all of them before, over the past 50 years) overblown, and Apple would rather spend their time and resources on other things?

There's no doubt that the recent advances in deep learning have improved ML/AI in certain specific domains ... but it seems like every 15-20 years or so we see an advance and an accompanying narrative that "AI is back! fully automated future is near!"... which fizzles out, again

Also, Apple has a more humanist tradition than Google, FB, etc, and it's my impression that they value the human element perhaps more.

Sure, there's Siri, but Siri strikes me more like an ongoing experiment than a fully fledged whole hog "let's put all our eggs in this ML/AI basket"

dstaten 2 ago 3 replies      
siri has gotten worse over time, or at least that's why friends and i have noticed
emehrkay 2 ago 1 reply      
Off topic a bit, but this article makes me wonder where does one start with Deep/Machine Learning/AI? I've seen a few posts the past few days talking about the topic (Deep Learning with Python, etc.), but what are the core requirements regarding math, statistics, programming, etc? Where should a web developer start?
doe88 2 ago 0 replies      
It isn't clear from the article do they still use Nuance?
caycep 2 ago 0 replies      
They were certainly at NIPS. I got a nice Apple pen and ski hat...
Introducing OpenStreetView openstreetmap.org
452 points by progval  6 ago   79 comments top 11
andrewljohnson 5 ago 1 reply      
Similar to http://mapillary.com.

It's great to see more open data like this, lots of uses for creating maps by hand or even machine learning.

rwmj 5 ago 2 replies      
Very cool. OSM has been the default map on my phone for years. I do worry that the name is a little close to Google's trademarked Street View (see list of Google TMs here: https://www.google.co.uk/permissions/trademark/trademark-lis... ). They might wish to avoid an unnecessary lawsuit by calling it something else.
nixos 5 ago 3 replies      
I understand that it's a volunteer effort, and volunteers don't have the hardware required for even 180 degree view, but most pictures are taken out of the front of the car, limiting its worth (2/3 of the picture is road, not the houses on the side).

It would be more useful if people stuck two phones (one on each side of the car) to take pictures of the passing buildings rather than road

dbrgn 6 ago 5 replies      
Exciting! Now instead of waiting for the street view car to pass through your neighborhood, you can add the pictures yourself!

I wonder how privacy issues are handled though. Is any blurring of car numbers or faces being done or planned?

ktta 5 ago 3 replies      
This is pretty cool.

What would be amazing to also have LIDAR sensors (EXPENSIVE! I know, I know) which would really make amazing data for self driving/completely autonomous vehicles; not only cars but also drones, etc.

Anyone know the pro/con for implementing LIDAR (although it maybe super late before we have enough data for effective navigation at long distances), and if it is worth it?

I think it would be really cool to have info about the heights of buildings, exact positions of traffic lights, etc. which are miniscule but crucial for low altitude drone navigation. Or can we infer almost everything from the picture data?

opk 5 ago 3 replies      
Interesting that some roads in Germany are covered, at least for now.
twelvechairs 5 ago 2 replies      
Props to the people who made this - fantastic public contribution. My big wish list as someone who works in urban design and planning would be this paired with 3d point cloud detection to make a 3d model of streets and spaces. Maybe one day eh!
francium_ 5 ago 2 replies      
Some of the images appear to be upside down. Is this a known bug?
astrostl 5 ago 0 replies      
Every huge entity having their own mapping system in order to grind out a minor efficiency is probably the biggest tragedy, to me, in modern services. Before mapping, it would be the TLS certificate racket, which LetsEncrypt appears to have dead to rights.
stelonix 5 ago 0 replies      
What I've wondered for a while is the feasibility of producing streetview imagery with drones. It seems to be the perfect solution for distributed street shooting.
ashitlerferad 5 ago 2 replies      
Anyone know what the business models of Telenav and OSV are?
USCIS Proposes Rule to Welcome International Entrepreneurs uscis.gov
379 points by jrbedard  22 ago   135 comments top 38
yurisagalov 22 ago 1 reply      
A less "PR"-ey take (but still PR-ey non the less) on the White House Medium blog: https://medium.com/the-white-house/welcoming-international-e...

As an international founder who has had to suffer the stresses of dealing with US immigration while building a company based in the states, this is incredibly welcome news.

beagle3 19 ago 3 replies      
Anyone here aware of tax implications of the parole status?

Usually, "us taxpayer" status is only triggered with a proper visa (H1B, L1, O1, etc.) or permanent resident status (green card). I've never met any mention of tax status on parole.

To someone who has any non-trivial financial life outside the US (which I expect to be true for most people who would apply for this), a us taxpayer status is a horrible curse:

You might get a call from your bank/broker/insurer back home, telling you that they have to close all of your accounts except perhaps a checking and 1 simple saving (and perhaps those too). If you accrued any benefits in your pension plan, you have to report them each year _and_ plan to pay yearly and/or dearly (because it is a PFIC[0]) for any profit made in that account, even though you cannot touch it for 30 years.

Also, if you own another business outside the US -- e.g. you had a previous startup, or operated through some kind of personal LLC providing services -- the "us taxpayer" status means that you have to start filing financial reports to the IRS as if those business were operated in the US (that is, according to US financial standards, regardless of where it operated - and that might mean double taxation despite any treaties in place).

Sure, you can just ignore it, and if your startup fails, no one will come after you. But of course, you plan to succeed, so be sure to read on PFICs, FATCAs, FBARs, and consult a US accountant that specializes in international taxation.

As a general rule, the US tax system assumes any financial dealings you have outside the US are an attempt to evade taxes, and penalizes that (whether by forms or by actual tax). Once it is assumed you plan to make the US the center of your life (green card, H1B, L1), those assumptions, through the "us taxpayer" status, affect you. If it is assumed you won't stay (e.g. F1 visa for studying), they don't.

So, what taxpayer status will parole put you in?

[0] https://en.wikipedia.org/wiki/Passive_foreign_investment_com...

vmarsy 21 ago 3 replies      
It looks like this rule would provide the entrepreneur a 5 year stay (2+3 additional if the start-up is doing well), but I don't see any mention of granting a green card, so what would happen to that entrepreneur on year 6?

EDIT: from the medium link posted in another comment by yurisagalov:

> DHS will also publish guidance to clarify when entrepreneurs may self-petition for lawful permanent residence (also known as a green card).


> That entrepreneur will have the confidence of knowing whether she is eligible for the temporary parole status pathway, allowing her to start growing her company here right away. Later, she will know under what circumstances she may qualify for the green card pathway, if she is successful in creating jobs for U.S. workers and attracting more capital from U.S. investors, allowing her to become an American over time. (Note that these pathways will likely also be open to "bootstrap" entrepreneurs who are successful in generating revenue from U.S. customers, without needing to rely on external financing.)

(From https://www.whitehouse.gov/blog/2014/11/26/entrepreneurs-wan... )

runesoerensen 21 ago 1 reply      
It's pretty cool that they're proposing parole rather than an actual visa - I never thought of that possibility, but it seems to fix a lot of policy-related issues, such as visa caps, and allow qualifying entrepreneurs faster entry. Might short-circuit the process and we don't have to wait for politicians to agree that a founder visa makes sense.

Founders also wouldn't have to get a certified LCA or meet minimum wage requirements, which can be pretty costly for a startup and is less relevant in case of founders (who are not taking anyone else's jobs or lowering average wages). And generally founders would probably prefer (and have more to gain from) spending that money to grow the company.

I suspect there are downsides to paroles vs visa (like fewer rights or difficulty obtaining green card/other visa types later maybe?), but this seems like a surprisingly good proposal to fix an acute issue.

e15ctr0n 20 ago 1 reply      
For those looking to get a much more comprehensive look at this proposed rule from USCIS, read the advance version of the notice to be published in the Federal Register:

https://www.uscis.gov/sites/default/files/USCIS/Laws/Article... (PDF, 649 KB, 155 pages)

The sections most relevant to those who wish to apply under this rule would be:

IV. Proposed Changes

A. Overview of Parole for Entrepreneurs (pages 30 - 32)

B. Criteria for Initial Parole (pages 32 - 52)

1. Recent Formation of a Start-Up Entity2. Applicant is an Entrepreneur Who is Well-Positioned to Advance the Entitys Business3. Capital Investment or Government Funding Criteria

C. Application Requirements for Initial Period of Parole (pages 53 - 61)

1. Filing the Application for Entrepreneur Parole (Form I-941)2. Requirement to Appear for Submission of Biometric Information3. Income-Related Condition on Parole4. Adjudication of Applications5. Limitation on Number of Entrepreneur Parolees Per Start-Up Entity6. Authorized Period for Initial Grant of Entrepreneur Parole

D. Employment Authorization (pages 61 - 66)

1. Employment Authorization Incident to Parole with a Specific Employer 2. Employment Authorization Eligibility for Spouses 3. Documentation for Employment Eligibility Verification (Form I-9)

F. Re-Parole (pages 67 - 84)

1. Criteria for Re-Parole 2. Application Requirements for Re-Parole3. Ensuring Continuous Employment Authorization 4. Technical Changes

G. Termination of Parole (pages 84 - 87)

1. Automatic Termination2. Termination on Notice

H. Automatic Adjustment of Investment and Revenue Amount Requirements (pages 87 - 88)

V. Statutory and Regulatory Requirements

C. Executive Orders 12866 and 13563

3. Population of Entrepreneurs Potentially Eligible (pages 103 - 116)

chx 16 ago 1 reply      
Who knows what happens. The Canadian startup visa in three years managed to bring in 100 people including dependents http://www.cbc.ca/news/canada/windsor/start-up-program-disap...
gorbachev 21 ago 0 replies      
What happens, if you get one of these paroles, but your business falls apart for whatever reason within the first two years?

There doesn't appear to be any specific language in the actual document outlining the rules changes, but it does say DHS would retain the right to terminate the parole at any time without prior notice and in DHS' discretion (pg. 84 in the PDF).

Looks like a little bit of gray area, unless there is some language in there about the parolees responsibilities to maintain the conditions under which the parole was granted and consequences if the parolee does not. I couldn't see anything like that during a quick read through the document.

kirubakaran 21 ago 0 replies      
Although they mention funding and grants as the criteria and not revenue, I hope "Partially satisfying one or both of the above criteria in addition to other reliable and compelling evidence of the startup entitys substantial potential for rapid growth and job creation." means that revenue will at least be considered to some extent.
runesoerensen 20 ago 0 replies      
Relevant essay: http://paulgraham.com/foundervisa.html

In 3-5 years we'll probably hear founders tell stories about how this new rule allowed them to establish very successful companies in the US. That should help prove the value of a proper founder visa and (assuming the immigration system is still broken at that time) pass legislation to fix it.

bozoUser 21 ago 1 reply      
Great news for all the non-US budding entrepreneurs. Founders have to satisfy either of 3 criteria stated -

1. $345K from private investors.

2. $100k from govt or Federal entities.

3. partially satisfying the above criteria(up for a toss with uscis).

Some nascent YC potential companies might still loose out but nevertheless theres some glimmer of hope. Hope the rule gets published.

Animats 19 ago 3 replies      
There's already a visa deal for foreign investors, the EB-5 visa. You have to invest at least $1M ($0.5M if you're willing to invest in a bad area of the US) and create 10 jobs.

That's why downtown Palo Alto has all those rug stores.

randall 20 ago 0 replies      
Wow!!! So great. I have two international cofounders. So stoked.
xbeta 14 ago 3 replies      
Is it really required to found a startup inside US to have a greater chance of success? Or more specifically in the bay?

I have seen many successful startups found in other countries (such as China) Alibaba, Tencent, etc.

Is this a push that all talented software engineers HAVE TO move to the bay to become successful?

I'm very doubtful on that claim.

vesh 21 ago 2 replies      
Does anyone know where and how the public can provide their comments? I cannot find a link on the page. Bootstrapped startups with no outside investments wouldn't be eligible based on this criteria.
vthallam 20 ago 2 replies      
A great start for sure.

>Receiving significant investment of capital (at least $345,000) from certain qualified U.S. investors with established records of successful investments

Not sure what does the qualified US investors mean though. Would seed funding/angel funding from individuals not be considered for this? I guess this has been added to avoid abuse by wealthy foreigners, but how do they determine the qualified individuals or companies?

NhanH 21 ago 1 reply      
The number doesn't quite work out: you got two years, and if the business can demonstrate passing certain benchmarks (500k funding or (500k revenue and 20% annual growth) or 10 'murican jobs), you get another 3 years. 5 years is enough if your business fails fast, but otherwise it's unlikely to be sufficient for building one (on average).

This is a parole rather than a visa, so it still leaves the question on what's the follow up afterward.

titomc 20 ago 1 reply      
What will happen after the 5 years stay ? Should the entrepreneur pack up and leave ? Why should an entrepreneur startup something when he/she is not sure about their residency in US. This rule says 'parole' & self petition, the residency path is not clear. And when it comes to residency, US immigration for skilled immigrants is so broken that you will regret applying for one.
benmarten 21 ago 1 reply      
Please note that this does not seem to be any improvement for citizens of "US treaty countries", e.g. most northern and western european countries. Where the needed capital investment is only needed to be substantial to get an E-2 investor/entrepreneur visa. E.g. 100-150k of initial investment needed as far as I know ;)
graeme 15 ago 0 replies      
Can anyone see if this applies to bootstrappers or solo founders?

It doesn't look like it. Referring to the type of business that can have significant revenue, but doesn't need investment or even necessarily employees.

yranadive 21 ago 0 replies      
" A subsequent request for re-parole (for up to three additional years) would be considered only if the entrepreneur and the startup entity continue to provide a significant public benefit as evidenced by substantial increases in capital investment, revenue or job creation."

How will they measure this? It's so subjective.

rrecuero 16 ago 0 replies      
As another international founder, it is definitely a great step in the right direction. I am still missing an easier way to bridge the transition gap into a Green Card, maybe a different EB?
nullcipher 21 ago 0 replies      
This is a great step forward. I actually like the idea of parole rather than a proper visa upfront. First, get entrepreneurs to US and let them show their skills. If they succeed, they can always apply for GC directly through the existing outstanding skills category or take an investor visa.
vadym909 15 ago 0 replies      
This is great for the US and many foreign entrepreneurs, but would suck for other country's startup efforts. Its like the Golden State Warriors getting Kevin Durant.
sethbannon 20 ago 0 replies      
"Give me your tired, your overworked, your entrepreneurs yearning to innovate."
AhtiK 21 ago 0 replies      
If entrepreneur can keep at least 50% of the company then already today L-1 could be a viable option.

Looks like the proposed parole is quite similar to L1 except instead of foreign company there's foreign private person direct relationship.

pyb 18 ago 2 replies      
This sounds like the holy grail for international founders, but how likely is it to pass ? Anyone here in the know ?
patcheudor 21 ago 0 replies      
When I looked at that title I thought: "new rule in ____ . Leaves drivers furious!" and wondered how HN was infiltrated with click-bait advertising.

The Internet has ruined me.

toodlebunions 21 ago 0 replies      
Seems like the necessary investment amounts are pretty low.
ones_and_zeros 21 ago 1 reply      
My prediction: This too will be gamed just like the rest of the immigrant worker visa systems (H-1B, OPT, O-1, EB-5, etc). That big fat vague 3rd criteria will see to it. A whole cottage industry of immigration attorneys who specialize in this visa will crop up to prey on unsuspected foreigners and collude with the knowing and corrupted ones.

The fact of the matter is most foreigners complaining about not being able to start businesses in the US don't have profitable or well funded businesses. The ones that do have profitable and well funded business can easily set up shop on US soil and don't need to immigrate here and can use L1 visas when they do need to come state side.

To me the most interesting aspect to this visa is it takes away the talking point from the H-1B proponents that immigrant workers are job creators because immigrants start companies that employ citizens.

gjkood 21 ago 2 replies      
If you are a hardware entrepreneur there may be better options than trying to set up in the USA.

Try setting up shop in Shenzhen, China. Just spent a week over there. It is truly a Hacker's Paradise.

You can validate your ideas and get an MVP out in much faster time than the USA. You can be close to your eventual supply chain when you do make it big. It may sound a bit like premature optimization but the advantages are huge.

Of course, it will not hurt to learn a bit of Chinese (Mandarin) if you do decide to take that route. At least start with a few useful phrases to help break the ice. 'Ni Hao' (Hello); 'Xie Xie' (Thank You) and 'Duo Shuo Qian' (How much does it cost?). Learning the number system will also help know what the price is.

I spent a lot of time in the Huaqiang Bei and LoWu malls.

A special shoutout to Andrew "Bunnie" Huang for his 'The Essential Guide to Electronics in Shenzhen'[1]. It was invaluable.

Another shoutout to my Mandarin teacher Larry Xue and the San Jose Learning Center.[2] Attending his classes made me so much more confident that I could manage there even with the language barrier.

1. https://www.crowdsupply.com/sutajio-kosagi/the-essential-gui...

2. http://sanjoselearningcenter.com/mandarin.php

drl42 21 ago 2 replies      
Summary: - New startup entity (< 3 years old) - Atleast 10% stake - No more than 3 applications per startup - Atleast $345,000(VC,Angel,Incubator) or $100,000(govt grants) or show public benefit if less funding available - Initial stay for 2 years - Employment authorization only from startup - Minimum salary at 400% poverty level - Spouse gets EAD, but minor children do not.

After 2 years, 3 years extension - Atleast 10% ownership and active role in startup - Atleast $500K additional funding OR $500K revenue with 20% growth OR 10 Full time Jobs


gordon_freeman 21 ago 0 replies      
This would definitely help foreign founders but after they have founded the company in US since 3 years. That initial period of founding a startup is the most difficult part for immigrants in USA as their visas are tied to employer (like H1B).

So would this "startup visa" help in anyway in that initial stage of founding a startup?

xyzzy4 21 ago 1 reply      
How about a rule to welcome anyone who isn't a criminal?
xenosapien 20 ago 0 replies      
My startup will focus on building walls. I believe this is a burgeoning market.
fmp 18 ago 2 replies      
There are six billion people in shitty countries, and one billion people in rich countries.

If you move the population of Somalia to Connecticut, what do you think happens?

lawnchair_larry 21 ago 3 replies      
Unfortunately wealthy foreigners are just going to abuse the hell out of this to drive up real estate prices.
tostitos1979 20 ago 7 replies      
I wish having an advanced degree was a criteria here. A few years ago, I noticed that Britain would give a blanket visa to anyone with an MBA from a list of top international schools. How about something like that for people with Masters/PhDs (in STEM) from top-50 schools in the world.

That said, the proposed rule might mean if one gets into YC/techstars, etc. they would be able to get a visa for the US easily.

The limited term and renewals do raise some flags. But I guess if you are successful with your startup in the given time period, you can apply for a green card through other categories like Extraordinary Ability, etc.

Instapaper is joining Pinterest instapaper.com
372 points by ropiku  3 ago   207 comments top 29
brians 3 ago 17 replies      
Bummer. I love and use instapaper, gathering articles for a few weeks to read at altitude. It's a great product, and I paid for a subscription these last years in the hopes that I could therefore continue to enjoy it.

Now it's sold to Pinterest, one of the two sites I don't bother with links tobecause I know Pinterest and Quora will require me to sign in rather than show me what they showed a search engine.

What else operates in this space? Pocket, I remember. ReadItLater used to exist, maybe still? Does Pinboard do this somehow, maybe with an RSS reader? Or do I have to pay for Paperback?

bthdonohue 3 ago 14 replies      
Hi all,

Brian from Instapaper here. There seem to be a number of comments expressing concerns about the acquisition, and I wanted to just jump in and offer to answer any questions you have about the acquisition.

Based on the comments I've read below the main concerns seem to be that Instapaper will either be shutdown or materially changed in a way that effects the end-user experience. I can tell you that neither of those are the plan for the short-term or long-term of the product, and I am personally looking forward to providing you with the same great service under a new owner.

Brian DonohueInstapaper CEO

bishnu 3 ago 2 replies      
Worth re-reading Marco Arment's blog post about selling Instapaper 3 years ago:https://marco.org/2013/04/25/instapaper-next-generation

"Instapaper needs a new home where it can be staffed and grown, but I didnt want to give it to a big company that would probably just shut it down in six months."


fishtoaster 3 ago 3 replies      
> The Instapaper team will be moving from betaworks in New York City to Pinterests headquarters in San Francisco

I wonder how the instapaper team feels about that (and/or how much say they had in it).

markpapadakis 3 ago 0 replies      
I used Instapaper for a long time before I switched to a Pocket.

There were issues with their parser they never fixed, has been consistently slow, articles would often be lost even though they were "saved" according to the apps that used the Instapaper API(or its browser extensions), it would take forever to download content over 3G ( maybe the client was downloading it? ). It just never got better for me.

They did introduce a killer feature(narration) and I switched to Instapaper from Pocket again for that feature alone (2 hours or commute time). Still annoying to use and as soon as Pocket implemented the same feature, I jumped ship again.

Pocket seems to be fast(at downloading content, and the iOS app is very responsive), their parser is improving and it's now by far my most used app on my phone.

I hope they get the resources they need to really improve their service though. They have lots of happy users and that must mean they are still doing something right.

jwr 3 ago 0 replies      
Well, they didn't use the word "journey", which is a plus.

But I'm worried. I find that increasingly my interests do not align with the interests of large companies looking for quick growth and large-scale "monetization". I would like to see more sustainable business models, where the goal isn't mad growth and sale to a larger entity, but long-term steady business.

idlewords 3 ago 2 replies      
dopamean 3 ago 1 reply      
If it's really going to be run as a separate property why does the team need to move to SF?
grandalf 3 ago 0 replies      
Instapaper is one of my favorite apps and I hope it doesn't change and that I can always access all my stored articles forever and ever.
nroach 3 ago 0 replies      
I felt a great disturbance in the Force, as if millions of voices suddenly cried out in terror, and were suddenly silenced. I fear something terrible has happened.
dmix 3 ago 2 replies      
Good opportunity to build an OSS alternative? Or even a side-project... in the possible event Instapaper gets sidelined. That always seems to happen. With the exception of maybe Reddit.
leonatan 3 ago 0 replies      
We'll read about it here soon:


reiichiroh 3 ago 1 reply      
Is Readability the corporation still solvent? They haven't had any app updates in over a year.
x1798DE 3 ago 0 replies      
I got an email to this effect today. I don't remember signing up to get emails, though presumably it was years ago if I did. I unsubscribed and marked as spam, but why is this something that required emailing people about?
elorant 3 ago 3 replies      
So what kind of value does Instapaper adds/will add to Pinterest? Will there be a pin it later functionality? I fail to see how Pinterest gains from such an acquisition.
d_theorist 3 ago 3 replies      
Is anybody else mildly annoyed by this trend of announcing acquisitions as 'x is joining y'? What's wrong with saying that y has bought x?
tedmiston 3 ago 0 replies      
I pay for premium today... I wonder if this line will disappear from the product page:

 Free Premium ... Support Instapaper's continued operation 

adamnemecek 3 ago 1 reply      
This is offtopic but am I the only one who keeps confusing Pinterest and Pinboard?
ryanmarsh 3 ago 0 replies      
I use Instapaper to highlight research sources from the web. Then export my notes to markdown using the iOS app. I hope to God they don't ruin Instapaper. I really need it.
Philipp__ 3 ago 0 replies      
Sad panda. I had amazing time with this app. Now I have to find alternative that will work well on iOS and macOS. :(
ryanmarsh 3 ago 1 reply      
Instapaper to me is the epitome of doing something that "doesn't scale", and yet it did.
leejoramo 3 ago 1 reply      
What do people use for the type of content extraction that the Instaparser API provides?
tempodox 3 ago 0 replies      
Too bad. It was nice while it lasted.
S_A_P 3 ago 0 replies      
romanovtexas 3 ago 0 replies      
Instapaper'd the link!
Cenk 3 ago 0 replies      
ihuman 3 ago 1 reply      
[Edited to be deleted by user since I can't delete this]
protomyth 3 ago 1 reply      
Its really not "joining" if you are cancelling a service as part of the announcement. What is so wrong with actually saying "we were acquired" and the new owner does not value some / all of our existing customers?
Amazon is piloting teams with a 30-hour workweek washingtonpost.com
320 points by djacobs  15 ago   233 comments top 36
gfody 13 ago 17 replies      
Something about this bothers me. Time-keeping is already not straightforward for an engineer. Because the work is done in our heads and doesn't necessarily stop just because the workday is over. You don't have to be sitting at a keyboard for your head to be grinding on a problem. More often some problem takes hold of you, occupying some mental bandwidth even while you're away, at home trying to enjoy time with your family or whatever - technically you're still working, your mind is preoccupied, you might not be able to sleep or when you do you dream about the problem.

From the outside it might appear like you came back to the office after a nice weekend break and quickly knocked out whatever task it was that was on your mind. But it's not that simple and after unloading that task, even though it's Monday you could be feeling like you need a break because you actually just worked through the weekend on it.

Because of this, I feel like engineers are already massively overworked and/or underpaid when you consider their salary based on a 40hour workweek when the real mental effort can be pushing 60-80 hours a week. Things like unlimited/discretionary PTO, flex hours, and management that understands the balance of overtime and undertime keeps things fair. Establishing a 30-hour workweek just seems like going hard in the other direction.

peatmoss 21 ago 0 replies      
The 75% pay / 100% benefits thing is likely the factor that prevents many companies from doing this. This is part of the reason I love the idea of a single-payer health system. If companies were no longer on the hook for the most expensive benefit, then more flexible wrking arrangements with concordance between work and pay would be possible.
mark_l_watson 1 ago 0 replies      
Makes sense to me. For most of my 40+ years of working, I worked a 32 hour work week and this was mostly for large corporations. I simply informed HR that I would not be working Monday's and to pay me 80% of my salary. It always amazed me that I was able to do this decade after decade, but the trick was that I worked really hard the 4 days I was in the office.

I mostly used the extra time for friends and family, and to write books.

BookingPotions 10 ago 1 reply      
As an amazon SDE myself, one who doesn't think working at Amazon is that bad, but do admit that it is challenging, and often do work much longer then 40h a week, I would need to have a guarantee that my pay cut comes with a no more then 30h a week clause.

I'd like to see proper hour counting, like a check in and check out. Where any hour above 30h comes at an extra cost to Amazon, like double pay. So that they would be incentivised to actually tell me to stop working and send me home.

I know some people might say, that's up to you, just don't let yourself work extra, but at a company like Amazon, you can actually lose your job or at least not be promoted from delivering less then the other employees. You're ranked against your peers, so deciding to work only 30h would hurt you in the long run if the others started putting in 35h, 40h, 45h, etc.

pjmorris 13 ago 3 replies      
'Amazon is piloting 25% pay cuts'.

I wonder if the group that does this will be viewed as lesser, e.g. work on less interesting projects, be less likely to be promoted, than the "40" hour employees.

smb06 14 ago 1 reply      
I think what they really mean is "30-hour workweek while being physically present in the office". I would find it difficult to believe that employees wouldn't continue working from home under pressure from management or simply because of tight timelines.
Falkon1313 8 ago 1 reply      
> "Even names like that, 'part-time' or 'reduced,' make it seem like a deviation from the norm, like you're doing less."

This is a telling statement. Why are we still stuck on a minimum of 40 hours being 'full-time'? After over a century of productivity increases, and with ever-increasing automation, we could soon be at the point where a 15-hour work week is the equivalent of an old 'full-time' work week. Now is a good time to start nudging down expectations.

The pay cut is wrong, however. The fact that people are producing much more now in 30 hours than they used to produce in 40 argues against that. If a company is profiting from the benefits of that productivity, but can't afford to pay the employees for their work, then it needs to change something else.

krisdol 14 ago 2 replies      
The folks I know at Amazon may have been in the office for a normal amount of time, but they worked long night after long night from home. If they're approaching a 30 hour work week as a reduction from 40, it's bound to fail. They have to realize it would be q reduction from 60 or 80
AlphaWeaver 11 ago 0 replies      
"Amazon chief executive Jeffrey P. Bezos owns The Washington Post."
amzn-336495 9 ago 0 replies      
What is likely occurring at Amazon is that the board has outlined that executive and line managers should be paid based on number of reports beneath them and this is a scheme simply to get more butts in seats so manager can be paid more. Before you say that can't possibly be correct, consider you don't know Amazon. At Amazon this sort of out in the open cheating or ability to game the system is seen as a mark of power. It's the same as cheating vendors or publishers. It's seen as a positive. Secondly the management culture is that employees are mere chaff to be used in any way possible to increase management compensation and power. There is absolutely no way this has anything to do with making Amazon better as a business.
yazaddaruvala 13 ago 3 replies      
So while this is really good for people that have dependents at home, or other consistently demanding activities. I would not enjoy such an arrangement.

The arrangement I'd be most excited by is a team with forced three month sabbaticals yearly (or honestly even 6 month sabbaticals). Of-course with the normal amount of vacation time added as well.

The way it would work is: Say a team is supposed to have 8 people. Hire 25% more people and schedule sabbaticals such that there is always ~8 people working. Additionally, you could ensure that there is never a time where two developers go more than 2 months without working together.

Benefits: Increases the supply in the job market; Reduces income disparity; Improves employee work-life balance.

spectrum1234 13 ago 1 reply      
This is fantastic news. It's absurd that it's basically impossible to have a successful career working a bit less than full time. This is way overdue in the modern world and hopefully becomes an option in more companies going forward.
kiddz 7 ago 0 replies      
Just wondering, but it strikes me as odd to have the Washington Post report anything about Amazon, since bezos owns them both. It's like Bloomberg reporting on Bloomberg's flirtation with a third party bid.
itaysk 4 ago 1 reply      
"These 30-hour employees will be salaried and receive the same benefits as traditional 40-hour workers, but they will receive only 75 percent of the pay full-time workers earn."

How does this sentence make any sense?

partiallypro 12 ago 1 reply      
I've been told by people that work for Amazon that they can barely keep their teams properly staffed and working over is just a common occurrence, I doubt a 30-hr work week is viable for a company that is more or less (now) a logistics company. Doing so would require higher headcounts, losing money, or both. Not a viable solution, even with the automation push Amazon has.

The only type of companies I could see getting away with this would be pure software companies or agencies. Otherwise, I can't see it fitting many models or personal finances.

namelezz 24 ago 0 replies      
> 30-hour workweek

Amazon, is that 30 in decimal or hexadecimal?

rezashirazian 12 ago 0 replies      
I have quite a few friends who work at Amazon and what I hear from them is that the quality of your experience is completely dependent on team you're on.

There are teams where all horror stories you hear are common occurrences(although less so in recent years) and others where it's an absolute pleasure.

elihu 7 ago 1 reply      
I work for Intel, and just a couple weeks ago switched to a 32-hour-a-week schedule. I had to get approval from my boss, and presumably my boss had to get approval from his boss, but it was a pretty easy switch. So far, it's been great. Benefits are the same, salary amortized at 80%, and stock grants are reduced a bit more than that.

I think most people would rather have the extra money, for various reasons. If I owed a couple hundred grand on a mortgage or was saving to send my kids to college, I might think the same but as it is I have no house and no kids and relatively low living expenses, so I can afford to be a little bit self-indulgent and take a 3-day weekend every week.

randyrand 13 ago 5 replies      
When I was at amazon I was routinely putting in 170hr+ work weeks. It was impossible.
sangd 14 ago 0 replies      
This is an excellent step to getting people more motivated and productive.I don't find working more hours improve my life in any positive way. It's much less satisfying as I want to commit some hours to learning or creating something new.
tdumitrescu 14 ago 3 replies      
3/4 pay for 3/4 butt in seat time. Seems like Amazon is getting a great deal here!
stefs 4 ago 0 replies      
i've been working a 30h/week job, 6 hours a day, with flex hours (without overtime pay but comp time) for the last couple of years and i absolutely love it.

last job i worked at had 42 hours and after 2 years i've been totally burnt out; now i'm motivated, usually well rested and concentrated. when my concentration drops i go home for the day, if i'm working on a hard problem that occupies my mind i'll stay a couple of hours more (but only if i want to, which i usually do). i haven't had to do crunch time for several years and when that was the case they asked if you wanted to volunteer and if you said no that was ok.

also if my project lead tells me there's not much work to do right now and if i want to take off time now would be a good time to do so i'll usually do it.

additionally, if you want to it's possible to work from home if there aren't any reasons speaking against it (meetings) - many colleagues work one fixed day a week from home (i don't because i love the quiet conditions and free, potent coffee).

in my opinion this benefits both me and my employer. i'm motivated, concentrated, productive and loyal - currently i can't imagine working somewhere else full time, even for higher pay. i do regard my employer as fair and really want the company to succeed, not only because of my workplace security but also because i think they're doing it right and that's how it should be done. there's no us-vs-them mentality.

pay is good but i'm probably not going to become a fabled startup millionaire here; quality of life is, in my opinion, unbeatable. currently i don't know any people who lead a more comfortable life than i.

nemesisj 7 ago 0 replies      
My company (Administrate - http://www.getadministrate.com) went to a 4 day, 32 hour week (but paid for 5) a little over a year ago. It's been a huge success, and we haven't seen any drop in productivity. We have had to be a little bit more efficient in some areas but overall the productivity gain was more or less "automatic". I'd recommend trying it to anyone, but understand it can be a scary thing to implement, mainly b/c it can be an easy thing to blame when things aren't going well. We have had some brief periods (usually when we're hiring) where teams have had to work 5 days, but overall we've been pretty consistent.
groaner 5 ago 0 replies      
I would much rather have 75% pay for 13 weeks of PTO. Highly unlikely anyone will ever offer that.
losteverything 9 ago 0 replies      
The most difficult thing to do is to stop work after 30 hrs (or whatever the time-to-be-paid is) It is counter to stop working if the job is not completed. But one must learn the discipline to stop.

Even if you can squeeze more productivity into 30 hours the temptation should be resisted.

The outfit I work for intentionally gives 9 hours of work to new employees and has them work an 7.5 hour day. It's intentional and brilliant. New people do not know they don't have to do 9 in 7.5. Old timers coach them but it is a very hard thing to tell someone to work slower.

I think Amazon will realize more productiviy in 30 than in 40 (in some cases)

atopuzov 3 ago 0 replies      
Cool, perhaps that's why I was required to work more and be oncall all the time, have meetings after 18:00, have no life, have high blood pressure.
techsupporter 12 ago 0 replies      
This is one of the things I really, really love about being in my current Ops/Engineering position: I have a defined shift. I work 4x10, fixed hours (flexible within certain rules, like if I'm running 15 minutes late or leave 30 minutes early, my boss doesn't come down on me), a few weeks of vacation, and no on-call. I still get to work on interesting projects and* I have people who work before my shift and after my shift who also handle tasks and projects so there's no institutional reason to come in early or stay late to pull extra hours. (Barring a major service meltdown, of course.)

To each their own, but this is why I don't want to dev for a living.

* - 3 day weekends every week is an awesome thing for me.

20yrs_no_equity 11 ago 0 replies      
This plus stack-ranking means 60 hour workweeks. I worked at Amazon for several years, their culture may pretend to support this, but, like almost everything you read about Amazon, it's PR fluff. Never going to happen.

Simply only showing up in the office 30 hours a week would be enough to put your team on the bottom end of the stack.

Amazon is organized such that the politics are vicious and anything that can be used to put another team down (And thus elevate your team in the stack) will be used.

Managers like Bezos are proud of creating this toxic cult like culture because they rationalize it and are not interested in hearing about how they are screwing up.

A real example of this is Bezos claiming after the NYT article that if anyone saw abuse they should email him directly... and now the ex-amazon alumni group has grown by several people who did exactly that and were fired.

My boss was committing felonies on the PacMed grounds on a regular basis, drove %80 of his team to leave, and he still got promoted.

Because he was good at politics and BS (and terrible at actually getting product done, easily wasting %25 of our time with nonsense because he didn't understand how the system worked but wanted to "manage" (which really meant micro-manage.))

Felonies, I'm not kidding.

ebbv 13 ago 1 reply      
I hate to be cynical (really, I do) but this seems like a clear attempt to soften Amazon's current public image as being a horrible place to work. It seems like all surface/headlines and no real substance. The headline makes you assume "Oh employees are going down to 30 hours a week but still being fully paid." because in order for this to be really noteworthy at all, it would have to be, right? But that's not what this is. They are basically just knocking people down to part time but keeping benefits, apparently?

That doesn't seem like a good deal to me. I don't want a 25% pay cut for 25% fewer mandatory hours of work. As other commenters have pointed out, unless this comes along with a reduction in responsibilities and/or increase in staff, the same amount of work still needs to be done. And most exempt employees already work more than 40 hours a week.

Add to this the fact that the Washington Post is owned by Bezos, and this just seems like a clear PR stunt to me, and a lame one at that.

freestockoption 13 ago 1 reply      
I thought Amazon employees were routinely burned out. At least that's the perception I get from reading the news. Maybe this is to help combat it?

What's next? Unlimited PTO? :)

sidcool 8 ago 0 replies      
With the speed and aggression that Amazon moves, it's going to be a challenge to implement this for Engineers.
balls187 8 ago 0 replies      
Get two jobs working 30 hours, each paying 75%, and you'll work the same as one demanding software job, and make 50% more!
mbloom1915 12 ago 0 replies      
guys lets not pretend Amazon doesn't operate in Europe and already have this setup - if its already working over there dipping their toes in the US is reasonable. it's a pilot.
megablast 9 ago 0 replies      
Not for the guys in the factories though.
DominikR 9 ago 1 reply      
Could it be that this has to do with Obamacare? I read that it is cheaper to have part time employees in the US right now, but I might be wrong on that.
jecjec 8 ago 0 replies      
The most important thing about this is that it will give other companies institutional cover to implement this within their firms.

I love my job but I have had jobs I've hated. Work sucks, go Amazon. If widely adopted, this would represent a massive reduction in taxation imposed on the average American family. A two-income family working 25% less hours pays over 25% less in taxes. Progressive taxes work both ways, Feds :)

Facebook AI Research Team Open Source DeepMask and SharpMask facebook.com
358 points by natthub  1 ago   79 comments top 15
said 1 ago 17 replies      
Is there any way for those of us with average intelligence to contribute to tools like this?

I know I'm a decent developer, but I feel entirely entirely inadequate to participate in this enormous, scary world of AI.

tectonic 1 ago 4 replies      
Future Facebook AR goggles will float people's names above their heads, replace billboard advertisements with live Facebook ads, and make people you 'mute' become invisible. It'll be a weird world.
iraphael 1 ago 0 replies      
These seem to be the papers for DeepMask [1] and SharpMask [2]

[1] https://arxiv.org/abs/1506.06204

[2] https://arxiv.org/abs/1603.08695

minimaxir 1 ago 1 reply      
NB: The libraries use Torch.

I'm still playing around with fasttext (which is amazing btw) officially announced last week so I'm surprised to see Facebook Research announce and release another project so soon.

dharma1 1 ago 0 replies      
It would be nice to see results compared to other SOTA semantic segmentation approaches like




guelo 1 ago 1 reply      
I feel less and less willing to upload personal photos to social media sites where they will stick around forever attached to my identity while advancing computer vision techniques extract more and more information from them.
spitfire 1 ago 1 reply      
This is very timely for me. Right at the moment I'm working on image segmentation for medical imaging.

If I can draft off tech developed to sell more ads to do some good I'm all for it.

visarga 1 ago 0 replies      
Is it safe to assume this release, as well as fastText are meant as PR for hiring?
wibr 1 ago 0 replies      
Looks very promising. Self-driving cars will need algorithms like this to understand as much as possible of the environment that they see through the cameras. I think currently things like those ropes that the shepherd is holding in the picture are still very difficult to detect and classify correctly but they could just as well be a line across the road, maybe with a sign on it saying "closed".
achr2 1 ago 3 replies      
How 'deep' are the networks used in something like DeepMask, and how does it compare with the number of layers of the human brain?
msie 1 ago 0 replies      
Wow, the code is in Lua! Just a little surprised. Also I just read some LinkedIn post on the dearth of Haskell education in schools. Always on the lookout for more companies using Haskell and hoping in the field of AI, machine-learning.
avvakum 1 ago 0 replies      
SharpMask looks very similar to a year-old "U-Net" http://arxiv.org/pdf/1505.04597
ChristianGeek 1 ago 0 replies      
What's the performance like? Fast enough to handle live video with a high-end PC?
Happpy 1 ago 0 replies      
Additional patent grand + bsd. Why not apache 2.0 or mpl 2.0?
swframe 1 ago 1 reply      
Is there a link to get the code?
Fighting Cancer hintjens.com
497 points by egorst  5 ago   148 comments top 31
Practicality 5 ago 5 replies      
Just an interesting related tangent here: My father in law is dying of heart disease (will probably die in < 1 month) and we run into the same thing.

People see me upset and the first thing they say is he will get better, or, it will be ok. When I explain the situation, it's something like, "oh, well, I just KNOW he will get better. Don't worry"


In case you are wondering: He has already had open heart 2 years ago, 4 of 5 arteries are completely blocked again (and those are already bypasses that are blocked again) and the remaining 1 is at 40% blocked. He found all this out when he went to one of the best heart hospitals in the world and they sent him home: "There is nothing we can do." Again, the open heart to bypass all 5 was only 2 years ago, so the math is pretty straightforward on how long the one artery will last.

He has something in his genetics in his family that makes it so this will happen pretty much no matter what he does.

The advice is endless. He has done his research and I believe he has had some real success in delaying this, he has outlived his younger brother by ~20 years (who died in a similar situation at 34). But it's over now.

I am trying to help his family get everything in order. And while it's sobering, it can be a very positive experience. But I wish our friends would stop telling us he'll be fine.

darod 4 ago 1 reply      
I found out I had colorectal cancer 4 years ago at 33. It was a shock because I go to the gym 4-5 times a week and taught brazilian jiu jitsu nightly. There no living right or living wrong prescription that will spare you. It can catch anyone at any time just like the author states.

Aside from dealing with the disease, one of the biggest issues I found was disbursing information and managing the emotions of my friends and family. Everyone has questions on your daily status and a few think they can come in a provide the superman holistic miracle that will spare you from death. It's tough to balance it all.

Like the author, I hate this notion of "fighting cancer". Norm MacDonald sums it up best. http://www.cc.com/video-clips/8kgu68/stand-up-norm-macdonald...

karmajunkie 5 ago 2 replies      
This is a really important read.

Nobody really gets terminal illness until you're either terminally ill, or right on the fringe of it. Even those of us who are one degree away from it only have the vaguest notion of it. It seems like something on the order of 30% of human prose ever written struggles with this notion of mortality [1], yet very few words are devoted to how to care about someone who is terminally ill, and even fewer on how to be there for them, providing encouragement without some kind of cheerleading. Part of this is because everyone is different, and some people really do want some of that. But my experience with it in this day is that those people are in the minority.

Thank you Pieter, for sharing your words with the world on this most personal of experiences.

[1] Totally made up statistic based on my gut feeling, so please don't bother asking for a citation.

nixarn 5 ago 2 replies      
A bit offtopic, but Hintjens has got the coolest activity graph on GitHub: https://github.com/hintjens
univalent 4 ago 0 replies      
Powerful article. Something I can share from my own ordeal. When you first get diagnosed most people are strong and defiant "I'm going to beat this thing!". It is the following weeks and months, the follow up scans that show the darn thing is back or not reduced that eventually wear you down. If you know someone that's afflicted please keep in touch throughout or space out your acts of kindness. I found that initially everyone wanted to help (an outpouring) and in later months some help would have been useful.
ciconia 5 ago 3 replies      
I've just saw my father die of cancer a week ago. Fortunately, he did this at home and was cared for by his own family, right to the end.

I wouldn't describe this experience as sad or tragic. We knew for a while that the moment was coming. I was lucky enough to be with my dad when he took his last breath, and to have been able to say goodbye. I learned a lot about what to do and what not to do in the face of imminent death.

baldfat 4 ago 0 replies      
My son fought for almost 5 years (Ages 7-12).

The outpouring of support was unbelievable. People I didn't know would do amazing things for my son and family. I saw this boy with one of the saddest stories (He was adopted) and a broken spirit before cancer became a amazing young man in the midst of his slow painful death.

His own biological family did very little during this time. Father murdered his mother less then a year after his diagnosis and family and close friends just didn't come around after a few months, "To painful to visit." I would flip out! Then I realized you just get to find out who is a true friend and family. So some will leave people high and dry others will see you all the way through.

jMyles 5 ago 4 replies      
A beautiful read, for many of the reasons already mentioned here.

I'm inclined to rethink this one question, though:

> Can a single individual patient second-guess the medical machine? Is that really their duty?

I don't know if it's anyone's duty, but I think it's completely plausible for a single patient or small group of patients to arrive at a more patient-focused conclusion than the medical industry.

newscracker 5 ago 1 reply      
> The only way to beat cancer, really, is to die from something else first.

That was a short and pointed article.

More so after I read a short comic strip on PHD Comics about cancer [1], I can't help but think that "beating cancer" is a very tough (and impossible) goal for the ones suffering from it and the ones looking for better management or reduction of it.

That shining light of optimism after remission is tinged with a hint that a recurrence is just a little while away, and could possibly be the end of life. I also wonder if what happens before death is more painful than the heartbreak that death eventually brings.

[1]: http://phdcomics.com/comics.php?f=1162

danieltillett 5 ago 1 reply      
A important post for those of us that don't know Pieter since it reinforces that your personal attitude does not change the outcome of cancer [1]. Cancer is a horrific disease, but it is not one that bends to our will, only our science.

1. http://www.apa.org/monitor/jan08/cancer.aspx

listentojohan 5 ago 0 replies      
I also read one of his previous posts on how to prepare the family, and talk with friends about dying. I've rarely been so moved by a post and had such an understandment of the situation, than from his posts. I think they are a must read for most people, as we'll most likely encounter it either through friends or family, or ourselves.
idlewords 5 ago 0 replies      
This is a very touching and generous post. I remember the kinds of diet and treatment advice my partner would get when she was fighting cancer, all of it well-intentioned, and wish those people had read this article.

I came to the thread to recommend a recent book by Atul Gawande, Being Mortal, http://atulgawande.com/book/being-mortal/, which covers the difficulties of dying (from age or from illness) and touches on many of the same points as this amazing post.

csl 5 ago 0 replies      
If you haven't heard it already, there is a really good and candid podcast interview with Pieter Hintjens over at Software Engineering Daily: http://softwareengineeringdaily.com/2016/06/23/death-distrib...
pixelmonkey 5 ago 1 reply      
Related: A Protocol for Dying, an interview with him for The Changelog from June 2016.


kohanz 5 ago 0 replies      
An excellently written piece, with a perspective that can only be communicated by someone who is walking that path (so to speak). The thought that the whole "You WILL get better" or "keeping fighting!" notion highlights for me is how taboo of a subject death and dying is in many cultures (very much including Western culture). I truly believe the reason people say such cruel and selfish things is because they cannot bring themselves to talk about the topic of death. It's something we are taught to ignore until we cannot possibly do so any longer. I feel like we might live better lives if we talked about death and dying openly and throughout our lives.
jwdunne 5 ago 1 reply      
In a way, it's similar in nature to how people will, with best intentions, tell you to smile more if you have depression.

I think the issue is many people don't understand the problem and they can't see it. In fact, someone with a common cold can expect to get more sympathy and better advice than someone with a chronic illness. The second issue is that many people struggle to think of something to say, its quite uncomfortable and the automatic choice is to give advice.

FuNe 4 ago 1 reply      
" There are people who treat the dying as easy prey. "For some reason this hurt most. Maybe I expected way too much humanity off humans.
reactor 5 ago 5 replies      
Very sensible read, just one question, he mentioned "avoid junk foods, especially sugar", is sugar that bad for causing cancer?
milesf 4 ago 3 replies      
I lost both my parents last year to cancer. Both were Christians, and so am I.

I know many people find Christianity and the subject of faith to be uncomfortable, even offensive. But that's because the Christian message _is_ offensive. It makes claims that exclude all other options, that evil is real and that we are responsible for it. To me, either the message is true or it isn't. There is no grey. Either Jesus Christ was a liar, a lunatic, a legend, or He is Lord God Almight.

Penn Jillette, the famous atheist and half of Penn and Teller had it right, that if we removed all the scientific research in the world we would be able to rebuild it all, but religions would be all different. I agree with him, and so does the Bible. It says that God reached down to us, delivered messages in ways that statistically rule out purely human effort, and gave us a choice to trust Him or not (have a look at http://thebibleproject.com) . In the end, everyone's going to get what they want (if you want Jesus Christ you get him, if you don't you won't). That's why for my Dad and I, we both had to be convinced that the Bible was not simply human in origin.

Whatever your view, I can only speak to my own experience. The loss of my mom September 1st, 2015 to double-hit lymphoma was very, very painful (she was 68). But in the midst of the pain was a hope and peace as explained in Philippians 4:4-9 (http://bit.ly/phil4_4-9). Then, unexpectedly 120 days later, my dad died from lung cancer (age 72), leaving my brothers and sisters and I with a property and 47 years of marriage and memories for us to sort through and deal with.

We are all going to die. The question is not if but when. To put off the discussion about what happens after you die is to deny reality itself, and telling others not to have that discussion or that their position is stupid or foolish is really dumb. Oh, and in case you think the Christian message is foolish, the Bible agrees with you that it is http://bit.ly/1cor1_18-25

arisAlexis 5 ago 2 replies      
I am not sure why cryonics are not in the menu of every reader here when he has the opportunity (you don't have it if you die in a sudden accident) to subscribe when it seems inevitable to die.
_nullandnull_ 4 ago 0 replies      
Every time I see his posts I think about the book The Tibetan Book of Living and Dying. I would recommend reading it for anyone who might be having the discussion or dealing with death in their life.


raarts 4 ago 0 replies      
I have the utmost respect for Pieter, wherever I encounter his work it always shows passion and quality. Recently I read The Psychopath Code which I highly recommend especially to those who think they've never encountered one.While you can still read this: thank you. You have improved my life in multiple ways.
alfonsodev 4 ago 0 replies      
"..Yet you are only as strong as the work you do"

All the article is very inspirational, mundane things like not solved paperwork, can carry lots of head aches to the family, sadly I know it by experience.

nxzero 4 ago 0 replies      
Yes, all deaths are tragic, but increasingly feel that humanity does not have a good measure for priortizing its efforts.

As an example, over half of those diagnosed with cancer are over 70 years old and 100s of billions have been spent on research.

What is an object way to decide if all the effort spent of cancer research is of value relative to other area of research where progress might be made?

newuser1111 4 ago 0 replies      
I want to say this story is personal for me.
codingdave 5 ago 0 replies      
Some of this advice applies to people with chronic non-terminal health problems as well. We may not be dying, but the "helpful advice" from people who hear we have problems usually doesn't go over very well with us, even if we do smile and say thanks, to be polite.
Kenji 4 ago 1 reply      
Unfortunately, despite what this article says, there is a choice between fighting cancer or not: The best way to give up is stopping to eat (which probably isn't that hard if things like chemotherapy and severe illness pretty much remove any feelings of hunger).
dredmorbius 5 ago 1 reply      
I've been appreciating Pieter's posts, and his occasional comments on HN (I noticed one a few days back, on technical topics). I'm also increasingly apprehensive opening them. Though the only fatal disease I'm aware of fighting right now is life, my hope is to pursue my interests so long as I can.

I wasn't aware of Pieter before his recent blog topic started appearing on HN, though I'm pretty sure I'd come across his work. We're focused on different areas of tech.

I have seen cancer though, and much of what he writes here hits home, hard. I lost a very good friend, far too young, several decades ago. I'm looking at their picture now.

And remember going through much of what Pieter describes, though not as the central participant.

There were the other patients we met through treatment. Some of whom made it, some of whom didn't. And it wasn't necessarily those who appeared strong who lived.

There was the cheerleading and denial and people who were meddlesome. Those of us around the patient and family did what we could to steer the away. As Pieter says: the doctors tend to know the medicine pretty well (though chasing after them when things clearly aren't going right may be necessary). Unsolicited medical advice at this stage is almost always quite unwelcomed.

Small things can be huge.

What I remember, most, still: meals that showed up on the back porch with heating instructions. The neighbors had arranged amongst themselves a cooking schedule and coordinate this. No asking. No fuss. It just happened.

One less thing to worry about.

The other thing I remember was someone who, in all sincerity and good intentions, had forwarded information on a possible meracle cure. Laetrile. "The slickest, most sophisticated, and certainly the most remunerative cancer quack promotion in medical history," Wikipedia tells me today. We didn't have Wikipedia then, but I quickly established that this was in fact bunk.

It still makes me furiously mad: preying on sick people and those about the clinging desperately to any possible hope, in full knowledge that you're peddling bullshit. And those who get swept up in this and pass on the misinformation. Maybe that's why I've cracked down on online disinformation as well. It's not just duty calling....


Pieter's comments on how cancer is "fought" are also extremely good counsel. Some things can be manipulated and addressed directly. For others, you can only hope to set up the right set of circumstances to achieve the outcome you desire -- fighting cancer is more like tuning algorithms or seeds for some stochastic process -- a raytrace render or algorithmic music render, say, for those familiar with them -- than aiming a rifle at a target and taking shots. Our ability to directly influence events is limited, mostly you're managing the bits about you, your environment. Staying comfortable, staying sane. So much as possible.

In describing dealign with those around him. Pieter reminds me of a general classification I've used in other contexts for people:

* There are those who mean to do well, but are unable to. The cheerleaders and advice givers tend to fall into this category -- their harm isn't intentional, but it can be real all the same.

* There are those who cause problems through their own systemic operation. Healthcare insurance systems, vendors, legal processes, and the like. The issue's less one of having malicious intent, though here it's a lack of sensitivity to what their impacts on others are, or simply failing to care. The impacts on those who are sick or disabled are hugely magnified.

* Finally, there are those who are actively evil. Scammers, predators, sometimes even family or neighbors angling for what they hope they might be able to gain. This again makes me sick. There are no pits of hell deep or hot enough.

Many years after the experience I'd mentioned above (and after several others), I found a good model for offering care -- it's the concept of a kvetching order:


This consists of a set of concentric rings around a trauma, with the afflicted person at the center, and a growing set of less-affected care- and support-givers extending out. The basic idea is that care flows in, kvetches flow out:

The person in the center ring can say anything she wants to anyone, anywhere. She can kvetch and complain and whine and moan and curse the heavens and say, "Life is unfair" and "Why me?" That's the one payoff for being in the center ring.

Everyone else can say those things too, but only to people in larger rings.

Those who cannot (or will not) grasp and follow the concept are excluded.

The article also has another really wonderful piece of advice: that sometimes simply listening is the support that's needed. I've been on both the giving and receiving sides of that, and I'm not aware of when it's not been appreciated (though as with other advice -- people may differ, be sensitive to their needs).

One more thought: at least in Western cultures, there's often a profound lack of awareness of how to deal with death, impending death, or recent death. That's something which could use improving (and no, I'm not suggesting a YC opportunity). I very much appreciate Pieter's occasional communications for helping with that, at least here.

olantonan 4 ago 1 reply      
Quitting Twitter. Not quitting Twitter.

Dying. Not dying.

Sustrik is God. Sustrik is Satan.

What's up with this guy.


shanacarp 4 ago 1 reply      
First, for the general Reader:Outside of really breast/ovarian cancer, getting most common cancers at a young age is fairly within your control.http://www.nytimes.com/2016/07/06/upshot/helpless-to-prevent...

Pieter doesn't have a common cancer and he's way beyond that stage within stage IV cancer. In general it's really rude to say "fight more" because he's right, what do you think these patients are doing instead, joining the circus?Fighting isn't a great metaphor. The reason it got introduced was so that Reagan would create the National Cancer Institute, a Division of the NIH. A full page ad was taken out by Mary Lasker and the American Cancer Society to convince him that researching the causes and cures for cancer was a US healthcare priority (and to be frank, at the time, it essentially was) asking him to declare war on cancer in the NY times because the US was in the middle of the cold war. The other thing that the add did (along with the creation of the Jimmy Fund) was it normalized discussions of cancer in the US at that time.To explain how much of a big deal that ad was, my maternal grandmother died around 1973-1974 of cancer. The ad came out in 1969. My maternal grandfather is only now settling the argument if it was metastasis of her internal breast cancer from when she was younger, new breast cancer that spread to the bones, or a totally new bone cancer, because now we talk about cancer, whereas in 1969 -73 talking about cancer was difficult if not possible.


Aa a personal note to Pieter, if he sees this:1) I'm extraordinarily happy you are doing as well as you are in your end of days and I hope you are enjoying them to your fullest. i hope, for whatever it's worth, you are still experiencing moments of joy too.2)im slightly concerned as an American about your distrust of marijuna at this stage, especially since it seems like low pain and enjoying food is a high priority for you. In the US, marijuna is partially approved (don't ask) for cancer patients as an appetite stimulant and pain suppressor, and many of the chemicals in it are made synthetically and prescribed to cancer patients for the same purpose. Meanwhile, many opiates are appetite suppressants (that's the other reason behind the Medrol). Since you deserve to enjoy your time and have as many thalis as you want, just think about it. (Again, I'll totally admit that this is a bias of seeing Americans treated)3)do you need help getting the paperwork done. On a percentage scale, how much is left? How much can be done by volunteers/family/friends? (And I hate asking this, how much is in English, because I'm happy to volunteer, but I'm an English speaker...)4)thalis. Mmmm. Thalis.

frign 4 ago 2 replies      
Types github.com
438 points by ivank  3 ago   190 comments top 22
seanwilson 3 ago 4 replies      
Great overview article but I have a comment.

> In Idris, we can say "the add function takes two integers and returns an integer, but its first argument must be smaller than its second argument":

> If we try to call this function as add 2 1, where the first argument is larger than the second, then the compiler will reject the program at compile time.

> Haskell has no equivalent of the Idris type above, and Go has no equivalent of either the Idris type or the Haskell type. As a result, Idris can prevent many bugs that Haskell can't, and Haskell can prevent many bugs that Go can't. In both cases, we need additional type system features, which make the language more complex.

I really wish when people talk about dependently typed programming languages (e.g. Idris, Coq, Agda, ATS), they would mention the effort involved to get these compile time checks to happen.

These languages allow you to capture almost any program property you can think of as a type. Instead of being limited to saying function F "returns a number/list/string" like common type systems, you can capture arbitrarily complex properties like "F returns an even number", "F returns a permutation of the input list", "F always halts" or "F returns a proof of Fermat's Last Theorem". You don't get this for free and you have to help the computer verify these.

Proving F matches the type is arbitrary hard and is impossible to automated in general (e.g. the halting problem tells us we cannot always tell if a function halts). For example, consider the type "F returns a sorted permutation of the input list", where F could be an implementation of bubble sort, quicksort, merge sort, radix sort or something even more complex. Each algorithm will require a different and potentially complex proof and most of the time the programmer needs to help the computer find the proof. It's completely unlike what people are used to from type systems and is an even more massive leap than for someone moving from the type systems of e.g. C++/Java to Haskell.

I'm glad people are more aware of dependently typed languages but I feel the immense jump in how practical they are to use is severely overlooked.

twblalock 3 ago 8 replies      
The Idris example seems to need further explanation:

> In Idris, we can say "the add function takes two integers and returns an integer, but its first argument must be smaller than its second argument":

> add : (x : Nat) -> (y : Nat) -> {auto smaller : LT x y} -> Nat

> add x y = x + y

That's all well and good, if you know the values of x and y at compile time. Consider a program that reads x and y from STDIN. The user could provide an x that is equal to or larger than y (or could provide only one value, or values that are not numbers). I see no way to deal with that except to throw a runtime error. Is that what would happen?

naasking 3 ago 4 replies      
Decent overview of many concepts, but the opening line isn't strictly true:

> A type is a collection of possible values.

A type is a proposition, and "This binding has one of these possible values" is merely one type of proposition. So a type is much more powerful than a simple set-based interpretation, even though this is how most people think about it.

For instance, in Haskell you can encode a region calculus that ensures prompt reclamation of file handles and other scarce resources [1]. The relations between nested regions to ensure correctness seems awkward to interpret as a set, but it's straightforward if you just think of types as a relation.

[1] http://okmij.org/ftp/Haskell/regions.html#light-weight

junke 3 ago 1 reply      

 1 + eval(read_from_the_network())
> If we get an integer, that expression is fine; if we get a string, it's not. We can't know what we'll get until we actually run, so we can't statically analyze the type.

> The unsatisfying solution used in practice is to give eval() the type Any, which is like Object in some OO languages or interface {} in Go: it's the type that can have any value. Values of type Any aren't constrained in any way, so this effectively removes the type system's ability to help us with code involving eval. Languages with both eval and a type system have to abandon type safety whenever eval is used.

No, you can give EVAL a returning type of any (T) but still make type inference work.

 (+ 1 (eval (read)))
The meaning of a type in SBCL is: if an expression evaluates without error, then its type is ....In other words, the type of the above expression is NUMBER.

There is also a NIL type (bottom) which represent the empty set of values. If a function has a return type of NIL, it means that it does not return a value normally. This is the case for the ERROR function, or the (LOOP) expression.

wz1000 3 ago 0 replies      

 add : (x : Nat) -> (y : Nat) -> {auto smaller : LT x y} -> Nat add x y = x + y
Actually, (GHC) Haskell can express something like this type if you turn on the required extensions.

 data Nat = Z | S Z deriving (Eq, Ord, Show) data SNat (n :: Nat) where SZ :: SNat Z SS :: SNat n -> SNat (S n) type family Compare (a :: Nat) (b :: Nat) :: Ordering where Compare Z Z = EQ Compare (S a) Z = GT Compare Z (S a) = LT Compare (S a) (S b) = Compare a b type a < b = Compare a b ~ LT type family (+) (a :: Nat) (b :: Nat) :: Nat where Z + a = a a + Z = a (S a) + b = S (a + b) add :: (a < b) => SNat a -> SNat b -> SNat (a+b) add = ...

DSMan195276 3 ago 1 reply      
Being someone who enjoys C (Though I readily admit the type system is generally weak compared to the others on the list) there's a little misinformation on this page that I think it worth clearing up - though I think the majority of the information is still perfectly good:

C does not 'allow' you to do a lot of the things mentioned on this page, it just doesn't generally carry around all the information to check and make sure you don't. So the page is generally still right - it can be fairly easy to get memory-unsafe things past the compiler - though whether you want to blame the type-system or blame the compiler could be debated, probably a bit of both.

Reading past the ends of arrays is illegal, as is accessing a variable of one type through a pointer to another type. Some of these things will get checked or assumed by the compiler (And thus break code that does these things, and in some ways break them in subtle ways), but there's no guarantee that the compiler actually knows you broke the rules. This tends to be the main different from other languages like Java, where the compiler/run-time does carry around such information and thus can do such checks and does know you when you broke the rules.

With that said, while there are some things I don't like about the design of Rust overall, giving the compiler more information to be able to figure out the above is definitely one thing they did right in comparison to C. The problem C is approaching is that, even though compilers are getting smarter, enough code was already written before that was the case to result in breaking when you attempt to use 'smarter' compilers.

arianvanp 3 ago 1 reply      
> If we try to call this function as add 2 1, where the first argument is larger than the second, then the compiler will reject the program at compile time

This is not entirely true in general. auto is only a best bet. It might not be able to prove something right and will fail. For example, if it has to recurse deeper than 100 constructors. In this case, the user of the function has to provide a proof himself.

See it as a bloom filter. if it doesn't fail, you're sure your function is correctly called. if it does fail, it might still be correctly called but the compiler just didn't have enough time to prove it. In that case a user needs to convince the compiler that he is right.

titanomachy 3 ago 3 replies      
Someone who has experience with both Agda and Idris, could you comment on which is easier to get started with? Coming from a Haskell background, I'd like to try my hand at writing more interesting constraints into my types. Any experiences using these languages for (non-research) work?

I've played around with Coq a little and it certainly feels more like a proof assistant than a programming language. It was fairly confusing and unfamiliar, but interesting nonetheless. Something with slightly more traditional ergonomics might be easier to pick up.

one-more-minute 3 ago 1 reply      
If this topic interests you, Julia's type system is worth checking out [1]. Julia is unusual among dynamic languages not just for having a powerful type system (including parametric types), but also for making it idiomatic to make full use of types. Aside from the benefits to the programmer, this also allows the compiler to get to Rust-level performance where it's needed.

I find that this is a nice compromise that gets a lot of the benefits of static type systems (small, local errors as opposed to behavioural bugs, ability to express concepts and constraints) without the drawbacks (loss of interactivity, or inability to escape the type system when it's appropriate).

[1]: http://docs.julialang.org/en/latest/manual/types/

zeveb 2 ago 0 replies      
> The unsatisfying solution used in practice is to give eval() the type Any, which is like Object in some OO languages or interface {} in Go: it's the type that can have any value. Values of type Any aren't constrained in any way, so this effectively removes the type system's ability to help us with code involving eval. Languages with both eval and a type system have to abandon type safety whenever eval is used.

It's not quite true that they must abandon type safety: rather, they must abandon a certain kind of compile-time type check. It's not unsafe to use eval or Any values (i.e., it won't crash the system); one must simply examine the Any value (at runtime, natch) and do stuff with it.

athenot 3 ago 0 replies      
I've been very curious about Perl6's gradual typing[1] which, as I understand it, draws a lot from Haskell's type system but in an optional manner.

 subset NonNegativeInt of Int where * >= 0; sub fib(NonNegativeInt $nth) { given $nth { when 0 { 0 } when 1 { 1 } default { fib($nth-1) + fib($nth-2) } } }
[1] http://blogs.perl.org/users/ovid/2015/02/avoid-a-common-soft...

levbrie 3 ago 2 replies      
Kind of shocked nobody mentioned this, even if it is a bit of an aside, but umm, I've been dying for anything at all from Gary Bernhardt - I don't even know what to say except that it makes me hope however unrealistically that will one day get something like the magnum opus that is "Destroy All Software" from him again.
DanWaterworth 3 ago 2 replies      
> Haskell has no equivalent of the Idris type above, and Go has no equivalent of either the Idris type or the Haskell type. As a result, Idris can prevent many bugs that Haskell can't, and Haskell can prevent many bugs that Go can't. In both cases, we need additional type system features, which make the language more complex.

Type checking Idris is much simpler than Haskell.

NhanH 3 ago 3 replies      
The article is mainly focused on static typing. I guess it makes sense since the title is "Types". Although reading this, you might just think that dynamic typing is good for nothing.

The more practical part of the article is great. However, the theoretical part could use some works. Especially the terminology isn't actually clear.

Takes some example, what is "memory-safe" ? The only example makes it sounds like bound checking. And what does "no way to escape the language's type rules" mean? There are some usages of "valid program" and "invalid program" that makes my spider sense tingling as well.

> However, no dynamic language can match the speed of carefully written static code in a language like Rust.

> Any blanket statement of the form "static languages are better at x than dynamic languages" is almost certainly nonsense.

Those two quoted statements come from the article.

The section "Arguments for static and dynamic types" didn't mention a single reason why dynamic typing could be preferable over static-typing. It could at least have mentioned macro, or hand-waving claim that dynamic typing makes for prettier code at time (I think the author of python's requests library made that argument in one of the blog post).

I think the last part should be removed (ranking the language by their typed system power).

> There's a clear trend toward more powerful systems over time, especially when judging by language popularity rather than languages simply existing.

Well, it looks true since we didn't consider any dynamic language at all...

Focusing more on the "Concrete examples of type system power differences" would have been better for the article, I think. Something along the line of an example of Non-nullable type with something simple in TS, to optional/maybe monad, then dependent type that you can implement matrix multiplication (m * n) with (n * p) and got the type (m * p) back.

ozy 3 ago 6 replies      
So what might make dynamic languages easier to write in? I enjoy dynamic languages more because:

* "double kilometerToMiles(double km) { return km / 1.6 }" here the types are of really low value, yet I have to type them in.

* a mutable list of generic and variable arity event handlers. Really hard to specify as a type (most languages cannot do it). Really easy to use.

* co- and contravariance and lists, the math works exactly opposite of intuition.

* when you change your mind on the types/arity beyond what your IDE can follow, it is a lot of low value work

* if your test suite has 100% coverage of arguments/return-value and field usage, you have type checked your program.

If I am going to write down types, they better be of value to me. Like types that are null or non-null, or tainted (from the user) or non-tainted. Or two numbers that are kilometers vs miles. Or only accessible under a certain lock or other preconditions. And be flow sensitive (kotlin style null checks), not make me write even more code.

But if you are a dynamic language, do be strong typed (no "1" == 1). Also be dynamic in arity, that is (part-of) a type. Don't be handwavy scoped, lexical scoping is probably the only predictable way to do scoping. Fail fast, like prevent spelling mistakes for non existing field names for example. And allow this to work: "2 * Vector(10,10)".

And from that perspective there is still quite some room for improvement in all classes of languages, I suppose, and good that there is lot of movement and experiments today.

lqdc13 3 ago 0 replies      
I think types can be both frustrating and life-saving.

However, the "danger" for me lies primarily in implicit conversion. That is, Python vs Lua/JS/R. When writing large programs, some things happen implicitly and linting may not catch it.

Everything else besides implicit conversion just helps structure the programs in specific ways and helps speed them up.

rpazyaquian 2 ago 0 replies      
What types are and what they do is all well and good, but what I want to know is how they would help me write a larger application. Would it help it be more correct, be written faster, or be more flexible? Does it just help on a line-by-line programming basis, or can it have any ramifications on a higher level like product design?
eximius 2 ago 1 reply      
So, how do these advanced types work?

For their `add` example, I imagine just having x<y somewhere is sufficient. But let's say I write a sorting implementation but the compiler can't quite figure out that it is correct. Is there a provided way to tell the compiler 'trust me, this property is now true'?

Also, I'm assuming the compiler has two classes of errors here: a counter example vs. unable to prove?

Can anyone point me to documentation on these (in Idris, preferably)?

mamcx 2 ago 0 replies      
I read that type checking is easier with dependent types.

I'm barely aware of adga adn other langs that are used for profs, but wonder how useful (and what danger ahead) could cause to design a language with this for more mundane purposes (like, for example, build websites, data manipulation, gui, etc)

kpmah 3 ago 0 replies      
I wrote a little article, which has been posted here before, that shows the power of different type systems and what kind of bugs you can expect to prevent: http://kevinmahoney.co.uk/articles/tests-vs-types/
amelius 3 ago 4 replies      
> A type is a collection of possible values

Ok, but do two collections with the same values always correspond to the same type?

th0ma5 3 ago 1 reply      
Hope this makes it into Wikipedia instead of here.
Massachusetts to tax ride-hailing apps, give the money to taxis reuters.com
337 points by petethomas  6 ago   509 comments top 55
rev_bird 6 ago 15 replies      
This is a pretty surreal move:

>The law says the money will help taxi businesses to adopt "new technologies and advanced service, safety and operational capabilities" and to support workforce development... [Larry Meister, manager of the Boston area's Independent Taxi Operator's Association] said the money could go toward improving a smartphone app his association has started using, or to other big needs.

I'm not an anti-regulation guy most of the time, but this doesn't even seem like regulation -- they're literally giving money to a fading industry because they are incapable of keeping up. I don't want to see cab drivers turn into homeless drifters, but this just seems wrong.

Making a more efficient competitor pay to help taxi companies make their service better? The whole reason these companies exist is because taxi companies have been utter garbage for years and got the government to elbow out competitors. Am I missing something? Surely the burden for "do things better and buy nicer equipment" falls on the taxi companies, not the other organizations that are embarrassing them.

ricw 6 ago 3 replies      
This is unbelievable. Never have I used a more corrupt and unfriendly taxi service as the one in Boston / Cambridge. And they are supposed to be subsidised?!

I have 1) been driven round crazy detours to up my cost2) been forced to drive a detour because the card machine was broken and they only accepted cash3) been turned down because my destination wasn't far enough4) been shouted at because I was watching the free tv in the cab

In short, they suck. Badly. Thinking about it, I can't remember a positive experience. In contrast I've hardly had a negative experience with lyft/uber and mostly had very positive or positive experiences.

What's the best way of contacting this senator?!

jwallaceparker 6 ago 4 replies      
You and a friend are stranded on an island. You both spend all your working hours catching fish by hand in order to eat.

You use some leisure time to build a fishing rod, which enables just one of you to catch enough fish to feed you both.

It would be foolish to destroy the fishing rod to preserve both your jobs as fishermen. One of you would fish and the other would do something else, like build shelter and cut firewood.

It's just the two of you, so you don't literally exchange in barter, but an implicit exchange takes place when you share the fish, shelter and firewood.

You both enjoy a better standard of living thanks to the productivity increase created by the fishing rod.

A modern economy is much more complex but the same fundamental principle applies.

The ride-hailing apps make taxi drivers more productive by helping them connect with customers faster.

Now it takes fewer taxi drivers to satisfy the demands of customers. This frees up the labor time of the remaining taxi drivers to produce additional goods or services.

There is certainly temporary pain endured by workers who are displaced by the innovation. But it is folly to subsidize work that is no longer in demand.

akulbe 6 ago 1 reply      
If the taxi cab companies can't make it on their own, they should go out of business.

As a self-employed person, I shouldn't expect someone else to bail me out by their efforts. If I can't make it on my own steam, I close up shop.

When this happens, it's unfortunate, and unpleasant, but it's a simple fact of doing business.

Our parents and grandparents would balk and roll in their graves at the thought of this idea that someone else should shoulder the burden of bailing you out.

What ever happened to the idea of (in the case of business hardship or failure) picking yourself up, dusting yourself off and starting OVER??

cloudjacker 6 ago 2 replies      
During periods of extremely high surge pricing, I take regulated taxis instead. At first I think, 'wow I love competition its such a good thing this alternative service was around' but then I try to pay the cab driver, here is the experience in 2016:

- The meter system is broken

- The cab driver presses buttons frantically on a black box before resorting to hitting it in hopes that it responds to external stimuli

- The cab driver is unsure how to process credit cards (or even cash) in the event this black box does not start responding "Uhhh, ahhhhh, this stupid thing"

- If black box does begin responding, then lets advance to the credit card processing part, which also doesn't work

- FUTURISTIC WORLD CITIES in the US have taxis with screens in them so the rider can do the credit card processing on their own, except this doesn't work either "your screen isn't responding..."

brownbat 6 ago 3 replies      
There are two competing narratives about ride-hailing apps success:

1) They are profitable because they are finding loopholes in relevant legislation, or,

2) they are profitable because they are injecting competition and new ideas into a stagnating industry dominated by government-protected monopolies (or oligopolies).

Your opinion of the law will likely depend on which narrative you find more persuasive.

bawana 6 ago 2 replies      
The capitalist model, the concept of 'disruption', Adam Smith's precepts work well when they allow individuals to control their choices, their fate, their destiny. When the transactions exceed human scale - when they become so big, so far reaching, so expensive that the participants are powerless to make changes, then capitalism fails. It fails because the 'losers' suffer even if they try to adapt. They cannot compete because the the 'winning side' is so new with fewer legacy costs. The losers necessarily must die and become assimilated by the winning strategy. And the winners who skirt the regulations will close shop and disappear at the first sign of trouble.

I disagree with the concept of entitlement-taking money from the winner and giving it to the loser. But the winner MUST abide by the same rules as the loser. If the ride sharing industry has the advantage of venture capital startup money and newer (less costly to maintain) infrastructure, then their benefits to drivers should be GREATER than what the taxi industry provides, not less. The 20 cent tax should go entirely to the health care and insurance of the drivers of the ride sharing industry's drivers. Not the welfare recipients we call 'politicians'. That HALF of this tax goes to people who are ALREADY living off of money made by people who actually work really bothers me.

I would really like to see silicon valley startups that offer a new and better forms of local government. I would like to see our political system get improved by disruption. I really believe politicians are vastly overpaid for the non-service they provide.

mungoid 5 ago 1 reply      
I haven't read all 400-some comments, but in the several pages worth i read I haven't seen a single person mention looking out for their own welfare and keeping their future in their own hands. Just because a line of work has been around for decades doesnt mean it is always going to be a safe bet.

I've worked hard for the past 10 years to get to my middling-middle class income. Enough with my wife's to support our family. I could just be content with that and staying right here for the rest of my life, never wanting to do any more with myself, but I know things will change and i want to be prepared for it.

Maybe it's the brain wiring of a software developer, but I refuse to become complacent with where I am at. I already know I wouldn't last another 10 years if I don't regularly adapt, learn new technologies, stay current.

I think in a way, this isn't too much different than the housing bubble. Too many people thinking that everything is just going to keep going its steady pace forever and if anything bad does happen, someone will come to the rescue. I want to position myself so I don't need that rescuing.

a_c 6 ago 2 replies      
If this logic make sense, perhaps AirBnB need to be taxed to support hotel industry, Iphone should pay to nokia and internet should be taxed to landline
appleflaxen 6 ago 0 replies      
When obsolete industries / models / methods become replaced by something better, the best thing that can happen at a macro level is to have them phased out as smoothly as possible. If that smoothness can also be fast, then you will maximize your economic productivity as well.

So this decision is exactly the opposite of what you want... you are going to prolong the inevitable with a market-distorting subsidy.

It would be much better to tax Uber and require the funds go to taxi cab retraining scholarships, so that people who couldn't make a living driving a cab anymore have another option. Better yet: send them to customer service school, so that they can work as an Uber driver if they wish to do so.

lpolovets 6 ago 3 replies      
Obviously the logic is mystifying, but the math behind this is baffling as well:

https://en.wikipedia.org/wiki/Taxicabs_of_the_United_States#... says there are 1825 cabs in Boston. Boston is 10% of Massachusetts' population, so let's say there are 18k cabs in the state.

http://www.reuters.com/article/us-massachusetts-uber-idUSKCN... says 5 cents/ride-hailing app will go toward taxis, and Uber+Lyft do 2.5m rider per month.

So, the tax will produce 2.5m rides/mo * 12 months * 5 cents/ride = $1.5m/year.

That works out to... drum roll... $83 per taxicab per year. How is this supposed to help taxi drivers with anything? Maybe as ride-sharing increases, this becomes $200 or $400 per cab per year, but that's still a small sum. I imagine a few hundred dollars is a small consolation when you lose 50% or 90% of your business.

owly 6 ago 0 replies      
The 5 should be allocated to public transportation to keep late night bus service. This is pure corruption.
trengrj 6 ago 1 reply      
I'm in Sydney Australia and we have a one dollar fee for each Uber to bribe off taxi medallion owners. It feels completely wrong and Uber makes the fee separate on your card I think in protest.
qq66 6 ago 2 replies      
I think that this makes sense in concept if not in implementation. Taxi medallions were sold and traded under a certain set of laws which defined their value. The government decided to change those laws, so "buying out" the medallions at some fair price, paid for by the beneficiaries of the change in law, makes sense.
IkmoIkmo 6 ago 0 replies      
I fully understand one may tax a new successful industry, and then use the proceeds for individual citizens who lose jobs and need some support, training and job programs to help them back on their feet.

But this is ridiculous. The notion an uncompetitive industry, which is fading because ordinary people can and do choose a better product or service, gets money to stay alive with its shittier product, just makes no sense. It doesn't achieve what we want, and there are good alternatives.

MicroBerto 6 ago 1 reply      
Sounds like time for some politicians to get voted out of office. Voters may not care about certain things, but they definitely care about their transportation.
alanz1223 6 ago 0 replies      
this move seems like a desperate attempt to revitalize a dying dream... Taxis are dead, ride sharing killed it. End of story. Typewriter manufacturers werent given subsidies when the computer f*cked them over, were they?
nathan_f77 6 ago 0 replies      
I hope they're also taxing car manufacturers and giving it to horse owners.
ilostmykeys 6 ago 1 reply      
When Uber switches to a fleet of self-driving cars (with lower paid drivers, since they'll just sit and watch 98% of the time), this tax would be absolutely nothing short of taxing innovation to save an inefficient business model. Wow. I thought Capitalism was evil. Now I don't know what to think.
known 6 ago 0 replies      
"If it moves, tax it. If it keeps moving, regulate it. And if it stops moving, subsidize it." --Reagan
quickben 6 ago 6 replies      
Well, things should be equal. Why should a taxi service have to perform a regular vehicle inspection and a an uber driver be exempt from it?

Either make the Uber driver go through regular inspections or cancel the regulation for the cabbies.

ruffrey 6 ago 0 replies      
What prevents taxi drivers from taking Lyft and Uber shifts? Are there some kinds of restrictions on official taxi drivers? Do they need to keep a certain number of hours? Do taxi drivers have benefits?
maverick_iceman 6 ago 1 reply      
By this logic, Facebook should be taxed to give money to Myspace.
intopieces 6 ago 1 reply      
The weird thing is they don't even have a solid plan for what to do with the money. Or a plan at all, it seems.

The least they could do is create some kind of MassTrans App that shows a user options for navigating the state (uber/lyft/cabs/transit/greyhound etc). Otherwise it seems like collection for the sake of it.

xname2 6 ago 2 replies      
It should be totally opposite:

1) uber drivers are in lower income than taxis drivers

2) uber economy is new and innovative, while taxi is old and dying

tremendo 6 ago 0 replies      
Kodak and Polaroid should have thought of that. Get governments to tax digital photography, cameras and memory cards, in order to save their business and all those jobs they provided (as opposed those newly created, because pfft!). Never mind any benefit to the paying public.
nomercy400 6 ago 0 replies      
Isn't this unfair competition? Government meddeling with free market?

Also, does anybody know if taxi medallions expire?

jbmorgado 6 ago 0 replies      
I would understand if the state imposed this tax in order for this money to provide some social security to the drivers of Uber/Lift/etc. But taking this money away from them and giving them to the taxi lobby sounds almost like state imposed robbery.
losteverything 6 ago 0 replies      
Couldn't a "medallion cab" franchise that licenses tools (technology) to existing city cab drivers be created? So cabbies in Massachusetts can spend the tax money on their own uber via a franchise.

The franchise can be repeated all over for the "old industry" - In every big city.

So a user could choose their "Dominoes" app and they would be getting a legacy cab, not uber-lyft. And the customer would support existing men and women whose jobs are threatened. And like dominoes pizza, behind the scenes is a local owner (or local group of cabbies) that has to follow their own rules.

A cabbie could be hailed with 4 fingers and a thumb up or via two thumb touch. Have cake and eat it too.

chillingeffect 6 ago 1 reply      
Serious question here: how come taxi organizations haven't leveraged uber/lyft tech by now? Seems most kinks, tech and laws are figured out now. Why not upgrade using capital since the returns are obvious!?
douche 6 ago 0 replies      
Taxachusetts strikes again. Rather than let a shitty industry that already enjoyed monopoly protections, fail and be replaced with something more efficient, let's prop them up with a misguided subsidy/tax scheme.
yohoho22 6 ago 1 reply      
> The 5-cent fee will be collected through the end of 2021. Then the taxi subsidy will disappear and the 20 cents will be split by localities and the state for five years. The whole fee will go away at the end of 2026.
punnerud 6 ago 0 replies      
This is genial! This way we don't need the complains from "the old taxies" and can gradually reduce the tax when ride-hailing apps are established. This way we gradually remove the old ones :)
ovatsug25 5 ago 0 replies      
Interesting excerpt that makes me support the move:

 The 5-cent fee will be collected through the end of 2021. Then the taxi subsidy will disappear and the 20 cents will be split by localities and the state for five years. The whole fee will go away at the end of 2026.
They know they're fucked. The question remains: will they be able to use this temporary subsidy so they can become more "sovereign individuals" in the future?

halis 6 ago 0 replies      
They don't even pretend to care about every day consumers anymore.
finid 6 ago 0 replies      
The state wants to play Robin Hood.

Hey! Why don't they give the money to me. I can use it.

Next they will want to take money from the likes of Facebook and give it to struggling newspapers.

chmaynard 6 ago 0 replies      
> "They've been breaking the law," said Larry Meister of the Independent Taxi Operators Association.

If so, then the duty of government is to enforce the law (presumably with fines or prosecution). Imposing a new tax is another matter. It's amusing that a Republican governor supports a new tax of any kind.

astrostl 5 ago 0 replies      
Lots of jokes about horse-drawn carriages in here. Would note that, in order to legalize casinos in the STL area, they were required to give money to a dump of a horse track which would have otherwise sped (no pun intended) toward an even earlier grave.
40something 6 ago 0 replies      
Massachusetts is literally run by tyrants. No one cared when the attorney general trampled over the constitution by banning the sale of built to compliant "assault" weapons and now this. The disarmed populace has to just take it. The rulers know best.
sunshiney 6 ago 0 replies      
So Wix should be taxed with the proceeds given to developers it has has taken low-fruit from?
tiatia 6 ago 0 replies      
They should tax cars and give the money to horse carriages
chappi42 6 ago 1 reply      
As long as taxi drivers have higher salaries and are better protected this is a good move.

I very much salute the innovations uber enabled. Only there should be a law which guarantees a decent minimum salary for uber riders. No good to allow an income-race-to-the-bottom for drivers.

samuraibum 5 ago 0 replies      
This is why people should avoid paying taxes. Taxation is theft and your hard earned money will always be wasted anyway.
sevenless 6 ago 0 replies      
All hail the glorious US free market...
kevin_thibedeau 6 ago 0 replies      
If only the buggy whip industry had been so good at bending government to its will.
z3t4 6 ago 0 replies      
How do the app taxi buisness work? Are drivers employed by the app-maker or are drivers registered for VAT?
m1sta_ 6 ago 1 reply      
The tax collected should be forced to go directly to buyback of taxi licenses.
meira 6 ago 0 replies      
As a transition solution, this is genius!
kasajian 6 ago 0 replies      
This is so weird. I'm not sure it's legal.
madgar 6 ago 1 reply      
joshmn 6 ago 0 replies      
I'm all for taxing the rich to help the poor but not like this.
contemporary2u 6 ago 1 reply      
Food for thought:

I am working on AI robot that will ultimately replace human programmers/coders/software engineers. It will make human programmers obsolete.

What will most of you do? Will you go back to school to get a new degree/profession? Open a coffee shop perhaps? or apply for peace corps?

It is coming.. just remember that eventually anyone could be the next "taxi cab" driver with his or her job going away to automation.

Our society will have no choice but to adopt in more ways than one.

xyzzy4 6 ago 1 reply      
As a lifelong resident of Massachusetts, color me unsurprised. This is a tangent but I've also spent some time in India, and a $1.50 Uber ride there would be about $10 in Massachusetts. That has to do with other factors though such as PPP, birth rates and age demographics, immigration policies, number of skilled vs unskilled workers, etc.
mrerrormessage 6 ago 0 replies      
I haven't read all the comments in this thread, but I'm surprised at the outrage over a one-nickel tax. This seems like a fairly simple, straightforward rule. Is anyone going to decide not to Lyft/Uber over 5 cents? I don't think so.

Let's also not forget that these ride hailing services are MASSIVELY subsidizing the cost of rides in order to attract drivers/users. That hurts business for taxis. In my mind, this is a sort of protection that ensures taxis stay in business as another, publicly regulated option. What happens in other areas of Uber/Lyft kill all the taxis and then decide doing business isn't profitable and leave (or all their drivers leave I've subsidies end)? This might seem like a farfetched scenario, but remember that Uber is still sitting in a large cash reserves. What will change when they need to turn a profit every quarter? If they have established a monopoly (even locally), they can charge users as much as they want. If they've established a monopsony on drivers, they can lower wages. In my opinion, subsidizing a long-standing industry from a monopolistic competitor with gobs of money to throw at the market looks like a good move.

thesumofall 6 ago 1 reply      
I believe they have a point. The traditional taxi industry _can_ be rescued. Living in Germany I use myTaxi which is basically providing the same interface as Uber but connects you with professional taxis that are regulated, insured, and being paid proper wages. There is simply no difference in convenience (even the opposite: you can book taxis in advance) and the driving experience itself is typically by far better than any Uber I've ever taken in the US (inexperienced drivers, often old cars,...)

So I would argue that the tax could help achieve a similar scenario in MA if invested properly and if Uber is forced to raise prices by bringing them to uphold professional standards. I was long enough in MA to know that regular taxis have a long way to go but there is hope

Apollo Global is buying Rackspace for $4.3B businessinsider.com
324 points by notmyname  1 ago   159 comments top 27
chollida1 1 ago 5 replies      
I guess it was only a matter of time before they got bought out.It's tough to compete against one of Google, Amazon or Microsoft, competing against all 3 at the same time in an area that all three consider to be core to their future must just be cut throat!

It's never great for a companies employee's to be taken over by a private equity firm, if you actually find someone whose had a good experience then please let me know, but given that this is a 38% premium over what RAX was trading at when the deal leaked on august 3rd, this is almost best case for rack space employee's and given that this is an all cash transaction they should get some liquidity out of the deal!!

Given that the RAX board unanimously approved the deal, I'm going to guess this is going through.

Often when a company is brought private by a PE firm they'll combine it with other portfolio companies before spinning it back out. I don't see any relevant companies in Apollo's portfolio that could be joined to RAX.

If you are wondering who in wall street makes money on these types of deal, its the usual suspects. Everyone wets their beak in take over transactions:)

- Financing provided by Citi, Deutsche, Barclays, RBC;

- Goldman advised RAX, Morgan Stanley also provided services in connection w/ deal; Citi, Deutsche, Barclays, RBC advised Apollo

colinbartlett 1 ago 1 reply      
> $32 per-share-offer represents a premium of 6 percent to Rackspace's Thursday closing price.

Quite a fall from almost $80 in 2013.

I was a satisfied Rackspace customer back around 2001 when I had a web hosting business, but it was truly a premium service - very expensive compared to competitors. We ended up going with our own bare metal eventually. Then, when everything moved to the cloud, Rackspace seemed a little behind the times and Heroku and AWS got my business.

colinramsay 1 ago 4 replies      
Over the past six months I've been battling with poor service from Rackspace, with hosts mysteriously dying and their agents are trying to upsell me (load balancers for a single server, for example). We're migrating away but this doesn't surprise me.
20years 1 ago 1 reply      
I still have a couple of dedicated servers with Rackspace. Been with them for over 10 years and am sad to see this. They truly did have the best support and were an amazing partner to my business in the earlier years.

I knew this was coming and have moved a lot of our stuff off in preparation. Partly because of the unknown but also partly because their support has diminished over the past 2 years.

I was kind of hoping Amazon would acquire them. I don't have much faith with the purchase being a PE firm. Time now to move the rest of our stuff off.

rkrzr 1 ago 4 replies      
Rackspace has been looking for a buyer for a while. I suspect that their business is not in terribly good shape.

They even started consulting on AWS deployments a while back: "Need some help moving your servers over to AWS? We're here to help!"

jszymborski 23 ago 3 replies      
I'm praying that Mailgun doesn't get affected by this, they're absolutely awesome for the small shops like me (and easy to integrate).
akulbe 22 ago 1 reply      
I understand that publicly traded companies are one cornerstone of our economy.

That said, it's depressing to see companies get bought and sold just to move money around, and the people that work in those jobs completely ignored, or just seen as pawns to manipulate for nothing more than the bottom line.

To me, when a company goes from private to public, it's not something to celebrate in the long-term.

The company's focus inevitably seems to go from doing/creating something innovative, to maximizing shareholder value at any expense.

Rackspace was awesome. RIP Rackspace. (I don't know this for a fact, of course... but as others have surmised already, this will likely be just another pump and dump.)

gtrubetskoy 1 ago 1 reply      
Interesting. In contrast just about a year ago Verio's web hosting assets were sold to EIG for a mere $36 million. Both companies were founded around the same time ~ 1996, and at some point at the top of the dot-com boom were the two dominant dedicated server providers out there.
abakker 22 ago 1 reply      
Rackspace's problem is this: they are not really a hosting business. They are a Managed services business. They USED to be a hosting business, but it turned out that their real value add was in running clouds for companies that couldn't do it themselves. My guess is that the real reason they sought this is that the hosting business is not going to grow, and they don't want to invest in it. Instead, they are going to transition into becoming a managed services provider for Openstack private clouds (customer premises or equinix), Azure and AzureStack, and AWS.

Many enterprises are not making the transition to Cloud cleanly, and Rackspace is positioning themselves as the premier services provider to deploy, manage, and monitor cloud usage for many organizations.

laveur 1 ago 0 replies      
I've been a loyal rackspace customer since 2011. I really hope that if this is indeed true, as I will wait until Rackspace officially announces it, that Appollo doesn't destroy the good things Rackspace has going for it. Mainly their wonderful customer support.
kolbe 1 ago 1 reply      
Nice. So in a few years, there will be one less competitor in this space.
spriggan3 1 ago 0 replies      
I'm sure Rackspace is a great service for established companies and startups but for the rest they were always too expensive. I never felt like the premium paid made a difference.
inputcoffee 1 ago 3 replies      
I like to look at comparisons like this:

That is about 1 Yahoo in 2016

Or about 3 youTubes in 2006

Or 1.5 Lucasfilms in 2012

Or 0.2 Whatsapps in 2014

EDIT: Whatsapps number corrected, thanks.

thedlade 1 ago 3 replies      
Their service is very expensive compared to Google, AWS and the rest. I wonder how they've managed to survive on their own for so long
jdpedrie 1 ago 3 replies      
Has there been any analysis on the upswing in acquisitions of cloud computing and storage companies? First EMC gets bought by Dell, now Rackspace is getting picked up. Is it just in response to growth on the part of Google and Amazon?
nl 23 ago 0 replies      
So I think people are missing some points around this deal.

Investment firms like hosting companies for two reasons:

1) They give predictable revenue, which is a great thing. Even if the profit rate isn't amazing, the revenue gives a lot of cash-flow.

2) They (often) own large infrastructure asserts (data centers), which can be depreciated and used as a tax write-off.

Not saying that they won't want to take costs out of the business too, but the motivations for a purchase like this aren't as simple as one might think.

kondro 12 ago 0 replies      
This is the wrong time to be in the datacentre business. Especially one that traditionally provides a high-touch, traditional bare-metal based model.

It probably doesn't make a dent in their revenue, but Xero is just completing their migration from Rackspace to AWS for reasons they don't articulate well.

ksec 12 ago 1 reply      
So what other BIG Cloud Hosting Companies are there left?

Google, Microsoft, Amazon, IBM ( SoftLayer) OVH, AliYun.

keenerd 22 ago 0 replies      
As someone who's first VPS was from Slicehost: Oh, not this again.
clavalle 22 ago 1 reply      
I wonder if this will mean an influx of capital in the Austin/SA area.
mathattack 22 ago 0 replies      
Is the playbook "Financial engineering, and decrease support for existing customers"? (Financial engineering meaning load up on debt, where interest payments can be written down)
pbarnes_1 16 ago 0 replies      

SoftLayer has been growing substantially after the IBM acquisition. This space is pretty interesting.

soperj 16 ago 0 replies      
Wonder what this means for mailgun. They have wonderful customer support.
mbesto 1 ago 1 reply      
Interesting year so far for PE M&A in SaaS companies

Vista - Cvent $1.65B

Vista - Marketo $1.80B

Vista - Ping $600M

Apollo - Rackspace $4.3B (moreso IaaS)

Thoma Bravo - Qlik $3.0B (debatable SaaS)

ramaro 20 ago 0 replies      
Is it too late to sell it to Yahoo instead?
mkj 1 ago 1 reply      
Are Apollo likely to make money from it?
joering2 23 ago 1 reply      
I'm only missing one thing out of your comments -- what will change for someone who spends over $10,000 per month across US, UK and HK on servers and hosting.

Is it time to move forward?? Will my hosting be affected??

       cached 27 August 2016 15:11:02 GMT