hacker news with inline top comments    .. more ..    23 Aug 2016 Best
home   ask   best   3 years ago   
Nasa just made all its research available online for free independent.co.uk
761 points by signa11  2 ago   48 comments top 18
thearn4 2 ago 3 replies      
I'm actually curious what is new within this. As it stands, NASA research pre-prints are available on the tech reports server (http://www.sti.nasa.gov/), typically right after the export control review process.

edit: maybe it's related to certain journals where there wasn't access on the STI server? Seems odd to me. For the 8 years that I've been at NASA, it's always been expected that my group's work had to be accessible.

edit2: not all rules are universal across the agency, so my experience may be too specific to Glenn Research Center/my division. In any case, the more open, the better.

WalterBright 2 ago 2 replies      
> Nasa announced it is making all its publicly funded research available online for free

This is the way things ought to be for all publicly funded research, not just NASA. Thank you, NASA, for leading the way.

The beginning of technological progress for mankind started with writing, the beginning of the industrial revolution started with the printing press. Having all the world's knowledge available on your desktop just a click away is the beginning of another exponential leap forward.

codyb 2 ago 1 reply      
Link to the actual data [0].

Looks pretty neat actually. This seems to stem from an executive order by President Obama in 2013. Mobile browsing is okay but I'm excited to check out some of the APIs when I get back to my computer a bit later on. They're seperated by category (Earth Science, Aerospace, etc).

Seems like a lot of the data is already queryable by their api's and I assume there are data dumps and research papers available as well.

Very cool. There is a serious wealth of data and apis available for tinkerers and builders these days from watson to NYC data to Nasa!

[0] - http://www.nasa.gov/open/researchaccess/pubspace

dmix 2 ago 0 replies      
The mars tsunami link is a fun read:http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4872529/

> We conclude that, on early Mars, tsunamis played a major role in generating and resurfacing coastal terrains.

That's a fascinating thing to to think about. Reminds me of the the ocean planet they landed on in Interstellar. Although Mars had giant tsunamis caused by impacts instead of perpetual waves caused by the gravity of a black hole.

iamleppert 2 ago 3 replies      
Great that they're finally getting around to doing this 3 years after the order was signed.

Shouldn't it be a law that any research done with public dollars should be made available to the public for free? Any costs associated with the publishing should be built into the funding.

owenversteeg 2 ago 1 reply      
Ok, so maybe I'm missing something, but what's new here? Is it just the portal?

I've been reading NASA technical papers and publications for years, and although most of the research I was reading was very focused in a specific field, everything I wanted to see was freely available. Even some of the publications linked in the article have been online for a decent amount of time.

joeyrideout 2 ago 0 replies      
Searching the web portal leads to a PMC database query with the filter "nasa funded". Here's a link to a query with just that filter, which returns all 863 articles:


paul_f 2 ago 0 replies      
This article is so misleading. NASA is requiring that the pubished research papers be public. The research discoveries are still owned by the Universities that did the research - the Bayh Dole Act is still in effect. https://en.wikipedia.org/wiki/Bayh%E2%80%93Dole_Act
nxzero 2 ago 0 replies      
Saying that "all" of NASA's research is being made available is a stretch given it's known they work on a lot of classified projects with the military, intelligence, etc.:


todd8 1 ago 4 replies      
Not NASA, but related: I really want the raw climate data and atmospheric models to be made open and publicly available. I don't understand how researchers can claim that climate change is the most serious problem facing our species while at the same time hiding what they are doing. I know that some data is available, but considering that our government funds most of the research why haven't they put this stuff up on github?

Is the secrecy really necessary in order to get tenure and win grants?

thefifthsetpin 2 ago 0 replies      
Title should read "some" rather than "all"
girishso 2 ago 0 replies      
At last... I remember Richard Feynman complaining exactly about this!
halis 2 ago 0 replies      
Great! I glanced over it and I think the next step is to have them provide a For Dumbasses version of pretty much all of it...
eecks 2 ago 0 replies      
Has anyone looked at the resource? Are there data sets and other similar things that could be used in novel ways?
Azuolas 2 ago 0 replies      
following Space X as a top competitor
2 days ago 2 ago 1 reply      
jpeg_hero 2 ago 1 reply      
monksy 2 ago 1 reply      
Including about set design of the moon for the landing?

(For those who just threw a fit.. it's a joke)

Japanese writing system basics candyjapan.com
944 points by bemmu  3 ago   334 comments top 47
viraptor 3 ago 4 replies      
This was an interesting symbol to choose for an internet explanation. It took me a while to realise that the rectangle is actually the symbol that I'm supposed to see, rather than a missing glyph.
imron 3 ago 2 replies      
In Chinese is pronounced sh.

As someone who speaks Chinese, I got a chuckle out of reading 'put your favorite snack in your and t it!' due to the association in my mind of that character and its Chinese pronunciation, immediately followed by a 't'.

primitivesuave 3 ago 4 replies      
Another interesting aspect of traditional Chinese characters is that complex words are expressed by combining simpler symbols. For example, the Chinese word for computer is . The first character represents "electricity", and the second character represents "brain". Which is really what a computer is, an electric brain. Similarly, a computer programmer is , where the three symbols are "rule", "order", and "person" - one who orders rules.

An interesting consequence of this is that you only need to learn around 3000 symbols to read a Chinese newspaper, just like how you can ascertain the meaning of an unfamiliar English word by having knowledge of a small set of Latin/Greek roots.

anqurvanillapy 3 ago 4 replies      
In Chinese, 'eat' is usually ''. We use ' (le)' to say we 'ate' and ' (zi) ' or ' (zh)' for 'eating'.

It is really interesting that in China people often ask their friends '? (Have you eaten?)' rather than ' (Hi)' in daily life. So initially I thought this post was describing something in Chinese w/o my noticing the URL.

euske 3 ago 6 replies      
Yeah yeah, this all makes sense until you see , which isn't "a three-mouthed monster" but "goods".

Languages are weird, man.

andreygrehov 3 ago 5 replies      
I liked the beginning and expected the next word after "mouth" to look almost the same, with a subtle change, which would be a logical extension. But it was quite a jump from a simple square () to god-knows-what ().

Also, why is translated as "Eclipse" in Google?

Sniffnoy 3 ago 2 replies      
Not sure how great an explanation that really is. I like Zompist's explanation: http://www.zompist.com/yingzi/yingzi.htm
zatkin 3 ago 1 reply      
Aren't we sort of starting to doing this with the introduction of emojis? They're a little bit ambiguous, but they do have meaning behind them, nonetheless.
rett12 3 ago 4 replies      
Sometimes I wonder if all these people that criticize or that think that a Latin alphabet can be adapted seamlessly to all languages have tried to study past a beginner level anylogographic language.
dhfromkorea 3 ago 2 replies      
Another interesting side-effect that the compactness of Chinese symbols(other variations: Hanja in Korean, Kanji in Japanese and so forth)allowed was a higher chance of survival against natural disasters like wild fires or crimes like thefts or vandalism.

It was/is far easier to ensure redundancy of scripts and books since the costs of reprinting/copying was far lower compared to other forms of phonetic systems.

The compactness explains how so many archaic, buddhist scripts could survive to this day.

Grue3 3 ago 2 replies      
means "enter", means "mouth". means... "entrance". Actually for most kanji there is no single meaning. Some meanings might even have nothing in common with each other, because they've been based on ancient Chinese wordplay or something.
daveheq 3 ago 1 reply      
"Since we already have symbols for all the sounds we can pronounce"... Not with 26 letters we don't. Other languages have other sounds that English can only try to emulate, and even English has sounds that require multiple letters.
thaumasiotes 3 ago 4 replies      
If substitutes for mouth -- the adjective form of "mouth" is "oral", an etymologically (and audibly!) distinct word. Should that use too?

If king is , kingly is ly, and royal is al, what is regal?

If mouth is and mouthed is ed, why would ate be t rather than ed?

Japan misinterpreted the Chinese writing system (already terrible) into easily the worst writing system known to mankind. It won't look cute when you go beyond two symbols.

partycoder 3 ago 1 reply      
The only problem is that you learn all 2000 at the very minimum and more than that if you actually want to do something practical.

Each one has more than 1 reading, a particular stroke order, and many other things.

dghughes 3 ago 0 replies      
I just discovered NativLang on YouTube so this really is in my zone of interest today.

I've just spent the last few hours learning all about languages how they developed and each culture's spin on adding as much meaning as efficiently as possible to written symbols. I've always loved languages so this was more of a brush up plus learning.

It seems and rightly so ambiguity is death to any characters and efficiency is also fundamental to the character.

I'm not Korean but I like their style literally I like how their language style is so efficient in context to mouth position. It was created because Chinese characters didn't suit Korean language. Japan also streamlined Chinese characters to better suit their culture.

Mayan is another wild language full of meaning in such compact symbols. I had a hard time following their characters.

falcolas 3 ago 1 reply      
Everything old is new again.

I have no , but I must ...

Edit: Nevermind, HN swallowed the Emojis.

paradite 3 ago 0 replies      
This has the added advantaged of being recognized by both native Japaneses and Chinese speakers instantly, as long as we keep it to kanji.

I recognized what the author is doing from the start as a Chinese speaker.

kevindeasis 3 ago 6 replies      
How many symbols are there? And how would the keyboard look like though?
rezashirazian 3 ago 2 replies      
That was cool. Interesting enough, you can come to the same conclusion with emojis.
andrezsanchez 2 ago 0 replies      
I think it would be interesting if there were a Latin equivalent of Chinese characters. Different roots could be represented as different characters, some could be used for each of the suffixes like "ly", "tion", etc., and the characters would be joined together to create words like in Chinese.

Different Romantic languages could be represented this way. In the same way that Mandarin and Cantonese use the similar character sets with different pronunciations, and with some characters specific to each one, different languages that have Latin roots would have a few of their own characters specific to their own language, but mostly drawing from the Latin pool.

The pronunciation for each would have to be memorized of course.

frostymarvelous 3 ago 3 replies      
After viewing this a couple of hours ago, ads for Candy Japan are popping up for me on Facebook.
Joof 3 ago 2 replies      
Japanese is an unusual language to write. It's influenced by Chinese (in two separate eras), English and perhaps many other systems. Symbols alone aren't a perfect fit for the language (since they add tense and such), but neither is an English style alphabet.
nxzero 3 ago 1 reply      
>> "Since we already have symbols for all the sounds we can pronounce [in English]"

English has 26 letters, but there are 40+ sounds; which is to say that there are NOT symbols for all the sounds.

codedokode 3 ago 0 replies      
The problem is that many characters look similar but have nothing in common. For example, the article mentions the character meaning "food". So when after reading the article you see similar character like you might think it is somehow related to food. Well, it is not, it just means "good". And another similar character meaning "long" or "leader" also has no relation to previous two.

And when you get to more complicated characters like it becomes even more confusing.

sideproject 3 ago 1 reply      
Really like the live streaming of Google Analytics at the bottom. :) Now I have an idea how much traffic you receive when you reach the #1 spot on HN on the weekend.
danielrhodes 3 ago 2 replies      
I'm interested in how this compactness changes the expressiveness and evolution of the language. In English, as in other languages written in the latin alphabet, you can make changes to the spelling of a word and create new meanings rather fluidly and it's usually easy for the reader to comprehend. I have no knowledge of Japanese, but are there similar possibilities? How do new words/slang get created?
drwicked 3 ago 2 replies      
I'm kindof fascinated by the idea of livestreaming the pageview statistics. I had a surreal moment realizing the weird recursion. Is this something people do now?
giancarlostoro 3 ago 2 replies      
Ancient Pictographic (before Paleo) Hebrew is like this too. Except they only have 22 letters, there's a 23rd but it is no longer part of the 'Aleph Bet'. Hebrew originally had no vowels, and thus today they only put the vowel system in when it's a word people don't normally know how to pronounce. Chinese is similar as well, and I'm sure there's a couple more languages.
fogleman 3 ago 2 replies      
That was fun. Will there be a second lesson?
ryao 3 ago 1 reply      
and are also Chinese characters for mouth and eat, although I had to check google for . Initially, I thought that he was teaching Chinese until I read was eat (which needed Google) and that it was "Japanese".
raverbashing 3 ago 0 replies      
wch4999 3 ago 0 replies      
Well, reading the first part I just realized this is japanese/chinese! In Chinese all characters are like or . Sometimes we can also break the character down to several parts to understand its meaning.
sova 3 ago 0 replies      
As someone who has spent years of their life dedicated to mastering Japanese, I must say, you sir are a genius.

By the way, I'm happy your site is rockin! I saw it when you launched candyjapan and I am happy for you =)

dingo_bat 3 ago 2 replies      
Slightly off-topic but can anyone here comment on the candies they ship? Are they just ordinary sweets? I've never heard about Japan being famous about candies like Switzerland is about chocolate.
alecsmart1 3 ago 1 reply      
This is off topic, but I can't find any pricing on the candyjapan.com website when visiting on mobile (iPhone). Anyone can tell me how much it costs per month?
haddr 3 ago 0 replies      
This article is cool, I love the way is actually encourages to learn something by starting with something simple that you can grasp in seconds.

I was like ()!!!

bootload 3 ago 1 reply      
@bemmu is that a square symbol or something else?
billmalarky 3 ago 0 replies      
Love the live video analytics! Such a simple yet effective hack.
seanmcdirmid 3 ago 3 replies      
BlakePetersen 3 ago 0 replies      
That path tho, /%E5%8F%A3 rendering as /, so slick
smnplk 3 ago 0 replies      
I'm struggling to find a comment that is actually related to the story of CandyJapan and not that character.
fiatjaf 3 ago 1 reply      
Can I believe this? Does Japanese really mix characters that mean things with characters that mean sounds?
ChuckMcM 3 ago 1 reply      
Ok, thats a fun hack.
itaysk 3 ago 1 reply      
why is this post getting so many voteups?
weinzierl 3 ago 2 replies      
Flagged for moderator attention: The link of the article just goes back to this page on HN. Is this an error or did I miss the joke?

Other articles are fine. Reload, clear cache and reload didn't help.

EDIT: Unflagged and sorry for the noise. I clicked on the article's domain and not on the article's title () just as fenomas suspected.

monomaniar 3 ago 0 replies      
Nobody said these are all Chinese letter? Japanese is "invented" and forced educated by Meiji government one hundred years ago. https://en.wikipedia.org/wiki/Meiji_Restoration
disruptalot 3 ago 0 replies      
Finding it surprising no one has corrected the article in that you technically learned Chinese and by proxy Japanese. Chinese traditional characters + meanings largely carried over from Chinese as well.I'm studying mandarin and i was enjoying it until I was told I just learned "Japanese".
Less stress, more productivity: working fewer hours is good for you and your boss codewithoutrules.com
581 points by itamarst  4 ago   291 comments top 42
pc86 4 ago 16 replies      
> Long hours: "It's 5 o'clock and I should be done with work, but I just need to finish this problem, just one more try," you tell yourself. But being tired it actually takes you another three hours to solve. The next day you go to work tired and unfocused.

> Shorter hours: "It's 5 o'clock and I wish I had this fixed, but I guess I'll try tomorrow morning." The next morning, refreshed, you solve the problem in 10 minutes.

I have too much experience with this. I remember when I was self-employed I humble-brag tweeted about my production deployment ~15 minutes after starting work. A friend of mine asked why I hadn't just deployed last night and the quote above was my first thought (in addition to the fact that you should never deploy to prod then go to bed). I was done with work, so I stopped. It avoids hitting that "grind it out" phase where you're lucky just not to do more harm than good.

bcrescimanno 3 ago 7 replies      
Just this week, I've been collecting feedback from my team. Some of the most powerful I received was, "Since you took over, I feel like I've gotten more done with so much less stress and extra hours. Until you came in, I didn't believe that we could really have work-life-balance and be MORE productive and now I know we can."

Having been in companies where my approach has been called everything from "lazy" to outright "damaging," this feedback was both rewarding and validating.

Edit: For those who are in jobs where you're putting in those long hours, I can tell you there are managers out there who believe in the message here. I have three of them working for me right now and I constantly reinforce the philosophy of working sustainably. Don't settle for less than a manager who respects you the person--not just you the engineer!

jhummel 3 ago 6 replies      
I started a consulting/services company a few years ago, and this year was finally in a position to start hiring. I'm a big proponent of this type of thinking, so while I couldn't offer the best salary, or the best benefits - I could offer a better work/life balance. I knew it was a win-win for us and the new employee.

We landed on a 36 hour work week. 8-5 Mon-Thurs, then 8-12 on Friday. We had more applicants than I ever dreamed of, and scored a great hire that was coming from the 70-hr-a-week startup life. Everyone's been extremely happy over the past six months, and I don't feel like I'm missing out on any productivity what-so-ever.

gthtjtkt 3 ago 6 replies      
> One programmer I know made clear when she started a job at a startup that she worked 40-45 hours a week and that's it. Everyone else worked much longer hours, but that was her personal limit. Personally I have negotiated a 35-hour work week.

Wow, what a depressing time to be employed. The old standard (35 hours) is now considered a "short" workweek, and only the most desirable employees have the leverage to request it. Not to mention stagnant wages, rapidly rising costs of living, and off-hour availability expectations.

How did we end up here?

imh 3 ago 7 replies      
>There are companies where this won't fly, of course, where management is so bad or norms are so out of whack that even a 40-hour work week by a productive team member won't be acceptable. In those cases you need to look for a new job, and as part of the interview figure out the work culture and project management practices of prospective employers. Do people work short hours or long hours? Is everything always on fire or do projects get delivered on time?

I have never figured out how to ask this. It always feels like I'm asking, "I don't like to work much, is that ok with you?"

dasil003 3 ago 2 replies      
Having spent a lot of time both as a manager and as an individual contributor, here's the rub (at least for programmers): software developers are a classic market for lemons. The bigger the org and the more operational complexity the more space there is for poor developers to hide out and draw a pay check. But even at a small scale it's remarkably difficult to measure productivity actually.

Therefore, what you need is intrinsic motivation. You need engineers who really care about solving the right problem at the right time in the right way. 35 hours or 70 doesn't matter if they can't get that right, and it's very hard to have management that is qualified to make that judgement on people (especially as an org grows and management becomes a full time job). So the problem is that given this opacity, finding someone who works 70 hours a week is a better proxy than someone who seems excited about a 35-hour work weekthe former are people who are clearly driven whereas the latter are literally everyone. It's far from a good metric, but perhaps the best one available to the pointy hairs of the world.

Speaking from personal experience, it wasn't until I had a kid that I found out what I could do in a 40-hour week. Once I had that time constraint, it forced me to be more efficient in a very deep and fundamental way from the core of my being. When I was 25 I would hit 5pm and think to myself: I still have another 8 hours to solve this problem before bed. Perhaps this is post-hoc justification, but I think that makes sense when you are just starting out and not really competent yet. Once you've passed the magic 10k hours or whatever, I think turning over problems in ones subconscious can provide a lot more of the value. So sleep / exercise / meditation can all help elevate your performance far more than extra hoursbut only if your brain knows what it's doing.

tonyjstark 3 ago 3 replies      
Of course every personal experience is just anecdotal but since I only work 24 hours a week I feel able to educate myself, do sports, feel more healthy. Before that I worked one week full and one week only 4 days repeatedly and I really got the feeling that coming in on Mondays after a short week was so much easier. I felt more relaxed and satisfied because I could do other stuff on the weekend than trying to catch up with buying things and all that organizational junk you have to do as an adult.

Now I work 6 hours a day, 4 days a week and I don't take long breaks anymore. I stopped to read blogs or news at work after lunch, I just work. Would be really interesting if this is just me or if it could be scientifically proven that working less is more.

codingdave 3 ago 6 replies      
Even better, drop hours as a metric of work. Offer the flexibility to work whatever hours are most productive... and find a boss who doesn't care whether that is 40 or 20, as long as he gets the value he expects from your time.

After all, employment is a business contract. You are given money in exchange for the value you add to the organization. Not hours... value.

rabino 4 ago 2 replies      
I was expecting the list of sources at the bottom of the article to be a mile long. Not a single one. Hot damn, so much broscience and red herrings.
shams93 3 ago 0 replies      
Its the Theater of Productivity, similar to the TSA. You can see Meyer's approach to productivity was ultimately just as successful as the TSA on secutity. Yeah doing an early stage startup it could be on the side of a day job so yes you are going to inevtiably put in a lot of hours for $0 doing a startup. But for regular businesses that are well established such as Yahoo this isn't productivity its Productivity Theater in the same way the TSA are Security Theater.
anupshinde 3 ago 1 reply      
I think the key is less-stress and not shorter hours. Meetings, especially long ones, induce stress.

I've tried reducing hours (as a freelance remote developer). I cut my working hours to 25 per week from somewhere around 60ish per week. (And I could afford losing out money)

At first, I felt I was missing something. I thought it was the money. After being completely off work for a month, I realized I was missing the people (Slack chats, meetings, fire-fights)

When I started back with another client, working just 25 hours per week was much much much harder. None of these worked: 8x3days, 4x6days, 6x4days, 5x5days. Few weeks I could not even complete 15 hours, and other weeks I was over-working. I was still stressed. Finally, what worked was odd - (10-12 hours)x2.5 days. So I ended up working a bit more hours than I wanted to. It was proven again that it takes time and focus to pickup momentum and costs a lot more to loose it frequently. And I still work on my other stuff totalling to about 45-50 hour work-weeks and still feels much less stressful.

jupiter90000 3 ago 1 reply      
There are certain jobs where it may not work, but from what I've seen places I've worked, 40hr workweeks seem like a holdover from some, in my opinion, truly bizarre cultural expectations. I mean, at some offices I've worked people are basically shooting the breeze for at least 10 hours at the office a week with coworkers. Some of them don't like being at home with their families and so seem to think work is a way to get away from their responsibilities outside of work. I'd rather have that 'time bullshitting with coworkers' to do things I like to do not at work.

Is this some holdover from previous generations of just wanting to pressure everyone to do what they had to do? Sure, sometimes things need to get done, but alot of jobs aren't saving lives or doing much amazing, but "hit that deadline or we're going to die" is almost the implication made in business in the US in my experience...

Hopefully we'll stop this work addiction some day and realize lots of things can get done without making people be behind a desk an arbitrary number of hours per day...

dsugarman 3 ago 0 replies      
Henry Ford originally found that 40 hour work weeks were optimal for productivity for his factory employees. Before this, they only had one day of rest and were generally overworked and inefficient. I think the work of a factory worker is vastly different than the work we do today behind a computer, especially programmers where your brain power is everything. It would be interesting if a big enough company were to do some proper research on the most efficient working patterns of a programmer today.
Apreche 4 ago 3 replies      
Recruiters. Find me a company that believes this, and I'll actually reply to you.
fizixer 3 ago 0 replies      
In my experience, in order to maximize the productivity in creative work and still work 40 hours, you have to pick the best 40 hours out of a 168 hour time frame:

- If it means you work '13 hours 20 minutes' on Monday Wednesday Friday, so be it.

- If it means you work '5 hours 45 minutes' Monday thru Sunday, so be it.

- If it means you work one way one week, the other way the next week, so be it.

- If you work best in the middle of the night, so be it.

- If it means you work 10 hours everyday for 52 days straight, and take the rest of the quarter off (39 days!), so be it.

The problem is, we have to do this thing called 'work/life balance' in which we are at work 9 to 5, whether we feel fresh or tired, and have to forget about work for the remaining 128 hours.

Combine that with the need for in-person interaction with the team, the boss, and what not, it gets worse.

IMO you can't have it both ways.

zxcvvcxz 3 ago 3 replies      
Yeah.... not so much for me. I'm going to give an alternate point of view.

To become world class at something, you need to be obsessed. You need to put in as many hours as possible and just love what you're doing.

"Oh but you won't be as effective after 8 hours of work!"

So what. Let's say I'm only 20% as effective after 8 hours. I'll take it -- 20% of the last 5-6 hours of the day towards my craft over 0% doing 'normal' things (drinks on the patio? Pointless travelling? Whatever my fellow annoying millenials like to do).

This is the kind of mindset you need to reach the top of a field. And it should develop naturally, you should really want it. Whether it means outdoing everyone at your company, getting that prototype done two months earlier, closing more leads, getting that tricky piano passage, whatever.

If you love what you're doing, more is more. Because you probably can't help yourself. If you feel like working more, just do it. Don't let normal social expectations hold you back, especially if you're young, because you only get so long to become great at something.

I'll finish with one final caveat. Figure out what level of sleep, exercise, and nutrition your body needs to sustain your desired work habits. Get those right ASAP, keep trying modifications, and realize it's different for everyone's unique biology. I like 8 hours of sleep a night, weightlifting 3x/week at 45-60mins each, and a certain amount of protein and certain vitamins (and coffee of course, ha). You'd be surprised how these 3 lifestyle factors can make such a huge difference in your energy levels and hormones -- one can literally become a different person!

sakian 3 ago 4 replies      
How does a company like SpaceX fit into this? They seem to promote the opposite and are seeing some pretty amazing success. Do they succeed because of or despite working long hours?
yodsanklai 3 ago 2 replies      
The most (professionally) successful persons I know worked very hard at least at some point in their life. For instance, I know tons of professors that pretty much work all the time. Same thing with former schoolmates working in finance now.

I'm not saying it's the road to follow, and obviously, not all careers are comparable, but I think that in order to reach their full potential, people have to (and can) work a lot.

SonicSoul 3 ago 2 replies      
There is no science here what so-ever. Just a bunch of feel good rah rah BS about working less hours and getting more done. Some fodder to go to your manager with I guess.

Who said that 35-40 hours is optimal? How can you even quantify it considering every job is different. I've had jobs where 50% was brainless support so i'd work 10-12 hr days and my brain was totally fine to keep going. Some jobs come with inevitable social BS that will take 50% of your time anyway.

The problem is that management and developers have different incentives. Managers [in more cases then not] are pushed by executives to justify their worth and deliver results.

Developers want to create, but also don't care as much about the bottom line. They want to have a life outside of work.

It's that simple. Let's not go pretending like anything beyond 40 hours takes 10x time to get done because it simply doesn't. It's just an empty thing to say like "you should give me a raise because of inflation".

There are plenty of people working their own startups doing 80-100 hr weeks and getting shit ton done. Does it mean that you should do this at a corporation? Maybe not, maybe yes. It all depends on the trade offs either side is willing to make. Managers will of course be incentivized to push for more productivity per employee.

Places such as Basecamp take a stance against this incentive gap. They're actively working at making employees take more vacation, sleep better, and have a better work life balance. This must improve employee morale and retention. No idea what it does to bottom line or employee paychecks.

As a manger I've had people that worked 40- hrs and delivered amazing results. I'd never even think about caring how many hours they've put in. Then there were others that worked more but delivered a mound of technical debt or nothing at all. They were usually the ones most vocal about working too many hours. There are very few people out there that can come in, get straight to work, and deliver quality. Simply making a blank statement that they will do more by working less is silly.

ivanhoe 3 ago 0 replies      
I've noticed this when playing chess, which is I guess a pretty good way to measure one's analytical capabilities and concentration. When playing against a computer in the evening after 8+ hours of work my games are significantly weaker; I can't anymore beat the same game level that I easily win in the morning. By looking into game stats I've noticed that in the evenings I make significantly more mistakes, don't see good moves, make hasty decisions more often, etc.
nathan_f77 3 ago 1 reply      
Right now I'm working 20 hours per week as a contractor. It's beautiful.

My hours are also pretty flexible. If I'm really in the zone, I can work a 10 hour day. That's what I did today. I started at 3pm, had a break around 7pm, and I just finished what I was working on at 2am (30 minutes ago).

Yesterday, I wasn't feeling motivated at all, so I just ended up taking the day off and doing something else.

I'm living in Thailand, so normally I'll work from 10pm until 2am, so I have overlap with clients in the US. I wake up at 10am, and it feels like I have the whole day to do whatever I like.

I'm earning less money, but I'm somehow saving much more than I did in San Francisco.

atom-morgan 3 ago 0 replies      
Imagine trying to convince a slave master that freedom, in the long run, will lead to more productivity on a scale that they can't even imagine. I believe working fewer hours is a modern day version of that.
dumbfounder 4 ago 2 replies      
Sounds nice, but maybe some evidence would be good?
namenotrequired 3 ago 1 reply      
I thought this was about working fewer hours than full time, but apparently it's about working no more than full time.
ErikAugust 3 ago 0 replies      
Tired is the worst time to code. There are other things you can do to be productive - but try not writing mission critical code when you are exhausted.
afarrell 3 ago 1 reply      
This also applies if you are in university. Your ability to learn is directly related to the quality of sleep you have gotten. If your school believes "sleep is for the weak" and makes labs due at 6am, please realise that this is bullshit and find tools/habits to enforce a different discipline.
Abdizriel 3 ago 0 replies      
I agree with that. Last ~1-2hours in job I feel bored and everything is not so productive than it should.

I would like to work in/create place where engineers work for ~30h a week including lunch/coffee time.

I think with that kind of work everyone would be happy and could easly balance work/life

tzakrajs 3 ago 0 replies      
The key to this article at a macroscopic level is the willingness to quit when the manager or culture are unwilling to adapt to your fewer hours proposition. We can positively change the industry for everyone by uniformly following this guidanace.
aledalgrande 3 ago 0 replies      
I find that, for boring tasks or everyday work, this is true. I cannot use "turbo mode" for too long.

But if there is something really challenging in which I plunged in, and at the same time I am getting results, it will break the rule. I think much of what written here makes sense: https://en.wikipedia.org/wiki/Flow_(psychology)#Conditions_f...

Also, if you really want to be productive, it's not about sleep, but about having very defined and organized tasks, so you limit the number of open decisions you have to take in a day.

Noseshine 3 ago 0 replies      

Does anybody have time to comment on the article submitted by OP? :)

mindfulgeek 3 ago 0 replies      
The best "full time" work environment I experienced was a 9-5 week with morning/afternoon 3-hour coding blocks (no interruptions tolerated) and an hour lunch. The other hour was spent in the standup, email, one-off conversations and blogging/community work. We were a highly production and creative team.
ShakataGaNai 3 ago 0 replies      
This is one of the reasons why more telecommuting/remote work is "better". Since you've got no boss looking (literally) over your shoulder, you can't be judged on hours. Then things like output or project completion actually bubble to the top as most important.
us7892 3 ago 0 replies      
Bottom of the article has a link...to learn from your mistakes (perhaps because you worked too many long hours.) We can get emailed a list of his mistakes. https://softwareclown.com/
jsprogrammer 3 ago 0 replies      
Good article, but I don't see where it addresses how working fewer hours is good for your boss (though, you can try to read in to it). The article even says that it (working less hours) won't work at some companies.
rdlecler1 2 ago 0 replies      
For much of the article the author is saying that with less time to complete a project you now use your time more wisely and you'll get the same amount of work done in a shorter window. If that was the case then managers should instead add 20% more to every project. Now you'd be delivering 20% more with the same resources as before. Not buying it.
Waterluvian 3 ago 0 replies      
I've been fortunate enough that my employer encourages people to find the patterns that work for them. Now that I've been there a few years, I've been taking greater liberties on when I come and go. Some days I just wander home at 3pm because it's just not happening. Other days I'm there 8-8 and skip lunch because I'm so excited to see a new feature work. I've never been so productive or happy.
franciscop 3 ago 0 replies      
I thought this was about reducing it from 40h to 30h or so, but it was about fighting for the 40h themselves. I am in Japan now and things are really crazy here for my friends.
rdlecler1 2 ago 0 replies      
For much of the article the author is saying that with less time to complete a project you now use your time more wisely and you'll get the same amount of work done in a shorter window. If that was the case then managers should instead add 20% more to every project.
zooko4 3 ago 0 replies      
Hey, I know the guy who wrote this! As a boss, and in fact as the former boss of itamarst, I agree with this strategy. I want employees to produce high-quality work, to be predictable in their pace, to not get burnt out, and to be happy and pleasant to work with.
stevenwiles 3 ago 2 replies      
known 3 ago 0 replies      
Not for Non-IT
mastratton3 3 ago 2 replies      
I agree with the fact about cutting out meetings but I don't know if I agree with everything here.

While its easy to say "Just leave at 5pm and come back refreshed" is easy to say, I found personally to have learned to most on the days when I've worked until 2am really digging into a problem and getting it fixed. I also have found that the best people I've worked with have a similar get it done attitude. Obviously this isn't sustainable but I think required from time to time.

I also think this lacks the notion of "how much do you want to move up vs stay in your current role". If you don't have any desire for more responsibilities than this applies, but I think if you want to move up/learn new skills, reducing the hours works against you.

I guess I follow the mantra of, "The first 40 should be productive and for the employer in the role I was hired for but I should spend additional time on top of that learning new skills"

Browsing your website does not mean I want your spam medium.com
571 points by davezatch  22 ago   225 comments top 52
trjordan 20 ago 11 replies      
> This transaction breaks a core promise using the internet: just because I visit a website doesnt mean I consent to getting spam from it.

No it doesn't. There is no core privacy premise of the internet, and certainly not one that everybody used it signed up for.

I'm not condoning this behavior, but we're in territory that we don't have prior art for. It used to be totally fine for one shopkeeper to mention to another that he saw a customer looking for a particular item. When you do it at scale, the old rules don't apply.

If you think it's spam, hit the spam button in gmail and get rid of it. Use an adblocker. Talk to your congressman about data privacy and sharing laws, because we don't have anything that's effective. Frankly, continue to write Medium posts, because it raises awareness :) But, I disagree with the notion that this is a solved problem with bad actors, because we're in unknown waters.

davb 18 ago 6 replies      
I once had something similar, if not worse, happen.

I was researching some network equipment, looking at lots of websites and comparing products.

Then my desk phone rings. A call being passed from the switchboard - someone asking for the person responsible for IT purchasing.

It was a sales rep from a network equipment distributor, saying they noticed I was browsing their website and wanted to help me through the purchasing process.

I had never used their website in the past. No-one from my company had. I never signed up. I didn't login. I was bewildered.

I asked how they got my details. The rep said they pay a third party remarketing agency for contact details of people who visit their website.

We were a really small company, with no DNS PTR on our main (NAT'd) public IP. We did have an A-record for our mail domain pointing to this IP.

As the sales rep didn't know my name, all I can assume is that their remarketing agency was looking up our public IP addresses in some IP-to-business database, populated by email headers or sign ups at other user sites.

In any case, I wasn't pleased and was pretty surprised at the rather aggressive sales technique.

JohnTHaller 17 ago 3 replies      
I've been getting more spam lately from "legitimate" companies. One of my email addresses leaked from a major open source project I corresponded with. Harvesters found it and now sell it to every small business and entrepreneur marketer you can think of. I get spam from CDNs, off-shoring companies, SEO/SEM, marketing, you name it.

Lots of them use sketchy services like reply.io to make it seem like a real person sent the email. And then another that looks like a reply to the first when you don't respond. And then another and another. Like Katie Malone at HawkSEM.com who 'personally' spammed me another 'reply' today. Essentially, folks like reply.io and similar automate the process of repeat spamming. Even their tag line is "Send Cold Emails That Feel Warm".

Here's a reality check for you: sending "cold emails" to a list of email addresses you bought makes you a spammer. Even if you try to make them appear personal. The giveaway is the tracking image (usually hidden or 1px by 1px white of course) and tracking links in every email so they can track whether you opened it and whether you clicked anything along with the unsubscribe link at the bottom. Except they don't label it as unsubscribe. It says "If you don't want to get any more emails from me, just let me know." with 'just let me know' as a link.

Be sure to mark every email like this you receive as spam so you don't get any more and so their reputation decreases enough to route all of this spam to everyone's spam folders.

cyberferret 15 ago 3 replies      
I am really beginning to hate browsing the web these days... Especially poop up dialogs asking for my email as soon as the mouse cursor leaves the active browser screen. With an average of 20 browser tabs open, while one is loading I often go to click on another to check on something, and this instantly triggers a flurry of popups begging me to stay/subscribe.

Also the retargeted ads that follow me everywhere now. MOST of them are for companies where I have ALREADY bought something, so they are wasting their ad spend on chasing an existing customer, not a likely prospect.

This has made me resolve to try and make the web a less shitty place, one web site at a time - and I have ensured that my web projects absolutely DO NOT have any popups or cross site tracking in there (aside from normal analytics that is only used in house).

[I accidentally mis-typed 'pop up' above but LOVE the Freudian slip so will leave it as-is].

r1ch 20 ago 3 replies      
I've had several companies ("data partners" they call themselves) approach us to add these scripts to our websites. All of the ones I've seen use MD5(email) for the "anonymous hashing". I mentioned our privacy policy doesn't allow us to give out user emails, and their marketing guys never seem to understand that MD5(email) is basically the same thing. I even made a video example https://www.youtube.com/watch?v=ViCjzJpEaJw that failed to convince them.
arnaudlaudwein 17 ago 1 reply      
This is legal[1] in Europe if you consented to receive marketing emails from "partners" of a website you subscribed to (through an opt-in, not an opt-out checkbox).

You subscribe to website X, you opt-in to offers from third-parties, and this allows X to share your e-mail address with Criteo. Then Criteo sends you marketing e-mails for the account of Sears (but they surely don't share any PII with Sears - the e-mail is sent by Criteo).

The logic isn't that "browsing Sears is considered as having a preexisting business relationship with them". It's because users opted-in to third-party communications from a website they may have signed up with, back in 2008.

Other similar use cases include sending you an e-mail for website X when you browse website Y because they know you are in front of a computer/phone and this increases chances of opening e-mails.

Doesn't make it more or less "right" though and it's surely very surprising for users, myself included.

(On a tangent, what still looks like a legal gray area to me are the Data Management Platforms (DMP) - everyone shares user data in a big bucket/database provided by a common partner, all users are identified with IDs but not directly with PII, how much data can companies push/pull legally?)

[1] Not a lawyer but worked with legal teams on these topics. Laws still differ slightly depending on the European country you're talking about, but the GDPR will soon be unifying data privacy regulations. Right now the French and German Data Privacy regulations are some of the most restrictive ones.

kazinator 20 ago 3 replies      
I wrote myself a web application called Tamarind that runs on my web server for managing throwaway mail aliases.

"Tamarind" == "Throw-Away Mail Alias Randomization Is Not Defeatable"



I log in with my IMAP4 user name and password, and then get a simple UI with a table of my aliases, and attached memo strings (which can contain URL's that get converted to links). I can edit these, change their order (select multiple, move to top or bottom, etc) create new ones and delete. When I create an alias, it goes "live" instantly, and when I delete one, it goes dead. Dead means that the address is "unroutable" at the SMTP level; it bounces.

I keep a few aliases from Tamarind in my wallet, in case I have to hand out an e-mail address in "3D life" to some untrustworthy outfit to be eligible for some promo or whatever.

gwbas1c 20 ago 3 replies      
This is why I own my own domain and have a catch-all email address. When I give a company my email address, I use (companyname)@domain.com.

They all forward to gmail; where it is very easy to filter out (companyname)@domain.com once shenanigans like this happen. It's also easy to track down and shame companies for doing this, too.

rapht 2 ago 0 replies      
I personally switched to the following policy a year or two ago to avoid all this crap:1) NoScript extension filtering everything except the base domain => no third party scripts are allowed except when I explicitly allow them2) Cookie Whitelist extension to allow cookies only from domains I choose, only when I need => no third party cookies allowed, ever3) Block incase the webpage tries to load iframe ads4) a unique email address per service (like amazon.[5 random chars]@mydomain.com) so if all else fail and your address gets in the hands of somebody who should not have it, you know where it came from and can expose them
chime 19 ago 5 replies      
I feel less paranoid now for my browsing process. Almost everything I search is in an incognito window, from shopping and research to programming and how-tos. And when I'm done with looking for a new dog leash or Python module, I close that window. Only things in my main browser are the regular sites I visit and am logged into (email, HN, reddit etc.)

I started this after learning about the filter bubble but I've noticed how helpful it is when searching on Amazon, Wayfair, or Sears. I get non-machine-learned results every time while my wife using her primary browser with cookies often cannot see the same results I do. If I find something on Amazon, I copy-paste the URL without the ?query-string and replace 'www' with 'smile'. It seems like a hassle but it's no different from cleaning your feet before stepping inside the house after playing in the park.

This post just highlights that my practice to avoid unpermitted-profile-building-and-linking is for a good reason. I also have my own @example.com domain that I use and have certainly caught companies selling my info. However, even without being emailed, I don't want algorithms the determine what is best for me based on criteria I choose not to share.

idlewords 17 ago 1 reply      
It's a little rich to write this complaint on Medium, a site that has been uniquely aggressive about tracking its readers' behavior (it has a script that phones home with your position on the page, and its URLs abuse the fragment identifier to track who you got the link from).

If you dislike surveillance capitalism enough to write an essay about it, think about where you're publishing it.

DoubleGlazing 1 ago 0 replies      
My wife's cousin had something like this happen to her two years ago when she was planning her wedding.

She browsed a few specialist wedding sites for inspiration and when she went to to some well known retail sites to start pricing things they seemed to know she was getting married and promoted wedding goods and services on their front page to her.

It freaked her out no end. I suggested a few plugins that seemed to put a stop to it. But a few weeks later she did start getting wedding related snail mail spam.

Its very creepy, especially after the whole Target teenage pregnancy thing.

Animats 19 ago 1 reply      
You still have third-party cookies enabled?

Go to Options in Firefox under Privacy, and set "Accept Third Party Cookies" to "Never".

sklivvz1971 16 ago 1 reply      
I just mark all this stuff as spam, including stuff from legit companies that might have tricked me into subscribing to some list.

The thing is, I am never, ever interested in receiving marketing emails. Every single time, without doubt, I opt out of marketing emails. So if I receive one it means that one of these things holds true:

1. It's just spam

2. The website used some dark pattern to trick me into subscribing to something I did not want to

3. The website assumed consent and didn't bother asking

Guess what -- I'm perfectly fine burning all of this crap with a spam filter. It's a waste of time, and time is my most precious asset.

tempestn 14 ago 0 replies      
> Only when we craft the email on behalf of our advertisers, we receive your name, surname and email address from our partners, should you have consented to receive their emails marketing.

> Lets ignore the fact that they assume Sears had my consent (they didnt).

Just a note: I think what Criteo is saying here is that you gave permission to some third party to use your email for marketing purposes and to share it with their "partners", not that you gave Sears permission to use it. But they shared it with Criteo and Criteo shared it with Sears (or sent the email on their behalf) so technically there is "consent". (Of course in practice it's often possible to supposedly give such consent without ever realizing what you're opting into.)

gwbas1c 20 ago 2 replies      
This is why gmail has a big fat "REPORT SPAM" button. Shenanigans like this are SPAM, and should be reported accordingly.
jdavis703 19 ago 3 replies      
This is why I have the username part of my email address tailored to each site/service I register with. So I have a hackernews@example.org, amazon@example.org, etc. Human beings get my real email though (because it would be weird if I told John Smith to email me at johnsmith@example.org). If people start abusing this (politicians do this a lot), I can just block say timkaine@example.org, and never hear from them or people they've sold and traded my email to.
jimsug 5 ago 0 replies      
And this is why I use uMatrix, despite the little bits of extra hassle I go through when visiting sites for the first time.
rootlocus 18 ago 1 reply      
I was thinking whether or not sending these emails actually helps companies like Sears by bringing in customers, and whether or not (to an extreme) they might depend on them to survive as a profitable enterprise. What came to me as a revelation is that it's irrelevant. If their income relies on bothering everyone who comes across their website, tricking them into clickbaits or spamming them with (possibly malicious) ads, it might mean their services are not enough to justify their existence. As such, I decide not to pity them, and happily continue loving my adblocker.
r721 20 ago 2 replies      
This is the reason I keep "Block third-party cookies and site data" option checked in Chrome.
Jaruzel 6 ago 0 replies      
From the Article:

 I am signed up to some platform which is a Criteo partner. Its entirely unclear who this partner is. While Criteo boasts a close partnership with Facebook, Facebook claims that they do not share personally identifying information such as your email address with ad partners. Regardless, a platform with my email address gave it to Criteo.
This issue is exactly why I use specific email addresses for each website. I tend to follow the pattern <websitename>@mydomain.com. That way if a site leaks my email address to spammers (either intentionally or accidently) I know which site it was, and immediately boycott them in future and move that email address into a blacklist.

For big sites I cannot boycott, I simply register a new email address with them (i.e. <website><number>@mydomain.com), and move the original into the blacklist.

As I run my own on-premises email system, I can't benefit from crowd-managed spam systems, so keeping a lid on the incoming spam is very much a pro-active action for me.

andrewaylett 4 ago 0 replies      
Privacy Badger is pretty good at blocking things like this -- it watches out for domains that are third-party for more than one site, and blocks requests to them. Does require some tweaking for genuine CDNs (and indeed comes with a yellow-list of common domains that will receive requests but not cookies) but generally very useful.


TeMPOraL 17 ago 0 replies      
Yesterday, after many years, my curiosity finally got better of me - I started playing World of Warcraft. Since my head is now full of thoughts about MMO, excuse me for saying this:

There should be a new class - or race - added to fantasy worlds. The Marketers. More evil than demons, undeader than the Lich King. Their gameplay mechanics would be based around earning gold by draining their own souls, as well as the souls of characters around them. Their primary combat role would be casting annoying debuff spells at everyone around, friend and foe alike.

Seriously though, this article basically says that someone out there has reached another level in insidiousness. If it was an MMO, we could at least form a raiding party and get rid of the problem once and for all.

dnh44 18 ago 1 reply      
nashashmi 15 ago 0 replies      
I don't understand why everybody does not block third-party cookies by default. I took a stab at my cookie list and found 300 cookies from advertisers and intel gatherers. I deleted them selectively, but I did not want go through that again, so I blocked the third-party ones.

Some have been explicitly aloud because I trust them, like google analytics. But other google cookies are prohibited, like plus.google.com. Facebook is explicitly blocked. Doubleclick is blocked. some websites will not work if certain third parties are blocked, so i have to explicitly allow them once i realize the problem.

loup-vaillant 16 ago 0 replies      
How fitting. I have just received a mail from Medium with no "unsubscribe" button because I commented on it.
jamiesonbecker 19 ago 1 reply      
tl;dr: related: Amazon sold (or gave) my secret Amazon email address to third parties without my express consent rather than using their remailers.

I have exactly one email address that I use for Amazon, and I've never used it elsewhere for anything else.

I occasionally receive emails from vendors (through the vendors' mail servers themselves, not remailed through Amazon per mail headers) at Amazon that I have bought things from (via one-click) as gifts and I am 100% sure I never gave them my email address or replied to any email from them.

An example vendor is a large outdoor clothing store that I bought a North Face jacket for a relative from. I'm now on their mailing list. In the ultimate irony, I could just click unsubscribe but it's actually good stuff ;)

Thanks, Amazon.

a_imho 5 ago 0 replies      
I know there are a couple of solutions out there, but what exactly stopping the main email providers to offer on demand proxy addresses for one's main account? I think there is a legitimate demand for it, but not enough to actually sign up for yet another service.
upofadown 18 ago 0 replies      
>The CAN SPAM act actually allows direct marketing email messages to be sent to anyone, without permission, until the recipient explicitly requests that they cease (opt-out).

Isn't this the root problem here? It is hard to see how you could even start to fix this sort of thing without fixing the spam law first.

cptskippy 16 ago 0 replies      
I've been using a catch-all email domain for years where anytime I give out an email address, the local part is a description of the party receiving the address (e.g. bestbuy.com@mydomain.com).

If I receive spam at a particular address, it's easily blocked and I know who leaked it.

An interesting side effect of receiving email from so many different addresses to the same inbox is that I often receive the same spam to multiple addresses simultaneously. This is easily caught by spam filters and so I never have Spam in my inbox. It also makes identifying false positives in my Spam box easy because they usually stand out against the repeated subject lines so it's a simple game of which one of these is not like the others.

justrossthings 9 ago 0 replies      
I talk a lot about this stuff with a friend doing sales operations at a hyper-growth startup in SF. With Criteo, tools like Reply.io and others he thinks we're going to see an event horizon where recipients of spam say enough is enough and online privacy finally becomes 'cool'.
chadgeidel 15 ago 1 reply      
Am I paranoid in assuming their "opt out" system is basically probably an "opt in"?

Related: I know that it's possible to "opt out" via the Direct Marketing Association communications (https://dmachoice.thedma.org/), but have thus far not done this as I assume I'll just get more junk mail.

__jal 18 ago 1 reply      
I think it is about time to build a one-click opt-out to preemptively opt-out of all of these scumbags' systems.
anilgulecha 19 ago 0 replies      
More directly - if you want your precious content/resources to make you money, make sure you send the bits over an authenticated & paid account.

Not behind an overlay, or with a adblock redirector or when the user-agent has 'googlebot' in it.

If you send the bits over, then I may consume them with no additional payment, whether via ads or mailing-list or account signups.

singold 14 ago 0 replies      
He talks about tge legality of sending the spam, but what about the legality of the partner he is really subscribed to that shared his information with a 3rd party? IANAL but AFAIK that wouldn't be legal in most countries
FussyZeus 19 ago 1 reply      
I just love the amazing "Terms of Service" that all of these ad companies have, letting you know that by virtue of loading an HTML page you've consented to have your personal information of ANY caliber spread all over their ad network, their "partners" networks, and to anyone else with a buck and a server, and immediately absolve themselves of any responsibility for what that might mean in terms of information falling into the wrong hands.

I can't think of another business that has this kind of insane amount of easy-to-start interaction that results in so much activity and yet can claim zero culpability for any consequences. It's as if you purchased an airline ticket and the ticket came with a 17 page document attached where they spell out that by flying on this aircraft you agree to have tickets pre-planned in your name for 24 other flights, the plane may or may not make a stop off in 6 airports en route to your destination, the pilot occasionally likes to do barrel rolls and loops but he's real good at it so don't worry, and by the way occasionally the engines fall off but you don't get to sue us if anything goes wrong. ENJOY YOUR FLIGHT

imron 12 ago 0 replies      
And people wonder why adblockers are so popular...
DavideNL 19 ago 0 replies      
what i usually do is reply to their spam e-mail on a support mail address and ask them to stop sending me spam: waste their time the same way they waste my time... if everyone would do that the problem would be solved.
walrus01 18 ago 0 replies      
... and I just created a new spamassassin rule for criteo. Done and done.
davidgerard 14 ago 0 replies      
"Dear Criteo: You opted in to this box of dead rats we just sent you because you once visited a site that partners with our dead rat promotion service."
ahm750 10 ago 0 replies      
Faced a similar situation recently. It was both surprising and frustrating.
sandworm101 11 ago 0 replies      
>>> But until legislation catches up to regulating the negative consequences of retargeting, there may not be much you can do about this besides blocking cookies, ads, and opting out of Criteos entire system by submitting your email address here.

No no no. Handing over your email address to an online advertiser is a horrible idea. Do not engage them. Blacklist their content, their cookies, via whatever means you want (I use adblock) and be done with them.

An article that discusses tracking via online advertising but doesnt discuss blocking is very suspicious. The most powerful tool against the problem isn't worth even a mention?

ChuckMcM 17 ago 0 replies      
Set privacy badger to block all Criteo cookies.
joesmo 20 ago 1 reply      
I'm not going to wait for legislation to fix problems I can fix myself. You don't want this to happen? Make sure you have ad-blocking and third party tracker blocking on. I go a step further and use 'Quick JS Switcher' for chrome. By default JS is off and I only turn it on for sites I want. The percentage of sites that I turn it on for is minuscule. I'm seriously starting to question why this isn't the default setup for any freshly downloaded browser.
astdb 12 ago 1 reply      
Would disabling third party cookies have prevented this?
sneak 18 ago 0 replies      
uBlock origin plugin. Globally disable 3p resources for all pages. Manually greylist CDNs only for sites.

Browsing the web any other way is for schnooks.

dredmorbius 8 ago 0 replies      
I've mentioned putting the Winhelp2002 hosts file on my dd-wrt router a few times. I just checked to see if I need to add any specific entries.

 root@router:/tmp# grep criteo hosts0 cas.criteo.com dis.criteo.com dis.eu.criteo.com dis.ny.us.criteo.com dis.sv.us.criteo.com dis.us.criteo.com ld2.criteo.com rta.criteo.com rtax.criteo.com sapatoru.widget.criteo.com sslwidget.criteo.com static.criteo.net static.eu.criteo.net widget.criteo.com www.criteo.com
Apparently not.

Deets: https://ello.co/dredmorbius/post/v9l7zvlyynvl1pskbwssmq


LoSboccacc 16 ago 1 reply      
here's another special snowflake that does the internet equivalent of 'I want to talk to the manager'

it is their website, so if you don't like their terms, leave.

I do run adblocks etc to avoid drive bys, but if a website relies on delivering me spam&malware I just avoid it

Also why is this guy so upset about retargeting now? it has been a thing for more than a decade, and without it one would just see some generic spam instead of these sorta ineffective reminders. creepy? sure! then again there's one simple trick to make them disappear: don't go on shady websites where you're the product and bring your money elsewhere.

20 hours ago 20 ago 3 replies      
not_a_codfish 16 ago 2 replies      
saltyhiker 16 ago 1 reply      
The real problem here is not the chain of marketing tech that allowed this, the issue is that the marketing message itself sucked. If the message was valuable, many people wouldn't have been bothered by receiving it.

As for the message itself, if their intent is to sell you that specific item you searched for, they should say so. Of course, they need to avoid the creepy-factor, which, along with laziness are the two reasons they may have ended up with the junky message you received.

malchow 8 ago 1 reply      
Please. There is no core privacy premise of the internet. The core premise of the internet is one protocol to deliver meshed knowledge to any computer. And the commercial possibilities of the internet are what have underwritten the growth of the network.

Reaction like this one make me think: entitled.

But they also make me think: unrealistic. How much should hypertargeted ads really bother us? Call me when they are using my bank account and medical records to show me ads. Not my browsing history, over whose exposure I have complete control, and which doesn't really expose very much about me or my family.

Node.js is one of the worst things to happen to the software industry (2012) cat-v.org
488 points by behnamoh  17 ago   363 comments top 54
imagist 7 ago 0 replies      
It's telling that none of the posts defending Node.js are talking about it's technical merits. They're all saying:

1. Attacks on people--you're being too negative, you're saying this to feel superior.

2. Choosing Node.js is a tradeoff! We can't really say what you get in exchange for using this crappy ecosystem, but "tradeoff" sounds good even if you're trading using a reasonable ecosystem for nothing.

If you really think Node.js isn't a flaming pile of crap, I challenge you to come up with something it does that isn't done far better in another ecosystem.

lootsauce 8 ago 16 replies      
There is an easy sense of superiority that comes with derision of "X" and the authoritative sounding romanticization of idealized "Y" seemingly adds weight to the argument. Clearly these "Y" people are smarter and better than all those "X" people.

Look at all this horrible code! So sad that all these people are not as smart as me. Look at this horrible language! So sad the people that created it are not as intelligent as me!

Really? There is no more "problem" with Node.js than there is a "problem" with any other platform. There is no more problem with JavaScript/ES-(name your flavor) than there is with any programming language. Different languages are different. Different platforms are different. Of course every system has its own problems. Sometimes people who appreciate them call these "tradeoffs" or the superior types call them idiotic.

The cliche of hacker news haters is really really really getting old. So here are some things that are actually good.

As much of a pile of hot steaming code as it is, Babel as an idea (AKA transpiling one language to another) is pretty cool. Of course you can do this in other places but its featuring prominently in the JS community leading to an interesting result. The language and its features become configurable, easy to adapt and change and evolve over time and suit to your liking. This is interesting!

Finally as opposed to what others may have said about the community being childish I have found the opposite. I find it to be very welcoming and supportive, friendly and honestly creative. Of course there are lots of negatives, lots of horrible code, lots of mistakes happening. But what is missed in all of this? Theres A LOT of stuff happening that is good even great! It's beautiful chaos! So go on hating, but I see lots of great stuff out there. As one great systems and iOS developer told me the other day "Have you tried Express? Its awesome!" HA yeah. But he just tried it, and loves it!

Oh but look at that callback YUCK! Cmon

Jemaclus 11 ago 19 replies      
Disclaimer: I worked with Node.js full-time for about 14 months, ending about 9 months ago. I'm willing to admit that the following complaints may or may not be out of date, especially with ES6 and newer Node versions. Proceed with grains of salt at the ready.

There are a lot of things to like about Node.js, but the primary thing that bothers me about it is the obsession with async. It's a language/framework designed to make pretty much everything async.

In the real world, almost everything is synchronous, and only occasionally do you really want async behavior. By that I mean, you almost always want A, B, C, D, E, F, G in that order, and only occasionally would you say that you want async(H,I). But with Node, it's the other way around. You assume things are async, and then specify synchronous behavior.

Instead of assuming A; B; C; D; E; F; G;, you wind up with a ton of code like A.then(B).then(C).then(D).then(E).then(F).then(G);

...I know that's a contrived example, and I know you don't really need to do it that way, but so many people do, and it really illustrates the point. In Node.js, you are explicitly synchronous / implicitly async. Most other coding paradigms (including Go) better match what I consider reality, which is that everything is implicitly synchronous, and you specify async behavior when you need it.

Basically, I think it's backward. But perhaps like the OP, I just can't wrap my head around it.

The NPM stuff... well, I think all ecosystems have their pros and cons. I'm not a huge fan of NPM, but it does the job for the most part, and I'm curious as to how people would actually improve it, rather than just complain about it all the time. I don't really have any good ideas (knowing nothing about how package management actually works under the hood).

leonroy 5 ago 4 replies      
I've been a software dev for 10 years. When I started I saw the transition from Perl to PHP and a lot of snobbishness from the former towards the latter. Seeing the changing of the guard in web languages was pretty instructional and it's something I see again and again.

I think basic CompSci courses should really have a course or two on managing software projects and handling the problems of what framework do I use to build my new software app? Because fundamental language or framework decisions have both a technical and a business component and even as a front line programmer it helps to be aware of both.

Node.js is a great environment for getting a server side app going fast and it has very good tooling thanks to the rest of the JS community with additions like npm, gulp, bower, express etc. There's obvious benefit in having shareable libraries between client and server side and most importantly (to software companies) hiring coders who can work with it is far, far easier than say finding that rarest of unicorns - an experienced Haskell developer.

If (and it's a damn big if) you outgrow Node.js you're doing well. Then (and only then) look at the alternatives like Play Framework, Spring Boot, Vert.x or whatever else floats your boat.

Rants can be useful in giving a kick up the asses of the relevant community to go address certain bug bears. This rant though is so damn generic it reminds me of those Perl developers at college pouring cold water over the idea of using PHP because they felt threatened by it.

rdtsc 16 ago 4 replies      
I usually ask people to explain why they picked or like (or dislike) a particular technology and that surprisingly tells quite a bit about their proficiency level.

At least in interviews I found it tells a lot more about their proficiency than say knowing how to invert binary tree in under 20min or solve a digital circuit diagram in with object oriented principles.

Node.js is a technology that raises red flags when someone advocates it. I've heard stuff like "it's async so faster", "it makes things non-blocking so you get more performance not like with threads", "you just have to learn one language and you're done", "...isomorphic something..." When digging in to discover if they knew how event dispatching works or how these callbacks end up called data comes in on a TCP socket, and there is usually nothing.

The other red flag is the community. Somehow Node.js community managed to accumulate the most immature and childish people. I don't know what it is / was about it. But there it was.

Also maybe I am not the only one, but I've seen vocal advocates of Node.js steam-roll and sell their technology, often convincing managers to adopt, with later disastrous consequences. As article mentions -- callback hell, immature libraries, somehow the promised fast performance guarantees vanish when faced with larger amount of concurrent connections and so on. I've seen that hype happen with Go recently as well. Not as bad, but there is some element.

Now you'd think I am 100% hater and irrational. But one can still convince me that picking Node.js was a good choice. One good thing about Node.js is it is Javascript. If there is a team of developers that just know Javascript and nothing else. Then perhaps it makes sense to have a Node.js project. Keep it small and internal. Also npm does have a lot of packages and they are easy to install. A lot of them are un-maintained and crap but many are fine. Python packaging for example used to be worse, so convincing someone with an "npm install <blah>" wasn't hard.

YeGoblynQueenne 5 ago 3 replies      
So, this is totally an ad hominem but the OP webpage has a post category titled "Political Correctness" [1] with one post titled "Claims that sexisim drives girls away from Computer Science are feminist bullshit." [2] and one link to an external site titled "Sex Differences in Mathematical Aptitude - By La Griffe du Lion." (a.k.a. "Lion's Claw") [3] starting off with this bit, styled like a research paper abstract (you know, to make it look seintifikal):

Mathematics is a man's game. A gender gap appears early in life, blossoms with the onset of puberty and reaches full bloom by mid-adolescence. It indelibly shapes women's prospects for doing significant mathematics. In this account of cognitive sex differences, Prodigy shows how sex-differentiated ability in 15 year-olds accounts for the exiguous female representation at the highest levels of mathematical research. A female Fields Medalist is predicted to surface once every 103 years.

I stress again that this is essentially an ad homimem: I'm absolutely flagging up this dude as a sexist baboon with brains the size of a small, frozen pea. And because I have much better things to do with my poor feminine brain (say, finish off my MSc dissertation, on a novel grammar induction algorithm) I'm not reading a single word of the OP.


[1] http://harmful.cat-v.org/political-correctness/

[2] http://harmful.cat-v.org/political-correctness/girls-in-CS

[3] http://www.lagriffedulion.f2s.com/math.htm

annasaru 7 ago 0 replies      
J2EE is up there.. We have a generation of corporate Java programmers who love boilerplate code and obfuscation.

High-School teachers inflicting Java on teens - cringe-worthy. Many public schools only offered Java if you elected programming/CS (in High School). Only recently has Python made inroads.

jondubois 6 ago 2 replies      
I don't see anything wrong with Node's approach to concurrency. It uses IPC which is much more scalable than threads and mutexes. Also, you don't have to use callbacks anymore, now we have Promises and there are tons of libraries that allow you to do reactive programming so you can just wire-up streams of data together in complex sequences.

This article is just inflammatory and illogical.

I think Node.js is one of the best backend engines which was ever created - And I've programmed in everything including AVR Assembly Python, PHP, C#, C/C++, Java and many others. I like Node.js the most.

And yes, you're right, the Node.js community isn't a 'proud' community - We're more interested in constantly improving than sitting there being satisfied with ourselves whilst bashing other tools.

There is no perfect tool/stack; they all have pros and cons. It's all about personal preferences.

spion 7 ago 1 reply      
The pot is calling the kettle black.

Its always funny to read something like this coming from proponents of Go. Now thats a language that threatens to single-handedly bring us back 30 years in the past. A typed language invented in 2009 lacking generics and algebraic data types, and being proud of it (or at least its users being proud) - thats pretty much at the exact same level as JS "reinventing" callback-based async programming in 2009. And its users were similarly proud of this in 2010-2011.

At least Brendan Eich is modest enough to apologise for JavaScript's flaws and push as much as possible for fixes without backward compatibility breakage of the web.

dang 12 ago 1 reply      
This post is baity but the thread turned out to be rather good, so let's try turning off the flags on it. If it ends up a flamewar we'll have to put them back on, so if you comment here, please keep the discussion thoughtful!

Btw, the top bit of the article is an HN comment by uriel from 2012: https://news.ycombinator.com/item?id=4495305, and the other part had a major thread in 2011: https://news.ycombinator.com/item?id=3055154.

sebringj 16 ago 3 replies      
The number one issue I've found with Node.js is when developers make things overly complicated for no apparent reason. The bar may be too low to get in so possibly you'll get a higher degree of poor design decisions.

The second would be the overuse of build scripts in that the build seems more complicated than the app in both time to get the thing up and complicated chaining steps. I've not had much fun debugging grunt, gulp or webpack in some of these fortune 1000 projects and I have a hard time wanting to give a shit about knowing them in great detail as the app should be the focus.

The parts of node that I most like are the core libraries that come with the install. When I try to stick with those as much as possible rather than using some half-baked npm module for every whim, I have a very pleasurable experience.

The async/wait and promises, etc along with piping streams are quite elegant in how modules can be snapped together like lego pieces but I find that people fuck it up terribly when they half know it as I initially did and it becomes akin to the messiness of es5 callbacks.

It does take some time to really utilize async well so I would recommend to read up on those concepts in great detail prior to jumping in.

Please npm responsibly.

wyqydsyq 10 ago 1 reply      
All the points in the first (Uriel) rant are attacking JavaScript itself, and most if not all those points are remedied by using ES2015 and FRP.

All the points in the second (Ryan) rant are attacking bloated/poorly-scoped software and abstractions, and none of the points (except mentioning $NODE_PATH) have any relevance at all to Node.js.

So really none of this is relevant to Node.js (or JavaScript) being inherently bad, it's just pointing out that people can use it for bad things, which is no different to any other language or runtime.

diegoperini 25 ago 2 replies      
Facts are nouns, adjectives are subjective. A subjective analysis is hard to trust. I hate Java, because I choose to do so. Java being a bad language cannot be proven by my personal opinion. People do amazing things with it, Minecraft was initially Java, Android apps run on JVM etc etc.

This article has more than 10 adjectives (many have adverbs attached to them or written like "... much more sane and reasonable language like Python ...") in its first five sentences. Thanks for the effort but no, I cannot trust any of it.

nilved 1 ago 0 replies      
I try to understand why people would write JavaScript but I just can't. The only explanation to me is that they're not aware of what they're missing out on, or the deficiencies of their own platform, in a Dunning-Kruger sort of way.

I would be interested in hearing from some JavaScript developers about why they use JavaScript, but I have probably heard and refuted it before. Especially after Ryan Dahl disavowed Node, I think it's time to reconsider your viewpoint.

ralusek 16 ago 1 reply      
"The async-style of programming is almost entirely inaccessible to my brain"

Proceeds to write article shitting on language which relies on this.

lllorddino 1 ago 0 replies      
> better alternatives around with much more sound models and environments, Erlang and Go

Node.js makes it extremely fast to build robust and fully featured web apps quickly. I'm using Go currently for the backend and it's a pain since everything has to be written from scratch. I still enjoy it since the language is simple.

agentultra 1 ago 0 replies      
The worst thing is a gross exaggeration. I've been guilty of these at times. It seems common. You have to be a very passionate person to care deeply enough to master programming. These are infuriating machines operating in the domain of discrete maths... one error and the entire building comes down.

That being said sometimes, when there's no one else you can kvetch with, a good rant is just what you need. Computers are stupid. Programming is horrible. Everything is terrible.

But Node.js is hardly the worst thing. Some perspective is required.

moonshinefe 9 ago 0 replies      
"processes orchestrated with C. Its a beautiful idea."

That's all well and good until programmers don't know how to manually manage their memory properly or use pointers and it creates massive security holes and bugs in general. Which is unfortunately very, very often.

Apparently this guy disagrees and that's fine, but in my experience he's wrong to bash complexity and praise C. I've read many large C projects' code, and it's about as complicated / messy as it gets.

Some good points in the article other than that.

punnerud 1 ago 0 replies      
NASA is using Node.js for its Mission Control Frameworkhttps://nasa.github.io/openmct/https://github.com/nasa/openmct
tgarma1234 2 ago 1 reply      
Well I am by no means a great computer programmer or anything so I can certain accept other people saying node is crap but as someone who has used node my reasoning was simple: if you are going to use javascript in the browser and pretty much everything about the "app" depends on what happens in the browser then why use some other language on the server for models and controllers with javascript in the views? Why not just make it javascript all the way down?

Sure, smarter people than me might know why I shouldn't use javascript for everything but for me I do not find it useful to try to figure out how to do so much work in Rails or Django AND THEN ALSO end up using javascript in the views every time anyway. I would rather skip using Rails or Django and just get to the fun part of figuring out what happens next when this button is clicked on the website. I don't think I am alone in that. Rails and Django were a means to an end but they set the bar too high in terms of "programming" and require a lot of programming knowledge that I don't actually need to know to do what I want to do now with node. Sure, my code might suck but unless you are working in some super professional environment with highly skilled managers, pretty much everyone's code sucks. It's about getting the functionality you want with the least amount of effort. Node makes that possible.

jtolmar 9 ago 8 replies      
So what do people use as a server these days?

All I want is to annotate some function calls with paths and have something handle http requests and mangle the input and output into json for me. Preferably in a boring safe language. Preferably in Java, the boringest safest language.

If I have to call a method on a provided object instead of returning in the name of async, great, that's fine too.

mcs_ 1 ago 0 replies      
There will come a point where the accumulated complexity of our existing systems is greater than the complexity of creating a new one. When that happens all of this shit will be trashed.

I think it applies to bank system, politics, and more in general to anything that can be improved and where there are enough money to think about re-design. At that point yes, another life cycle will starts.

progrocks9 1 ago 0 replies      
NodeJS works great as a simple web or network container, if you're trying to create a complex enterprise app in Node, basically you're doing it wrong. Simply it's difficult to design complex nested logic or algorithms in a clean manner. It's possible but it's harder than other languages.
bryanrasmussen 16 ago 1 reply      
I am certainly willing to entertain the notion that async models do not map naturally to most peoples' brains and as a consequence the callback style of concurrency might just be really difficult to handle. On the other hand the writer admits it may just be them not being able to understand it, which I think is also reasonable given that many people have managed to do more complicated stuff in much less than 2 months.

This post is a land of contrasts, in short.

yoshuaw 16 ago 1 reply      
Here's the post by Ry that was supposedly deleted: http://tinyclouds.org/rant.html

The point I took away from the post is that all abstractions carry a cost, no matter how elegant. Not "Ryan Dahl hates Node.js". But sure.

Seems the author is intent on complaining and that's cool with me, but if you're doing Node then yeah don't let people like this get you down. Node's fast, Node's flexible and has so many users that virtually any abstraction you like is available through npm. Callbacks are part of core because it's the lowest common abstraction for Async. Node is not Python.

Tired devs can complain all they like, but that doesn't make them right.

tflinton 3 ago 0 replies      
I've done a fair amount of messing around with javascript.

1. Recompiled webkit into javascript using emscripten/asm.js (webkit.js)2. Created a desktop version of node (tint2) using FFI calls for OSX/Windows3. Reconstructed much of the oracle o5login/tns/tti protocol for giggles (and if you want to see a very head scratching way of packing 64 bit signed ints, look no further than 11g).

What I can say is it isn't perfect. But it has one gigantic advantage is you don't need to build or compile to run, but still get nearly the same performance from compile time languages.

It's a near perfect prototyping language, and for most peoples use (e.g., 99% of crud/general business humdrum logic) it's perfectly fine.

Just stay away from relying on too many npm modules.

k__ 3 ago 1 reply      
What I find interesting is, that everyone has rather hard feelings for any language but Python.

There seem only 2 things people complain about Python, performance and the two major versions out there.

Overall I have the feeling most devs thing it's pretty okay.

coldtea 6 ago 0 replies      
What those repeating "async is good" don't seem to get is that we have had better ways to do async than what Node.js offers for decades.

Better as in, all of: faster, easier to reason about, better implemented, more robust.

Erlang is but one example.

zumu 11 ago 0 replies      
> As soon as I wanted to write any non-trivial code to read stuff from a database and do something with it, I got stumped - I didnt know how to proceed. I could write some code, but it would turn out very ugly. I couldnt write code that was pleasing to read (and it certainly wasnt pleasant to write).

When this was written, he had point, but with es6 I don't think this is a problem anymore.

Accordingly, I don't think this stands

> You lose expressiveness

Arrow functions and destructuring cut down on boiler plate significantly.

cdevs 3 ago 1 reply      
I used "callbacks" when needed in objective-C in my app days and got sh#t done just fine. I think the best thing to come from node is the ability for a younger crowd to jump into proton/electron desktop apps without needing to get into sdl, qt, or some platform specific language like objective c or the .net C# whatever family. Nose might not be the finally language to take us to that area but it's a start and a ton of awesome open source apps came out of the scene quickly.
everydaypanos 5 ago 0 replies      
Node.js solves the problem of "live webpages" w/ Websockets and Polling and chat apps and notifications etcetc.

And that counts. A lot.

Ericson2314 16 ago 3 replies      
I dislike shitty languages and successfully avoid using them myself, but I do have to be a bit happy with NPM ecosystem teaching people to reuse more code.
api 16 ago 0 replies      
Meh. It works. Npm is okay if you exercise quality control over what packages you use and keep dependencies down. The language is decently productive for scripting and everyone knows it so it's easy to find maintainers. It performs well and can be multithreaded just fine by running many concurrent node workers and using a good database and cache.

Go may be better but has a less rich ecosystem of turn key solutions and easy integrations. Java is a decent language but has an outdated ecosystem. .NET going OSS is interesting.

Ultimately what you do with languages and tools is more important than which ones you use.

aaronbrethorst 8 ago 1 reply      
Quasi-related, one of my favorite programming language essays of all time: https://eev.ee/blog/2012/04/09/php-a-fractal-of-bad-design/

From what I've observed, I feel like Node is where hipsters who decided that PHP was "icky" ended up.

kabes 6 ago 0 replies      
The article is not very relevant in 2016. Much of the arguments are not valid anymore.
rv11 7 ago 0 replies      
I see a lot of desktop apps made from electron/CEF using Node. examples are Brackets and Steam.
aaronpk 16 ago 4 replies      
Great line: "The only thing that matters in software is the experience of the user."
tim9009 7 ago 0 replies      
"The only thing that matters in software is the experience of the user.""if you add unnecessary hierarchies in your code directories, if you are doing anything beyond just solving the problem - you dont understand how fucked the whole thing is."

Oh really? So making the code easily maintainable and easy to comprehend is just a waste of time and simply ads complexity? Comments like that makes this hard to take seriously.

yAnonymous 6 ago 0 replies      
Yeah. I hate all these great text editors and other cross-platform software that Node brought about.
arisAlexis 9 ago 1 reply      
This whole thread would not exist when async await gets traction
supergetting 10 ago 0 replies      
I embrace change in life in general, such a view translates over to the complex ecosystems in technologies, I embrace and accept the complexity of every single technology, honestly it is quite amazing. It's good that we try abstracting away many things and make them more simple and less complex, but sometimes inevitably leads to more complexity and more shit, and I like it and it's fun to be part of such journey!
Jean-Philipe 5 ago 1 reply      
Uriel didn't quite explain why PHP is any better than nodejs. I'd be even interested to hear!
andrewstuart 6 ago 0 replies      
I rather like JavaScript. ES2015 anyway.
SimeVidas 16 ago 0 replies      
4 years old?
tim9009 7 ago 0 replies      
"The only thing that matters in software is the experience of the user.""if you add unnecessary hierarchies in your code directories, if you are doing anything beyond just solving the problem - you dont understand how fucked the whole thing is."

Oh really? So making the code easily maintainable and easy to understand is just a waste of time? Comments like that makes it hard to take the rest of what he says seriously.

neelkadia 4 ago 0 replies      
Can you make it good?
partycoder 11 ago 1 reply      
JavaScript in Web browsers has a lot of sandboxing (i.e: babyproofing). Node on the other side doesn't. If you are an experienced developer, chances are you won't do something lousy like coding by trial and error and playing with concepts you don't understand and copying and pasting from StackOverflow.

If you follow the golden rule of not touching anything you don't understand, you should be in a safe spot. But that's not the culture in the node community. The node community is all about sharing lousy snippets of code without error handling, without input validation, and without any regard for any non-functional requirement in poorly written blogs and npm. A community of excessive optimism and irrational risk taking.

Floating point numbers, for instance. Every number in JavaScript is a floating point number, and 99% of node developers don't understand a basic thing like how to compare 2 floating point numbers. And that's just the basics... let's not even discuss concurrency, parallelism, how to deal with files, memory, I/O...

phazelift 5 ago 0 replies      
Unbelievable, so many up votes for this complete idiot statement..
narendraj9 6 ago 0 replies      
Cannot access the site.
andrewmcwatters 16 ago 1 reply      
As much as I love Node.js, I also love this post. It screams the ever-repeated viewpoints of the experienced and tired developer.

There isn't enough of this on HN.

Wei-1 8 ago 0 replies      
cannot agree more, but we still use it.
sownkun 16 ago 0 replies      
bit of an overstatement if you ask me
ComodoHacker 7 ago 0 replies      

>I hate almost all software.

greendestiny_re 6 ago 0 replies      
I'd just like to point out the linguistic paradox of the "one of the worst" syntagm, as only one thing can be the worst, and "one of" implies a multitude of such things; if 20 things are the worst, then none of them are. In short, this is a lazy way for the writer to express his opinion but label it as a fact.
Almost 80% of Private Day Traders Lose Money curiousgnu.com
386 points by kabouseng  1 ago   253 comments top 46
habosa 16 ago 4 replies      
Excuse my meta-comment, but I notice that HN seems to have an obsession with proving that active trading is bad and that we should all buy low-cost indexes. And before anyone jumps on me, I 100% agree with that investment strategy and use it myself. I'd just like to comment on the intersection of that belief and tech enthusiasts.

It seems to generally come from a place of binary/analytical thinking which we're all so good at. Studies show that active investing is a losing battle for the average investor, so we assert that as fact and invest our money that way.

But I think deep down in myself (and probably others) it comes from a place of wanting to justify risk aversion. It seems that many people that we all know do become quite rich as investors, and the secret to their good fortune is not apparent. Studies promising us "they'll all lose in the long run!" makes us feel good about our decision not to participate in the game.

It's similar to the derision of startup equity, etc. I know many people who have become very rich at a very young age by latching on to the right startup, but I like my comfortable BigCorp job. The HN comments are a place where I can feel smart about my decision to not take risks even as we celebrate those who took risks and won.

Anyway I don't know why I chose to write this comment or explore this topic at this moment, I hope someone else reads it and understands my sentiment.

nicholas73 1 ago 8 replies      
This blog post did not answer it's own question because it's conditions were not day trading (over 3 trades in 12 months). That condition selects for people choosing individual stocks hoping for a moonshot, for which people tend to choose riskier stocks rather than stocks actually likely to make them money. So no wonder 80% lost money. On the other hand, notice that the 20% who do make money have a large power distribution curve.

A better way would have been to compare performance versus frequency of trading (I don't expect this to prove one way or another though). The fact is, there are many strategies one could take, and it's how well you execute them that counts.

Personally, I don't find daytrading riskier than holding stocks. By far my biggest losses come from holding the wrong stocks for a long period. It just seems riskier because you have to confront yourself with the possibility of loss each day, rather than hold "long term" and deny that you are wrong.

I now take the Doyle Brunson approach. The poker champion loved to pick up small pots and felt it was critical to do by aggressively playing small hands. That way, these little wins pay for the risk of playing bigger hands over time. I've had the same experience - my daytrading tends to be small money but it stems from work done for holding long term stocks. So why not put it to use?

But, to actually make good money daytrading is still really difficult. Commissions alone can make you have to be 55/45 correct, but there is also the steamroller affect where people tend to hold on to losses and double down further. So it's also about mastering yourself in addition to your market. Otherwise, there is not reason why you can't be better: you are putting in more work than others to make good decisions, and that's how you profit. Trouble is, when are you still outgunned informationally?

crucialfelix 1 ago 2 replies      
I did it for a while in the very exciting year of 2008. I see-sawed back and forth and was profitable for months at a time, but in the end I was down. I strictly kept my daily losses limited at $50/day. So really small risk. At the end of that year my day trading losses were about $1500 and otherwise I was in cash just watching the world burn. My long term funds lost more than that in a day.

I've lost out on more money by not being in the market long term going up.

Of course I thought I might get good at it (everybody dreams), but the main reason I did it was to learn to confront fear and to keep logical and strict to the trading plan in the face of it. I am still risk averse though, so I didn't really learn thoroughly.

The other main thing I learned is that if you aren't a robot then you don't have any business out on that killing floor. The age of human day traders is over.

Animats 20 ago 0 replies      
Even worse than day trading are "binary options". This allows betting on, for example, whether the price of gold will go up or down in the next minute. Most of those are total scams. There are about 100 binary trading firms based in Tel Aviv, Israel, and most of them are telemarketing operations for a software platform called SpotOption.[1] Features of the scam include 1) they're not brokers, they're the other party, so you're betting against the house, 2) the house controls the price info and may tweak it slightly to make you lose, 3) there's a "margin trading" system which prevents you from withdrawing until you've done a certain amount of trading volume, and 4) even if you win, they make it really hard to withdraw money. Most "investors" lose 100% of their money.

Read "The Wolves of Tel Aviv" in the Times of Israel.[2] All these outfits are prohibited from operating in the US (the CFTC catches some of them trying; "Banc de Binary" was forced to return all US customer losses) but they've been able to operate in the European Union by registering in Cyprus and using that as a passport to the entire EU.

[1] http://www.spotoption.com/[2] http://www.timesofisrael.com/the-wolves-of-tel-aviv-israels-...

inputcoffee 1 ago 3 replies      
Isn't is possible that the sample is biased because you only looked at the subset of traders who were not sophisticated enough to turn off the feature that allows others to observe their trades?
joosters 1 ago 1 reply      
The Financial Times often refers to spread betting companies as 'debt collection agencies with a side interest in trading'. Emptying the accounts of users and then collecting the additional money owed is what these companies do. Then they use some of the profits to advertise for new customers to continue the cycle...
brongondwana 1 ago 7 replies      
It's just slot machines with a more respectable coat of paint.
chollida1 1 ago 4 replies      
I re-read Michael Lewis' Liar's Poker once every few years. In new reprints he's added a prologue where he says something along the lines of "I tried to write a book about the horrors of wall street and instead it became a recruiting tool for wall street"

In a similar vein whenever something like this comes up I write something that says don't try and do this and inevitable I get 10+ emails from people saying "Ok I understand but I still want to do it anyway."

So if you really want to try and day trade, here is my 5 points of advice

1) You haven't done anything until you trade with real money. Ever played poker with friends that wasn't for money? You always get some jerk bluffing and going all in on Ace-7 because why not, it doesn't matter.

If you want to trade you need to learn your own emotional limits. Get invested as soon as possible. Learn what kind of stress you can deal with.

2) The average successful individual algo trade looks like John Turturro charter Joey Knish in rounders. You are always grinding out a living $100 to $1000 at a time. If that isn't' appealing then stop before you start.

3) You need money to begin. It takes as much effort to trade 2000 as it does 200,000. Above a million or so things change. A minimum amount I'd use would be 50,000. If you can't set that aside to trade then don't start.

I mean if you have a great year trading 5,000 and make 20% you have made $1,000. That's not worth the effort. But at say 100,000 your 20% is starting to be real money!!

Be clear to yourself that you are doing this for the money. Any other reason just gives you an easy out.

4) The most important number to look at is draw down, not sharp ratio. I see lost of people on quantopian show great backtested algos that return 200% over 5 years, but have a 70% draw down at one point.

Most people don't have the psychology to loose money. At 10% down you start to question yourself. At 20% down you get really grumpy in all aspects of your life. Most people don't have the ability to let their positions fall to 30% down. If your backtest has a draw down of more than 30% you don't have a workable algo.....period.

Sharpe ratio is great if you are an HFT firm trying to decide how to divvy up your money across 30 successful algos. You don't have 30 successful aglo's. You will be lucky to have one.

5) Don't use leverage....... you can break the other rules, this one you can't. I've seen people go bankrupt. Don't use leverage.

akrymski 1 ago 1 reply      
eToro wouldn't be used by any remotely professional day trader. Most of those accounts are most likely idle and judging by eToro's target audience - highly amateur. Whilst majority of course loose money, so do majority of businesses in any industry, from startups to restaurants to hedge funds. The nature of human economics is such that there's more money chasing opportunities than the value of those opportunities.

Suggested reading: "Economics in One Lesson"

songzme 17 ago 3 replies      
Here's a story of my mom's journey as a day trader: My mom got laid off in 2008. The job market was brutal and she had a hard time looking for a new job. She had to find ways to make money to support my sister and I through school so she turned to the only other place where she could earn some money - the stock market.

She started with around 100k, read up about stock options, and decided that shorting calls and puts would work well. The time value decay seems to work in the investor's favor. Every time she made enough money, she would take them out and buy houses (all cash) and start from 100k again. Now, 8 years later, she has acquired a few properties in West LA and Oakland and makes more rental income per month than my salary as a senior software engineer. She still trades today.

mholmes680 1 ago 0 replies      
I have a serious problem with the "conclusion" of this post. This data isn't normalized against some kind of control group to show that maybe day-trading during Brexit is a different beast than index-tracking during Brexit. Or maybe its the same beast, we can't conclude anything...

I'm beginning to realize the difference between "data scientist" and just looking at numbers to put them on a chart.

hal9000xp 21 ago 1 reply      
Since 2012, I'm interested in financial markets. I read some university books. I trade from time to time (mostly index ETF and futures/options for experimental reasons).

I would argue that ~80% is quite optimistic estimation. I think in long-term more than 99% of retail day traders lose money.

It's NOT because making money on financial markets is ridiculously difficult.It's because if you trade frequently, you get feedback from market pretty quickly.The feedback in terms of profit and loss of your money.

If you become day trader, I would guess that you want beat the market (otherwise you just buy and hold index ETF).It means that you as a day trader is willing to take risks to lose money in case you are wrong that you think you are able to beat markets.

What does it mean "to beat the market" in long-term? It means that you are smarter or faster or have some insider information. In other words it means that you have competitive edge.

Well, obviously, average person doesn't have competitive edge against hedge funds and other institutional investors who hire people with outstanding intellectual capabilities. Just read hedge fund interview questions to see how they filter people.

Other way to estimate your situation as a retail day trader: if hedge funds don't want to hire you, why do you think you are still capable to outsmart hedge funds while you can't answer their interview questions for trader role?

I don't say that it's inherently impossible to beat the market as a retail investor. It's fairly possible if you are REALLY smarter than others. And in this case, you most probably are capable to impress professional traders and investors while talking about markets. If so, they will bring you their money, and incorporate hedge fund, and you are not retail investor anymore. Just like Michael Burry did.

In other areas, ~99% of failed attempts is also true. For example, ~99% startups fail to make money. So it's normal state of affairs.

0xmohit 1 ago 2 replies      
And the primary reason is that one loses more in a single trade than what one probably earned in 5 or more trades.

Humans are inherently risk-averse when it comes to protecting their profits and risk-taking otherwise.

The possibility to make money increases if one acts more like a robot, i.e. acting strictly upon rules decided a priori.

koolba 1 ago 2 replies      
If you think that's bad, check out forex trading accounts. I don't have the exact stats on hand but an insane percentage of them get completely wiped out.

I'm not talking about losing some of your money, I mean 100% of the money in your account and possibly owing even more.

igf 1 ago 3 replies      
Restricted to the publically-available data from one particular and rather odd site, but probably not crazy.

At first it seems counterintuitive, shouldn't it be roughly 50% gaining and 50% losing? But I guess that people typically put in a modest amount of play money, and then tend to trade until either they run out or get discouraged from losses.

83457 18 ago 0 replies      
I looked into trading a while back after my dad lost a bunch in the market. I read a lot of info and in the end finally found someone who stated information like this in the link. In the end I learned that the vast majority of retail traders lose money, I think 85 or 90% was listed, the majority of people who get into professional day trading quit within a couple years, day traders make profit by leveraging their money many times over just to make a living, and day traders actually take an income instead of letting their money ride and losing it all in a few trades but this also keeps them from progressively getting richer. Day traders expect to lose on at least 40% of their trades which is apparently one of the biggest issues mentally with trading. None of the retail platforms explain these facts nor many other sources of info online.
thedevil 1 ago 1 reply      
The way the math works out with that distribution: only 80% of them lose money over one year, but as time continues over multiple years, the percent losing money approaches 100%.
samfisher83 23 ago 9 replies      
If someone is consistently good at losing money 60% of the time. Why not just do the exact opposite of whatever their initial hunch is. Then they should make money. Based on loss of 36% from the article that is more than just transaction and trade fees which are 1 to 2%of trade therefore if they just do the opposite of what they are doing they should make money.
NDizzle 1 ago 0 replies      
... of course, 50% will end up in the bottom half and 70% will end up in the bottom 70%.

- Charlie Munger.


glippiglop 1 ago 0 replies      
It's an interesting stat, but the chart doesn't speak as to the levels of experience of the traders. All beginners lose money, but it's entirely possible for an amateur to become a profitable day trader with time.

Speaking from experience, the real issue for me was that once I started to get good at it (after 2 years) I realised just how awfully boring it was. It's not intellectually demanding and for the investment of your time, stress and the risk involved, the rewards even if you reach profitability aren't worth it.

Now that I can reflect on it, I couldn't be happier that I stuck with software development instead.

lordnacho 23 ago 0 replies      
Here's a quote from a guy who runs one of these shops:

"90% of the customers lose 90% of their money in 90 days."

A friend of mine was chatting to him about doing some business.

I would think it's more about leverage than commission. Looking at FX, you can get tiny spreads, almost comparable to what I saw in a hedge fund. It takes a while to eat up a whole account on such small percentages. Leverage, on the other hand is something you can use to demolish capital over an afternoon. Ratios like 1/200 are a formula to go broke if you haven't had a look at something like Kelly Criterion.

grondilu 23 ago 2 replies      
Isn't day trading a zero-sum game? If so, it's not very surprising that the majority of day traders lose money. Every dollar you make comes from someone who lost it. So unless you have a competitive advantage, such as lower brokerage fees, you should stay away.

Me, I believe in https://en.wikipedia.org/wiki/Buy_and_hold. The money you'll make will come from the actual profits of the companies.

gedrap 1 ago 0 replies      
I think the analysis could be improved by excluding people who made a couple of trades worth of a few dollars just as an experiment, etc. This way, leaving only the people who took it somewhat seriously. Even then, the data would be heavily biased - it includes only traders who decided to use eToro AND keep the history public. This is far from a random sample and very biased.

However, I would not be surprised if the actual results (across all brokerages, etc) are close to that, or even more extreme.

Paul-ish 17 ago 0 replies      
>On the 1st of August, 2016, I downloaded the publicly available data through their ranking API. I selected all users who were active during the past twelve months, traded with real money, and had at least three trades. The results consist of 83.3k traders who fulfill these conditions.

What may be interesting is to see a graph where the number of trades a person executes is on the x axis and on the y axis is median return. Do people who make more trades make more or less money? It would also be interesting to look at a window larger than 12 months. Do people with trades going back years do better than people with trades going back months? Is this gambling, or can some people build some skill in this?

DrNuke 1 ago 1 reply      
Was all in into Barclays shares the day before it was going bust in 2008 but got out before 5pm fearing the night and the wiping out. Instead, Gordon Brown bailed them before the market reopened the next morning ("too big to fail") and I missed my once-in-a-lifetime opportunity to make the big money as an indie trader. Done.
anonu 17 ago 0 replies      
The smartest thing an individual investor or trader can do is put their money in extremely low-cost index funds and leave them there. I have worked as a trader at various large financial services firms for almost a decade now. Even though I believe I have more insight into the markets in general than the lay person - the moment I decide to trade for myself, I will not get the same quality of execution and breadth of services I would sitting at my work desk...
JackFr 18 ago 0 replies      
> The data source for this post is eToro, a brokerage company that offers a feature called Social Trading, which is social network for traders. It is enabled by default and allows users to view and copy other users trades. Therefore, everyones trading performance is publicly available who have not disabled Social Trading.

What if the winners are choosing not to make their trades public? That alone could introduce some serious bias.

unknown_apostle 1 ago 0 replies      
And then there are these terrible products they promote to retailers (at least in Europe). Like CFDs and such. Orders in these things don't even go to the market, like in those bucket shops from old days.
bedhead 1 ago 0 replies      
Only 80%? Shocked it's not higher.
praptak 20 ago 0 replies      
"Why I don't trade stocks and (probably) neither should you", a very good summary about why it is stupid(#) to try to be "smart" about stock market:https://news.ycombinator.com/item?id=6831461

(#) If it's not your full time job.

nstj 1 ago 1 reply      
Ironically, it seems the HFT's have started eating their own lunch too [0]

> Overall, HFT firms revenues in the US have slumped from about USD 7.2 bn in 2009 to USD 1.3 bn in 2014 (see chart 3)

[0]: https://www.dbresearch.com/PROD/DBR_INTERNET_EN-PROD/PROD000...

thomasrossi 19 ago 2 replies      
"80%" it's very unlikely, say it is true, you just do the opposite and you have free money. I suspect the correct overall percentage should be much closer to 50%. Finding a profitable strategy should be much more difficult than "do the opposite of day traders", right?
drcode 1 ago 1 reply      
I suspect this is a relatively new development- I think 5 years ago I would have been pretty comfortable with my chances against AI trading bots. But in 2016, not so much.

Based on my own anecdotal evidence, I think the large corps building HFT systems with AI have gone a long way towards eating the lunch of "lone wolf" day traders in the last few years.

atemerev 1 ago 0 replies      

Anybody who doesn't understand exactly where money are coming from and why, will be ripped off, and probably deserves it. If you want to gamble, you will be gambled. If you want to reduce your risks, you have to research accordingly.

Nihilartikel 21 ago 0 replies      
Did not read the article; but is anyone looking at the kooky square-wave action on SPY and DIA today? I'm totally hopping on to the next 60m cycle! Can't lose.

Or... maybe not

btbuildem 1 ago 0 replies      
It would be interesting to know whether the gains were concentrated among a small group of same individuals over time, or equally across the large group (ie everyone getting lucky once)
pbreit 19 ago 0 replies      
The headline is probably close but I don't think you can reasonably extrapolate out from only eToro customers.
eddd 19 ago 0 replies      
Afaik 98% of forex day traders loose money. You can't compete with scalpers with 10ms advantage.
EvgeniyZh 1 ago 2 replies      
Well, how else someone would earn huge amount of money in this negative sum game?
wehadfun 1 ago 1 reply      
80% are losing who the hell is winning and how? I fell like they are cheating or have some sort of advantage that is not easily available to private day traders and no I don't think that advantage is a PHD and fast data connection.
known 23 ago 1 reply      
Insider trading is not illegal;http://cnbc.com/id/43471561
cloudjacker 22 ago 0 replies      
Almost 80% of Hedge funds lose money too
danieltillett 1 ago 1 reply      
And the other 20% just lie.
paul_milovanov 23 ago 0 replies      
Furthermore, almost 100% of private day traders lose money!

from the your headline is meaningless and you should feel bad dept :P

HashThis 1 ago 1 reply      
I'd love people's opinion on this. From my understanding, HFT (High frequency trading) firms are front running stock orders. NYSE sells them data on stock orders. They buy data center space in order to front run stock orders. That is how the HFT guys make a profit when Day Traders can't make the same kind of profit. Is that true?
Evidence points to another leaker at the NSA reuters.com
367 points by mudil  23 ago   217 comments top 25
silvestrov 22 ago 3 replies      
Even if it's not Russia, the release of the hacking tools gives weight to the argument that nsa/fbi should not be able to demand companies to create a back door in their products.

> Apple: If we're forced to build a tool to hack iPhones, someone will steal it.

> FBI: Nonsense.

> Russia: We just published NSA's hacking tools


adamc 23 ago 4 replies      
I don't think calling a thief putting up hacking tools for the highest bidder "another Snowden" is particularly accurate. (I realize this was the article title and am not complaining about the post here. I fault whomever crafted the original title.)
tptacek 22 ago 4 replies      
Where by evidence for another NSA leaker, Bamford means literally no evidence. The fulcrum of his op-ed rebuts an argument nobody is making --- that Snowden himself disclosed this cache of exploit tools.

The prevailing narrative, one echoed by Snowden himself, was that this was likely taken from a staging server: a machine somewhere out on the Internet used as a pivot point for attacks. Snowden claims (I don't know with how much authority) that a compromise of one of those staging servers is not without precedent.

Nothing in this entire piece refutes or even engages with that narrative.

John23832 22 ago 5 replies      
> In addition, if Russia had stolen the hacking tools, it would be senseless to publicize the theft, let alone put them up for sale. It would be like a safecracker stealing the combination to a bank vault and putting it on Facebook. Once revealed, companies and governments would patch their firewalls, just as the bank would change its combination.

Why would this be bad for Russians (if this was indeed the Russians)? We can/should assume that Russia has it's own methods of infiltrating systems. The value of this data to them would be knowledge of how it's done, not necessarily hoarding and replicating how the NSA does it. If anything, having vendor's patch exploits that they're not using, but their enemy is, would be a great chess move.

advisedwang 22 ago 0 replies      
It seems likely (https://www.schneier.com/blog/archives/2015/04/counting_the_...) that there are actually quite a few whistleblowers/leakers out there.
redwards510 21 ago 1 reply      
Do people really think this "auction" is legit? That people can seriously bid on these tools? The whole thing plainly sounds like a big joke for maximum attention. There is no real intention to sell the tools, at least not this way. The terms of the auction are ridiculous; no one with enough money to make a serious bid would risk losing it all like that.

Perpetuating the meme that this is a serious auction is dangerous and faulty journalism. It is a publicity stunt to embarrass the NSA. Let's not get hysterical and pretend that some third world terrorist country could obtain the NSA's cyber capability by bidding all of their petro-dollars in this farce.

okket 23 ago 1 reply      
Edit: Apparently somebody agreed and removed the clickbaity 'Snowden' from the title. Thanks.
lostlogin 20 ago 0 replies      
The article sort of makes fun of the hackers writing, then immediately writes "loosing" instead of "losing".
zmanian 21 ago 1 reply      
Bamford's expertise in espionage is pretty similar.

There seem to be two plausible explanations for the Shadow Brokers release.

1. The doctrine of the US govt in cyberwar is proportionate response. This is either preemption or escalation on behalf of Russia. This assumes the attribution of Russia for Democratic political hacks are accurate.

2. This is further activity by whomever leaked the ANT catalog to Applebaum.

The Shadow Brokers are going to be difficult to attribute technically. Attribution is based more on your theory about what's happening the Russian covert escalation.

grandalf 20 ago 0 replies      
Once a useful zero-day has been discovered by an adversary, it may make sense to give up using it so that one's own side's computers are not vulnerable.

My guess is that the NSA has excellent methods for detecting DNS exfiltration and the recent tools are at least a decade old technology.

What's interesting is the disinformation value of intentionally releasing the tools, but to understand that we'd have to know who the intended adversary was.

PieterH 22 ago 2 replies      
As soon as the author began referencing Applebaum seriously, the article lost most of its credibility.
justcommenting 21 ago 0 replies      
Does anyone else recall Jacob Appelbaum referring to RC6 constants [0] in public talks (e.g. 30c3) back in 2013?

I'm surprised Appelbaum hasn't been directly suggested as a potential source[1,2].

[0] https://securelist.com/blog/incidents/75812/the-equation-giv...

[1] http://crypto.stackexchange.com/questions/12661/what-could-l...

[2] http://www.nakedcapitalism.com/2014/01/jacob-appelbaum-30c3-...

dredmorbius 22 ago 4 replies      
It's inevitable that there are, or were, or will be, other Edward Snowdens working at NSA. Persons who find that the Agency's mission no longer sits well with them.

The question is: who are they working for?

Snowden was working for the American People, and upholding the US Constitution.

To draw from some relevant if non-US history, Kim Philby's interests did not lie with his nation's subjects, despite his aristocratic pedigree.

atmosx 21 ago 0 replies      
> A more logical explanation could also be insider theft. If thats the case, its one more reason to question the usefulness of an agency that secretly collects private information on millions of Americans but cant keep its most valuable data from being stolen, or as it appears in this case, being used against us.

If there's anyone still questioning the results of Snowden's move, here you have it. The Reuters opinion is stating that this data is potentially being using against us. If the perception that the NSA doesn't collect for the public good becomes broadly accepted, change can be achieved at a political level.

bronz 21 ago 2 replies      
i dont understand why the highest bid in the shadow brokers auction is less than a thousand dollars
return0 19 ago 0 replies      
Why don't we have widespread use of tools that spoof online activity ?
bogomipz 15 ago 0 replies      
"In addition, if Russia had stolen the hacking tools, it would be senseless to publicize the theft, let alone put them up for sale."

No not all Russia was recently accused of hacking the DNC in the US, so it would a perfectly logical for one state actor to say to another "you do it too and here is evidence." Is that not so obvious?

partycoder 20 ago 0 replies      
Or probably it's information intoxication (leaking fake information)
Aelinsaar 22 ago 0 replies      
...It's like they have no idea what "Evidence" actually means, or frankly what the hell Snowden actually did; it wasn't selling hacking tools on the black market!
progressive_dad 22 ago 4 replies      
nxzero 22 ago 1 reply      
Anyone that expects anything related to "intelligence" to have logical explanations might very easily find themselves chasing tails.
SixSigma 22 ago 2 replies      
This is all smoke and mirrors.

The DNC is claiming a Russian Hack with wonderful support from the media. Reuters, for instance, gave over $1m to the Clinton Foundation.

When the FBI accused the N. Koreans of the Sony hack, at least there was some credible evidence conjoured up. Obama used an executive order to apply more sanctions on NK even though there were voices saying it was still inconclusive.

But now we are expected to believe the DNC has been hacked by the Russians in partnership with Trump.

Seth Rich, supposed DNC leaker gets shot in the back.

And now this.

The DNC stabs Russia in the back - Bill was happy to accept $500,000 for a speech in Moscow and the Clinton Foundation $millions just before Hillary authorized a major Uranium deal.

I sound crazed writing this, like I'm something from InfoWars. This election cycle is standing the world on its head. Be very careful who you believe.

the_wumpus 22 ago 0 replies      
One hundred Royal Zorkmids for the title author!
sickbeard 22 ago 1 reply      
Isn't this the reason why governments react harshly to people like Snowden and Manning? To discourage even worse breaches?
thomasrossi 19 ago 1 reply      
"Its one more reason why NSA may prove to be one of Washingtons greatest liabilities rather than assets." WOW, never though I'd read that on something like Reuters, he put his foot down.. :s
Fighting Cancer hintjens.com
486 points by egorst  1 ago   143 comments top 31
Practicality 1 ago 5 replies      
Just an interesting related tangent here: My father in law is dying of heart disease (will probably die in < 1 month) and we run into the same thing.

People see me upset and the first thing they say is he will get better, or, it will be ok. When I explain the situation, it's something like, "oh, well, I just KNOW he will get better. Don't worry"


In case you are wondering: He has already had open heart 2 years ago, 4 of 5 arteries are completely blocked again (and those are already bypasses that are blocked again) and the remaining 1 is at 40% blocked. He found all this out when he went to one of the best heart hospitals in the world and they sent him home: "There is nothing we can do." Again, the open heart to bypass all 5 was only 2 years ago, so the math is pretty straightforward on how long the one artery will last.

He has something in his genetics in his family that makes it so this will happen pretty much no matter what he does.

The advice is endless. He has done his research and I believe he has had some real success in delaying this, he has outlived his younger brother by ~20 years (who died in a similar situation at 34). But it's over now.

I am trying to help his family get everything in order. And while it's sobering, it can be a very positive experience. But I wish our friends would stop telling us he'll be fine.

darod 23 ago 1 reply      
I found out I had colorectal cancer 4 years ago at 33. It was a shock because I go to the gym 4-5 times a week and taught brazilian jiu jitsu nightly. There no living right or living wrong prescription that will spare you. It can catch anyone at any time just like the author states.

Aside from dealing with the disease, one of the biggest issues I found was disbursing information and managing the emotions of my friends and family. Everyone has questions on your daily status and a few think they can come in a provide the superman holistic miracle that will spare you from death. It's tough to balance it all.

Like the author, I hate this notion of "fighting cancer". Norm MacDonald sums it up best. http://www.cc.com/video-clips/8kgu68/stand-up-norm-macdonald...

karmajunkie 1 ago 2 replies      
This is a really important read.

Nobody really gets terminal illness until you're either terminally ill, or right on the fringe of it. Even those of us who are one degree away from it only have the vaguest notion of it. It seems like something on the order of 30% of human prose ever written struggles with this notion of mortality [1], yet very few words are devoted to how to care about someone who is terminally ill, and even fewer on how to be there for them, providing encouragement without some kind of cheerleading. Part of this is because everyone is different, and some people really do want some of that. But my experience with it in this day is that those people are in the minority.

Thank you Pieter, for sharing your words with the world on this most personal of experiences.

[1] Totally made up statistic based on my gut feeling, so please don't bother asking for a citation.

univalent 23 ago 0 replies      
Powerful article. Something I can share from my own ordeal. When you first get diagnosed most people are strong and defiant "I'm going to beat this thing!". It is the following weeks and months, the follow up scans that show the darn thing is back or not reduced that eventually wear you down. If you know someone that's afflicted please keep in touch throughout or space out your acts of kindness. I found that initially everyone wanted to help (an outpouring) and in later months some help would have been useful.
nixarn 1 ago 2 replies      
A bit offtopic, but Hintjens has got the coolest activity graph on GitHub: https://github.com/hintjens
ciconia 1 ago 3 replies      
I've just saw my father die of cancer a week ago. Fortunately, he did this at home and was cared for by his own family, right to the end.

I wouldn't describe this experience as sad or tragic. We knew for a while that the moment was coming. I was lucky enough to be with my dad when he took his last breath, and to have been able to say goodbye. I learned a lot about what to do and what not to do in the face of imminent death.

baldfat 23 ago 0 replies      
My son fought for almost 5 years (Ages 7-12).

The outpouring of support was unbelievable. People I didn't know would do amazing things for my son and family. I saw this boy with one of the saddest stories (He was adopted) and a broken spirit before cancer became a amazing young man in the midst of his slow painful death.

His own biological family did very little during this time. Father murdered his mother less then a year after his diagnosis and family and close friends just didn't come around after a few months, "To painful to visit." I would flip out! Then I realized you just get to find out who is a true friend and family. So some will leave people high and dry others will see you all the way through.

newscracker 1 ago 1 reply      
> The only way to beat cancer, really, is to die from something else first.

That was a short and pointed article.

More so after I read a short comic strip on PHD Comics about cancer [1], I can't help but think that "beating cancer" is a very tough (and impossible) goal for the ones suffering from it and the ones looking for better management or reduction of it.

That shining light of optimism after remission is tinged with a hint that a recurrence is just a little while away, and could possibly be the end of life. I also wonder if what happens before death is more painful than the heartbreak that death eventually brings.

[1]: http://phdcomics.com/comics.php?f=1162

jMyles 1 ago 4 replies      
A beautiful read, for many of the reasons already mentioned here.

I'm inclined to rethink this one question, though:

> Can a single individual patient second-guess the medical machine? Is that really their duty?

I don't know if it's anyone's duty, but I think it's completely plausible for a single patient or small group of patients to arrive at a more patient-focused conclusion than the medical industry.

danieltillett 1 ago 1 reply      
A important post for those of us that don't know Pieter since it reinforces that your personal attitude does not change the outcome of cancer [1]. Cancer is a horrific disease, but it is not one that bends to our will, only our science.

1. http://www.apa.org/monitor/jan08/cancer.aspx

listentojohan 1 ago 0 replies      
I also read one of his previous posts on how to prepare the family, and talk with friends about dying. I've rarely been so moved by a post and had such an understandment of the situation, than from his posts. I think they are a must read for most people, as we'll most likely encounter it either through friends or family, or ourselves.
csl 1 ago 0 replies      
If you haven't heard it already, there is a really good and candid podcast interview with Pieter Hintjens over at Software Engineering Daily: http://softwareengineeringdaily.com/2016/06/23/death-distrib...
pixelmonkey 1 ago 1 reply      
Related: A Protocol for Dying, an interview with him for The Changelog from June 2016.


idlewords 1 ago 0 replies      
This is a very touching and generous post. I remember the kinds of diet and treatment advice my partner would get when she was fighting cancer, all of it well-intentioned, and wish those people had read this article.

I came to the thread to recommend a recent book by Atul Gawande, Being Mortal, http://atulgawande.com/book/being-mortal/, which covers the difficulties of dying (from age or from illness) and touches on many of the same points as this amazing post.

jwdunne 1 ago 1 reply      
In a way, it's similar in nature to how people will, with best intentions, tell you to smile more if you have depression.

I think the issue is many people don't understand the problem and they can't see it. In fact, someone with a common cold can expect to get more sympathy and better advice than someone with a chronic illness. The second issue is that many people struggle to think of something to say, its quite uncomfortable and the automatic choice is to give advice.

milesf 23 ago 2 replies      
I lost both my parents last year to cancer. Both were Christians, and so am I.

I know many people find Christianity and the subject of faith to be uncomfortable, even offensive. But that's because the Christian message _is_ offensive. It makes claims that exclude all other options, that evil is real and that we are responsible for it. To me, either the message is true or it isn't. There is no grey. Either Jesus Christ was a liar, a lunatic, a legend, or He is Lord God Almight.

Penn Jillette, the famous atheist and half of Penn and Teller had it right, that if we removed all the scientific research in the world we would be able to rebuild it all, but religions would be all different. I agree with him, and so does the Bible. It says that God reached down to us, delivered messages in ways that statistically rule out purely human effort, and gave us a choice to trust Him or not (have a look at http://thebibleproject.com) . In the end, everyone's going to get what they want (if you want Jesus Christ you get him, if you don't you won't). That's why for my Dad and I, we both had to be convinced that the Bible was not simply human in origin.

Whatever your view, I can only speak to my own experience. The loss of my mom September 1st, 2015 to double-hit lymphoma was very, very painful (she was 68). But in the midst of the pain was a hope and peace as explained in Philippians 4:4-9 (http://bit.ly/phil4_4-9). Then, unexpectedly 120 days later, my dad died from lung cancer (age 72), leaving my brothers and sisters and I with a property and 47 years of marriage and memories for us to sort through and deal with.

We are all going to die. The question is not if but when. To put off the discussion about what happens after you die is to deny reality itself, and telling others not to have that discussion or that their position is stupid or foolish is really dumb. Oh, and in case you think the Christian message is foolish, the Bible agrees with you that it is http://bit.ly/1cor1_18-25

reactor 23 ago 4 replies      
Very sensible read, just one question, he mentioned "avoid junk foods, especially sugar", is sugar that bad for causing cancer?
kohanz 1 ago 0 replies      
An excellently written piece, with a perspective that can only be communicated by someone who is walking that path (so to speak). The thought that the whole "You WILL get better" or "keeping fighting!" notion highlights for me is how taboo of a subject death and dying is in many cultures (very much including Western culture). I truly believe the reason people say such cruel and selfish things is because they cannot bring themselves to talk about the topic of death. It's something we are taught to ignore until we cannot possibly do so any longer. I feel like we might live better lives if we talked about death and dying openly and throughout our lives.
arisAlexis 1 ago 2 replies      
I am not sure why cryonics are not in the menu of every reader here when he has the opportunity (you don't have it if you die in a sudden accident) to subscribe when it seems inevitable to die.
raarts 17 ago 0 replies      
I have the utmost respect for Pieter, wherever I encounter his work it always shows passion and quality. Recently I read The Psychopath Code which I highly recommend especially to those who think they've never encountered one.While you can still read this: thank you. You have improved my life in multiple ways.
FuNe 22 ago 1 reply      
" There are people who treat the dying as easy prey. "For some reason this hurt most. Maybe I expected way too much humanity off humans.
_nullandnull_ 23 ago 0 replies      
Every time I see his posts I think about the book The Tibetan Book of Living and Dying. I would recommend reading it for anyone who might be having the discussion or dealing with death in their life.


alfonsodev 23 ago 0 replies      
"..Yet you are only as strong as the work you do"

All the article is very inspirational, mundane things like not solved paperwork, can carry lots of head aches to the family, sadly I know it by experience.

codingdave 1 ago 0 replies      
Some of this advice applies to people with chronic non-terminal health problems as well. We may not be dying, but the "helpful advice" from people who hear we have problems usually doesn't go over very well with us, even if we do smile and say thanks, to be polite.
dredmorbius 1 ago 1 reply      
I've been appreciating Pieter's posts, and his occasional comments on HN (I noticed one a few days back, on technical topics). I'm also increasingly apprehensive opening them. Though the only fatal disease I'm aware of fighting right now is life, my hope is to pursue my interests so long as I can.

I wasn't aware of Pieter before his recent blog topic started appearing on HN, though I'm pretty sure I'd come across his work. We're focused on different areas of tech.

I have seen cancer though, and much of what he writes here hits home, hard. I lost a very good friend, far too young, several decades ago. I'm looking at their picture now.

And remember going through much of what Pieter describes, though not as the central participant.

There were the other patients we met through treatment. Some of whom made it, some of whom didn't. And it wasn't necessarily those who appeared strong who lived.

There was the cheerleading and denial and people who were meddlesome. Those of us around the patient and family did what we could to steer the away. As Pieter says: the doctors tend to know the medicine pretty well (though chasing after them when things clearly aren't going right may be necessary). Unsolicited medical advice at this stage is almost always quite unwelcomed.

Small things can be huge.

What I remember, most, still: meals that showed up on the back porch with heating instructions. The neighbors had arranged amongst themselves a cooking schedule and coordinate this. No asking. No fuss. It just happened.

One less thing to worry about.

The other thing I remember was someone who, in all sincerity and good intentions, had forwarded information on a possible meracle cure. Laetrile. "The slickest, most sophisticated, and certainly the most remunerative cancer quack promotion in medical history," Wikipedia tells me today. We didn't have Wikipedia then, but I quickly established that this was in fact bunk.

It still makes me furiously mad: preying on sick people and those about the clinging desperately to any possible hope, in full knowledge that you're peddling bullshit. And those who get swept up in this and pass on the misinformation. Maybe that's why I've cracked down on online disinformation as well. It's not just duty calling....


Pieter's comments on how cancer is "fought" are also extremely good counsel. Some things can be manipulated and addressed directly. For others, you can only hope to set up the right set of circumstances to achieve the outcome you desire -- fighting cancer is more like tuning algorithms or seeds for some stochastic process -- a raytrace render or algorithmic music render, say, for those familiar with them -- than aiming a rifle at a target and taking shots. Our ability to directly influence events is limited, mostly you're managing the bits about you, your environment. Staying comfortable, staying sane. So much as possible.

In describing dealign with those around him. Pieter reminds me of a general classification I've used in other contexts for people:

* There are those who mean to do well, but are unable to. The cheerleaders and advice givers tend to fall into this category -- their harm isn't intentional, but it can be real all the same.

* There are those who cause problems through their own systemic operation. Healthcare insurance systems, vendors, legal processes, and the like. The issue's less one of having malicious intent, though here it's a lack of sensitivity to what their impacts on others are, or simply failing to care. The impacts on those who are sick or disabled are hugely magnified.

* Finally, there are those who are actively evil. Scammers, predators, sometimes even family or neighbors angling for what they hope they might be able to gain. This again makes me sick. There are no pits of hell deep or hot enough.

Many years after the experience I'd mentioned above (and after several others), I found a good model for offering care -- it's the concept of a kvetching order:


This consists of a set of concentric rings around a trauma, with the afflicted person at the center, and a growing set of less-affected care- and support-givers extending out. The basic idea is that care flows in, kvetches flow out:

The person in the center ring can say anything she wants to anyone, anywhere. She can kvetch and complain and whine and moan and curse the heavens and say, "Life is unfair" and "Why me?" That's the one payoff for being in the center ring.

Everyone else can say those things too, but only to people in larger rings.

Those who cannot (or will not) grasp and follow the concept are excluded.

The article also has another really wonderful piece of advice: that sometimes simply listening is the support that's needed. I've been on both the giving and receiving sides of that, and I'm not aware of when it's not been appreciated (though as with other advice -- people may differ, be sensitive to their needs).

One more thought: at least in Western cultures, there's often a profound lack of awareness of how to deal with death, impending death, or recent death. That's something which could use improving (and no, I'm not suggesting a YC opportunity). I very much appreciate Pieter's occasional communications for helping with that, at least here.

newuser1111 21 ago 0 replies      
I want to say this story is personal for me.
olantonan 18 ago 1 reply      
Quitting Twitter. Not quitting Twitter.

Dying. Not dying.

Sustrik is God. Sustrik is Satan.

What's up with this guy.


shanacarp 23 ago 1 reply      
First, for the general Reader:Outside of really breast/ovarian cancer, getting most common cancers at a young age is fairly within your control.http://www.nytimes.com/2016/07/06/upshot/helpless-to-prevent...

Pieter doesn't have a common cancer and he's way beyond that stage within stage IV cancer. In general it's really rude to say "fight more" because he's right, what do you think these patients are doing instead, joining the circus?Fighting isn't a great metaphor. The reason it got introduced was so that Reagan would create the National Cancer Institute, a Division of the NIH. A full page ad was taken out by Mary Lasker and the American Cancer Society to convince him that researching the causes and cures for cancer was a US healthcare priority (and to be frank, at the time, it essentially was) asking him to declare war on cancer in the NY times because the US was in the middle of the cold war. The other thing that the add did (along with the creation of the Jimmy Fund) was it normalized discussions of cancer in the US at that time.To explain how much of a big deal that ad was, my maternal grandmother died around 1973-1974 of cancer. The ad came out in 1969. My maternal grandfather is only now settling the argument if it was metastasis of her internal breast cancer from when she was younger, new breast cancer that spread to the bones, or a totally new bone cancer, because now we talk about cancer, whereas in 1969 -73 talking about cancer was difficult if not possible.


Aa a personal note to Pieter, if he sees this:1) I'm extraordinarily happy you are doing as well as you are in your end of days and I hope you are enjoying them to your fullest. i hope, for whatever it's worth, you are still experiencing moments of joy too.2)im slightly concerned as an American about your distrust of marijuna at this stage, especially since it seems like low pain and enjoying food is a high priority for you. In the US, marijuna is partially approved (don't ask) for cancer patients as an appetite stimulant and pain suppressor, and many of the chemicals in it are made synthetically and prescribed to cancer patients for the same purpose. Meanwhile, many opiates are appetite suppressants (that's the other reason behind the Medrol). Since you deserve to enjoy your time and have as many thalis as you want, just think about it. (Again, I'll totally admit that this is a bias of seeing Americans treated)3)do you need help getting the paperwork done. On a percentage scale, how much is left? How much can be done by volunteers/family/friends? (And I hate asking this, how much is in English, because I'm happy to volunteer, but I'm an English speaker...)4)thalis. Mmmm. Thalis.

Kenji 23 ago 1 reply      
Unfortunately, despite what this article says, there is a choice between fighting cancer or not: The best way to give up is stopping to eat (which probably isn't that hard if things like chemotherapy and severe illness pretty much remove any feelings of hunger).
nxzero 23 ago 0 replies      
Yes, all deaths are tragic, but increasingly feel that humanity does not have a good measure for priortizing its efforts.

As an example, over half of those diagnosed with cancer are over 70 years old and 100s of billions have been spent on research.

What is an object way to decide if all the effort spent of cancer research is of value relative to other area of research where progress might be made?

frign 14 ago 2 replies      
Introducing OpenStreetView openstreetmap.org
434 points by progval  2 ago   79 comments top 11
andrewljohnson 1 ago 1 reply      
Similar to http://mapillary.com.

It's great to see more open data like this, lots of uses for creating maps by hand or even machine learning.

rwmj 1 ago 2 replies      
Very cool. OSM has been the default map on my phone for years. I do worry that the name is a little close to Google's trademarked Street View (see list of Google TMs here: https://www.google.co.uk/permissions/trademark/trademark-lis... ). They might wish to avoid an unnecessary lawsuit by calling it something else.
nixos 1 ago 3 replies      
I understand that it's a volunteer effort, and volunteers don't have the hardware required for even 180 degree view, but most pictures are taken out of the front of the car, limiting its worth (2/3 of the picture is road, not the houses on the side).

It would be more useful if people stuck two phones (one on each side of the car) to take pictures of the passing buildings rather than road

dbrgn 2 ago 5 replies      
Exciting! Now instead of waiting for the street view car to pass through your neighborhood, you can add the pictures yourself!

I wonder how privacy issues are handled though. Is any blurring of car numbers or faces being done or planned?

ktta 1 ago 3 replies      
This is pretty cool.

What would be amazing to also have LIDAR sensors (EXPENSIVE! I know, I know) which would really make amazing data for self driving/completely autonomous vehicles; not only cars but also drones, etc.

Anyone know the pro/con for implementing LIDAR (although it maybe super late before we have enough data for effective navigation at long distances), and if it is worth it?

I think it would be really cool to have info about the heights of buildings, exact positions of traffic lights, etc. which are miniscule but crucial for low altitude drone navigation. Or can we infer almost everything from the picture data?

twelvechairs 1 ago 2 replies      
Props to the people who made this - fantastic public contribution. My big wish list as someone who works in urban design and planning would be this paired with 3d point cloud detection to make a 3d model of streets and spaces. Maybe one day eh!
opk 1 ago 3 replies      
Interesting that some roads in Germany are covered, at least for now.
francium_ 1 ago 2 replies      
Some of the images appear to be upside down. Is this a known bug?
astrostl 1 ago 0 replies      
Every huge entity having their own mapping system in order to grind out a minor efficiency is probably the biggest tragedy, to me, in modern services. Before mapping, it would be the TLS certificate racket, which LetsEncrypt appears to have dead to rights.
stelonix 1 ago 0 replies      
What I've wondered for a while is the feasibility of producing streetview imagery with drones. It seems to be the perfect solution for distributed street shooting.
ashitlerferad 1 ago 2 replies      
Anyone know what the business models of Telenav and OSV are?
Massachusetts to tax ride-hailing apps, give the money to taxis reuters.com
333 points by petethomas  2 ago   506 comments top 55
rev_bird 2 ago 15 replies      
This is a pretty surreal move:

>The law says the money will help taxi businesses to adopt "new technologies and advanced service, safety and operational capabilities" and to support workforce development... [Larry Meister, manager of the Boston area's Independent Taxi Operator's Association] said the money could go toward improving a smartphone app his association has started using, or to other big needs.

I'm not an anti-regulation guy most of the time, but this doesn't even seem like regulation -- they're literally giving money to a fading industry because they are incapable of keeping up. I don't want to see cab drivers turn into homeless drifters, but this just seems wrong.

Making a more efficient competitor pay to help taxi companies make their service better? The whole reason these companies exist is because taxi companies have been utter garbage for years and got the government to elbow out competitors. Am I missing something? Surely the burden for "do things better and buy nicer equipment" falls on the taxi companies, not the other organizations that are embarrassing them.

ricw 2 ago 3 replies      
This is unbelievable. Never have I used a more corrupt and unfriendly taxi service as the one in Boston / Cambridge. And they are supposed to be subsidised?!

I have 1) been driven round crazy detours to up my cost2) been forced to drive a detour because the card machine was broken and they only accepted cash3) been turned down because my destination wasn't far enough4) been shouted at because I was watching the free tv in the cab

In short, they suck. Badly. Thinking about it, I can't remember a positive experience. In contrast I've hardly had a negative experience with lyft/uber and mostly had very positive or positive experiences.

What's the best way of contacting this senator?!

jwallaceparker 2 ago 4 replies      
You and a friend are stranded on an island. You both spend all your working hours catching fish by hand in order to eat.

You use some leisure time to build a fishing rod, which enables just one of you to catch enough fish to feed you both.

It would be foolish to destroy the fishing rod to preserve both your jobs as fishermen. One of you would fish and the other would do something else, like build shelter and cut firewood.

It's just the two of you, so you don't literally exchange in barter, but an implicit exchange takes place when you share the fish, shelter and firewood.

You both enjoy a better standard of living thanks to the productivity increase created by the fishing rod.

A modern economy is much more complex but the same fundamental principle applies.

The ride-hailing apps make taxi drivers more productive by helping them connect with customers faster.

Now it takes fewer taxi drivers to satisfy the demands of customers. This frees up the labor time of the remaining taxi drivers to produce additional goods or services.

There is certainly temporary pain endured by workers who are displaced by the innovation. But it is folly to subsidize work that is no longer in demand.

akulbe 2 ago 1 reply      
If the taxi cab companies can't make it on their own, they should go out of business.

As a self-employed person, I shouldn't expect someone else to bail me out by their efforts. If I can't make it on my own steam, I close up shop.

When this happens, it's unfortunate, and unpleasant, but it's a simple fact of doing business.

Our parents and grandparents would balk and roll in their graves at the thought of this idea that someone else should shoulder the burden of bailing you out.

What ever happened to the idea of (in the case of business hardship or failure) picking yourself up, dusting yourself off and starting OVER??

cloudjacker 2 ago 2 replies      
During periods of extremely high surge pricing, I take regulated taxis instead. At first I think, 'wow I love competition its such a good thing this alternative service was around' but then I try to pay the cab driver, here is the experience in 2016:

- The meter system is broken

- The cab driver presses buttons frantically on a black box before resorting to hitting it in hopes that it responds to external stimuli

- The cab driver is unsure how to process credit cards (or even cash) in the event this black box does not start responding "Uhhh, ahhhhh, this stupid thing"

- If black box does begin responding, then lets advance to the credit card processing part, which also doesn't work

- FUTURISTIC WORLD CITIES in the US have taxis with screens in them so the rider can do the credit card processing on their own, except this doesn't work either "your screen isn't responding..."

brownbat 2 ago 3 replies      
There are two competing narratives about ride-hailing apps success:

1) They are profitable because they are finding loopholes in relevant legislation, or,

2) they are profitable because they are injecting competition and new ideas into a stagnating industry dominated by government-protected monopolies (or oligopolies).

Your opinion of the law will likely depend on which narrative you find more persuasive.

bawana 2 ago 2 replies      
The capitalist model, the concept of 'disruption', Adam Smith's precepts work well when they allow individuals to control their choices, their fate, their destiny. When the transactions exceed human scale - when they become so big, so far reaching, so expensive that the participants are powerless to make changes, then capitalism fails. It fails because the 'losers' suffer even if they try to adapt. They cannot compete because the the 'winning side' is so new with fewer legacy costs. The losers necessarily must die and become assimilated by the winning strategy. And the winners who skirt the regulations will close shop and disappear at the first sign of trouble.

I disagree with the concept of entitlement-taking money from the winner and giving it to the loser. But the winner MUST abide by the same rules as the loser. If the ride sharing industry has the advantage of venture capital startup money and newer (less costly to maintain) infrastructure, then their benefits to drivers should be GREATER than what the taxi industry provides, not less. The 20 cent tax should go entirely to the health care and insurance of the drivers of the ride sharing industry's drivers. Not the welfare recipients we call 'politicians'. That HALF of this tax goes to people who are ALREADY living off of money made by people who actually work really bothers me.

I would really like to see silicon valley startups that offer a new and better forms of local government. I would like to see our political system get improved by disruption. I really believe politicians are vastly overpaid for the non-service they provide.

mungoid 1 ago 1 reply      
I haven't read all 400-some comments, but in the several pages worth i read I haven't seen a single person mention looking out for their own welfare and keeping their future in their own hands. Just because a line of work has been around for decades doesnt mean it is always going to be a safe bet.

I've worked hard for the past 10 years to get to my middling-middle class income. Enough with my wife's to support our family. I could just be content with that and staying right here for the rest of my life, never wanting to do any more with myself, but I know things will change and i want to be prepared for it.

Maybe it's the brain wiring of a software developer, but I refuse to become complacent with where I am at. I already know I wouldn't last another 10 years if I don't regularly adapt, learn new technologies, stay current.

I think in a way, this isn't too much different than the housing bubble. Too many people thinking that everything is just going to keep going its steady pace forever and if anything bad does happen, someone will come to the rescue. I want to position myself so I don't need that rescuing.

appleflaxen 2 ago 0 replies      
When obsolete industries / models / methods become replaced by something better, the best thing that can happen at a macro level is to have them phased out as smoothly as possible. If that smoothness can also be fast, then you will maximize your economic productivity as well.

So this decision is exactly the opposite of what you want... you are going to prolong the inevitable with a market-distorting subsidy.

It would be much better to tax Uber and require the funds go to taxi cab retraining scholarships, so that people who couldn't make a living driving a cab anymore have another option. Better yet: send them to customer service school, so that they can work as an Uber driver if they wish to do so.

a_c 2 ago 2 replies      
If this logic make sense, perhaps AirBnB need to be taxed to support hotel industry, Iphone should pay to nokia and internet should be taxed to landline
lpolovets 2 ago 3 replies      
Obviously the logic is mystifying, but the math behind this is baffling as well:

https://en.wikipedia.org/wiki/Taxicabs_of_the_United_States#... says there are 1825 cabs in Boston. Boston is 10% of Massachusetts' population, so let's say there are 18k cabs in the state.

http://www.reuters.com/article/us-massachusetts-uber-idUSKCN... says 5 cents/ride-hailing app will go toward taxis, and Uber+Lyft do 2.5m rider per month.

So, the tax will produce 2.5m rides/mo * 12 months * 5 cents/ride = $1.5m/year.

That works out to... drum roll... $83 per taxicab per year. How is this supposed to help taxi drivers with anything? Maybe as ride-sharing increases, this becomes $200 or $400 per cab per year, but that's still a small sum. I imagine a few hundred dollars is a small consolation when you lose 50% or 90% of your business.

owly 2 ago 0 replies      
The 5 should be allocated to public transportation to keep late night bus service. This is pure corruption.
trengrj 2 ago 1 reply      
I'm in Sydney Australia and we have a one dollar fee for each Uber to bribe off taxi medallion owners. It feels completely wrong and Uber makes the fee separate on your card I think in protest.
qq66 2 ago 2 replies      
I think that this makes sense in concept if not in implementation. Taxi medallions were sold and traded under a certain set of laws which defined their value. The government decided to change those laws, so "buying out" the medallions at some fair price, paid for by the beneficiaries of the change in law, makes sense.
IkmoIkmo 2 ago 0 replies      
I fully understand one may tax a new successful industry, and then use the proceeds for individual citizens who lose jobs and need some support, training and job programs to help them back on their feet.

But this is ridiculous. The notion an uncompetitive industry, which is fading because ordinary people can and do choose a better product or service, gets money to stay alive with its shittier product, just makes no sense. It doesn't achieve what we want, and there are good alternatives.

MicroBerto 2 ago 1 reply      
Sounds like time for some politicians to get voted out of office. Voters may not care about certain things, but they definitely care about their transportation.
alanz1223 2 ago 0 replies      
this move seems like a desperate attempt to revitalize a dying dream... Taxis are dead, ride sharing killed it. End of story. Typewriter manufacturers werent given subsidies when the computer f*cked them over, were they?
nathan_f77 2 ago 0 replies      
I hope they're also taxing car manufacturers and giving it to horse owners.
ilostmykeys 2 ago 1 reply      
When Uber switches to a fleet of self-driving cars (with lower paid drivers, since they'll just sit and watch 98% of the time), this tax would be absolutely nothing short of taxing innovation to save an inefficient business model. Wow. I thought Capitalism was evil. Now I don't know what to think.
known 2 ago 0 replies      
"If it moves, tax it. If it keeps moving, regulate it. And if it stops moving, subsidize it." --Reagan
ruffrey 2 ago 0 replies      
What prevents taxi drivers from taking Lyft and Uber shifts? Are there some kinds of restrictions on official taxi drivers? Do they need to keep a certain number of hours? Do taxi drivers have benefits?
quickben 2 ago 6 replies      
Well, things should be equal. Why should a taxi service have to perform a regular vehicle inspection and a an uber driver be exempt from it?

Either make the Uber driver go through regular inspections or cancel the regulation for the cabbies.

maverick_iceman 2 ago 1 reply      
By this logic, Facebook should be taxed to give money to Myspace.
tremendo 2 ago 0 replies      
Kodak and Polaroid should have thought of that. Get governments to tax digital photography, cameras and memory cards, in order to save their business and all those jobs they provided (as opposed those newly created, because pfft!). Never mind any benefit to the paying public.
nomercy400 2 ago 0 replies      
Isn't this unfair competition? Government meddeling with free market?

Also, does anybody know if taxi medallions expire?

intopieces 2 ago 1 reply      
The weird thing is they don't even have a solid plan for what to do with the money. Or a plan at all, it seems.

The least they could do is create some kind of MassTrans App that shows a user options for navigating the state (uber/lyft/cabs/transit/greyhound etc). Otherwise it seems like collection for the sake of it.

xname2 2 ago 2 replies      
It should be totally opposite:

1) uber drivers are in lower income than taxis drivers

2) uber economy is new and innovative, while taxi is old and dying

jbmorgado 2 ago 0 replies      
I would understand if the state imposed this tax in order for this money to provide some social security to the drivers of Uber/Lift/etc. But taking this money away from them and giving them to the taxi lobby sounds almost like state imposed robbery.
chillingeffect 2 ago 1 reply      
Serious question here: how come taxi organizations haven't leveraged uber/lyft tech by now? Seems most kinks, tech and laws are figured out now. Why not upgrade using capital since the returns are obvious!?
douche 2 ago 0 replies      
Taxachusetts strikes again. Rather than let a shitty industry that already enjoyed monopoly protections, fail and be replaced with something more efficient, let's prop them up with a misguided subsidy/tax scheme.
ovatsug25 1 ago 0 replies      
Interesting excerpt that makes me support the move:

 The 5-cent fee will be collected through the end of 2021. Then the taxi subsidy will disappear and the 20 cents will be split by localities and the state for five years. The whole fee will go away at the end of 2026.
They know they're fucked. The question remains: will they be able to use this temporary subsidy so they can become more "sovereign individuals" in the future?

losteverything 2 ago 0 replies      
Couldn't a "medallion cab" franchise that licenses tools (technology) to existing city cab drivers be created? So cabbies in Massachusetts can spend the tax money on their own uber via a franchise.

The franchise can be repeated all over for the "old industry" - In every big city.

So a user could choose their "Dominoes" app and they would be getting a legacy cab, not uber-lyft. And the customer would support existing men and women whose jobs are threatened. And like dominoes pizza, behind the scenes is a local owner (or local group of cabbies) that has to follow their own rules.

A cabbie could be hailed with 4 fingers and a thumb up or via two thumb touch. Have cake and eat it too.

punnerud 2 ago 0 replies      
This is genial! This way we don't need the complains from "the old taxies" and can gradually reduce the tax when ride-hailing apps are established. This way we gradually remove the old ones :)
yohoho22 2 ago 1 reply      
> The 5-cent fee will be collected through the end of 2021. Then the taxi subsidy will disappear and the 20 cents will be split by localities and the state for five years. The whole fee will go away at the end of 2026.
halis 2 ago 0 replies      
They don't even pretend to care about every day consumers anymore.
finid 2 ago 0 replies      
The state wants to play Robin Hood.

Hey! Why don't they give the money to me. I can use it.

Next they will want to take money from the likes of Facebook and give it to struggling newspapers.

astrostl 1 ago 0 replies      
Lots of jokes about horse-drawn carriages in here. Would note that, in order to legalize casinos in the STL area, they were required to give money to a dump of a horse track which would have otherwise sped (no pun intended) toward an even earlier grave.
chmaynard 2 ago 0 replies      
> "They've been breaking the law," said Larry Meister of the Independent Taxi Operators Association.

If so, then the duty of government is to enforce the law (presumably with fines or prosecution). Imposing a new tax is another matter. It's amusing that a Republican governor supports a new tax of any kind.

40something 2 ago 0 replies      
Massachusetts is literally run by tyrants. No one cared when the attorney general trampled over the constitution by banning the sale of built to compliant "assault" weapons and now this. The disarmed populace has to just take it. The rulers know best.
sunshiney 2 ago 0 replies      
So Wix should be taxed with the proceeds given to developers it has has taken low-fruit from?
tiatia 2 ago 0 replies      
They should tax cars and give the money to horse carriages
samuraibum 1 ago 0 replies      
This is why people should avoid paying taxes. Taxation is theft and your hard earned money will always be wasted anyway.
chappi42 2 ago 1 reply      
As long as taxi drivers have higher salaries and are better protected this is a good move.

I very much salute the innovations uber enabled. Only there should be a law which guarantees a decent minimum salary for uber riders. No good to allow an income-race-to-the-bottom for drivers.

sevenless 2 ago 0 replies      
All hail the glorious US free market...
z3t4 2 ago 0 replies      
How do the app taxi buisness work? Are drivers employed by the app-maker or are drivers registered for VAT?
kevin_thibedeau 2 ago 0 replies      
If only the buggy whip industry had been so good at bending government to its will.
m1sta_ 2 ago 1 reply      
The tax collected should be forced to go directly to buyback of taxi licenses.
meira 2 ago 0 replies      
As a transition solution, this is genius!
kasajian 2 ago 0 replies      
This is so weird. I'm not sure it's legal.
madgar 2 ago 1 reply      
joshmn 2 ago 0 replies      
I'm all for taxing the rich to help the poor but not like this.
contemporary2u 2 ago 1 reply      
Food for thought:

I am working on AI robot that will ultimately replace human programmers/coders/software engineers. It will make human programmers obsolete.

What will most of you do? Will you go back to school to get a new degree/profession? Open a coffee shop perhaps? or apply for peace corps?

It is coming.. just remember that eventually anyone could be the next "taxi cab" driver with his or her job going away to automation.

Our society will have no choice but to adopt in more ways than one.

xyzzy4 2 ago 1 reply      
As a lifelong resident of Massachusetts, color me unsurprised. This is a tangent but I've also spent some time in India, and a $1.50 Uber ride there would be about $10 in Massachusetts. That has to do with other factors though such as PPP, birth rates and age demographics, immigration policies, number of skilled vs unskilled workers, etc.
mrerrormessage 2 ago 0 replies      
I haven't read all the comments in this thread, but I'm surprised at the outrage over a one-nickel tax. This seems like a fairly simple, straightforward rule. Is anyone going to decide not to Lyft/Uber over 5 cents? I don't think so.

Let's also not forget that these ride hailing services are MASSIVELY subsidizing the cost of rides in order to attract drivers/users. That hurts business for taxis. In my mind, this is a sort of protection that ensures taxis stay in business as another, publicly regulated option. What happens in other areas of Uber/Lyft kill all the taxis and then decide doing business isn't profitable and leave (or all their drivers leave I've subsidies end)? This might seem like a farfetched scenario, but remember that Uber is still sitting in a large cash reserves. What will change when they need to turn a profit every quarter? If they have established a monopoly (even locally), they can charge users as much as they want. If they've established a monopsony on drivers, they can lower wages. In my opinion, subsidizing a long-standing industry from a monopolistic competitor with gobs of money to throw at the market looks like a good move.

thesumofall 2 ago 1 reply      
I believe they have a point. The traditional taxi industry _can_ be rescued. Living in Germany I use myTaxi which is basically providing the same interface as Uber but connects you with professional taxis that are regulated, insured, and being paid proper wages. There is simply no difference in convenience (even the opposite: you can book taxis in advance) and the driving experience itself is typically by far better than any Uber I've ever taken in the US (inexperienced drivers, often old cars,...)

So I would argue that the tax could help achieve a similar scenario in MA if invested properly and if Uber is forced to raise prices by bringing them to uphold professional standards. I was long enough in MA to know that regular taxis have a long way to go but there is hope

Everything I Am Afraid Might Happen If I Ask New Acquaintances to Get Coffee newyorker.com
346 points by misnamed  1 ago   211 comments top 37
ry_ry 20 ago 3 replies      
Hit 30, realised I had probably done as much in my chosen field as I was likely to.

Got sad, considered changing careers and slowly started getting more frantic about seizing my Big Chance. In the process I discovered I'm unspeakably awkward and horrible at networking. Developed a dislike for great networkers being Machiavellian shits.

Didn't get my Big Chance.

Turned 35, had long since stopped counting birthdays, and then one day realised that imperceptibly something had changed.

I wasn't sad anymore. At least not noticably sadder than anybody else if I took the time to look. I'd come to terms with my crippling imposter syndrome, and was free of the quixotic trappings of my equally crippling Dunning Kruger. Read Jon Robson's psychopath test and decided I probably wasn't a sociopath either - I was just a bloke in his mid 30s.

Realised I should stop self diagnosing psychological conditions via the internet.

I have a great family, a good job, I wake up every morning and genuinely look forwards to spending my day doing what I do. I still harbour secret dreams of world domination, but they exist withing a framework of reassuringly tedious goals, like making sure I have enough clean pants and socks for the week.

Learned to work the washing machine. Still don't like inviting people to go for a coffee.

howeyc 23 ago 13 replies      
Ah, the twenties.

How's the saying go?

"In your twenties you worry what people think of you, in your thirties you stop caring, in your forties you realize no one spends any time thinking about you anyway."

squeaky-clean 20 ago 4 replies      
I know this is article is humor, but it's also pretty spot-on for me (maybe a lot of us here). Most of my friends don't seem to understand the extents to which I overthink everything.

"Just ask. The worst that can happen is they say 'No', and by not asking, you're already giving yourself a no."

I get this advice a lot and particularly hate it. Getting a "No" is not the worst that can happen. A meteor hitting the earth is the worst that can happen (or maybe snakes, or crossfit). Worst that's actually happened to me? You go up to say something to someone, stutter a few times, throw up in your mouth a little, and then run away.

yusee 23 ago 5 replies      
Don't solve for people's feelings. If you have good intentions and are useful to others, they will be attracted to you.

Sometimes you will offend or annoy people. These negative experiences are growth opportunities. Apologize. Learn a social lesson. Next time you won't make the same mistake.

balabaster 23 ago 1 reply      
This made me want to reflect on my own list and realized that it's morphed somewhat over the years from:

- What if I invite them to coffee and I think they're super awesome and they think I'm an idiot


- What if I invite them to coffee and that 15 minutes turns into 4 hours and I still want to hang out more because they're awesome (which they will be, because frankly I find everyone fascinating); except now I have this disquieting sense of guilt because none of the other projects I have scheduled to complete in this time are getting completed... or started, or thought about.

- What if I don't invite them to coffee and they are or have some information to share that (combined with my own experience) is the key to my entire existence... or perhaps the future of humanity.

Maybe I let my mind get away with me sometimes...

Sometimes coffee is just a way to blow off 15 minutes with some amusing banter because sometimes it's better to step away from the computer than it is to keep bashing your head against it.

inputcoffee 1 ago 0 replies      
The crossfit line reminds me of this joke:

Crossfit is just like fightclub if the first two rules were the opposite.

Jonoco 18 ago 4 replies      
I asked a girl to coffee once and she ran off. She literally sprinted away. I had to append my list of worst outcomes.
Artoemius 8 ago 1 reply      
For people with social anxiety, this is true to the point of being not funny. This is almost exactly the kind of stuff I always worry about, although I'm usually a bit more negative about it.

(If I slip up and they accidentally see through me, see what a socially inept and boring person I am, they'll hate me forever and also they'll tell everyone in the world about their boring experiences with me, and no one will ever like me anymore.)

sebringj 16 ago 2 replies      
This is really for my fellow single male programmers.

I think programmers tend to identify with being socially awkward as we spend our time focusing on analytical problems usually by ourselves or with similar individuals which emphasizes our brain's wiring to be that way even more so over time, then the less traveled wiring weakens over time (social wiring, your developer buddies don't count).

The crux is we are all social creatures and need to feel accepted by the opposite sex. If we fall short, we will spin ad infinitum worrying about it. Its a deep rooted need within us and cannot be rationalized away, similar to rationalizing you are full when you haven't eaten for a long time. You must take action. Blame Darwin if you must but do something about it.

Because a lot of programmers like me are social pansies when it comes to taking risks, you can try this approach first. Do real weights and wind sprints and eat right to promote testosterone as looking like a dude attracts women. Running long distances way too often will make you a look like a unic twig, lower your testosterone levels, and rob you of muscle. I can't stress enough how the right eating habits and exercise will change everything, most likely within 90 days you'll see a huge difference like night and day, looking lean and mean. Your personality will also have changed, being more confident and bold with the added bonus of girls looking your way. You will become more attractive to females in other words, prepping you for the next step.

Now you are ready to take the issue head on and expose yourself to as many social situations as possible, forcing your social wiring to strengthen, but with the caveat of not giving a shit of individual outcomes, thinking of it more like the sheer act of doing this long enough will cause things to move in the right direction. To truly not give a shit is freeing and what's odd about that is people (especially women) seem to like you much more when you are worry free and have a natural comfort about yourself. Women truly are like buses where the next one will come along simply because there are so many fucking people in the world. Break some eggs and get out there.

godshatter 22 ago 7 replies      
My insecurities all run in the opposite direction. What if someone invites me to coffee? Do I tell them I really don't like coffee? Do I tell them that I really don't want to spend $4+ for a sugar-laden caramel frappe machia-whatever and that I'd rather just get a diet coke from the vending machine? Should I be "that guy" and get a hot chocolate that turns out to be a squirt of chocolate sauce in milk, and feel let down? Should i get a donut and nothing to drink?

Once we're sitting down and talking, I'm fine. I just don't know how to deal with my little "non-conforming weirdnesses" with others.

coldshower 23 ago 1 reply      
Creating fictional outcomes is pretty common (it is for me), but this game has definitely helped overcome that: https://is.gd/GAbuN6
kresimirus 23 ago 2 replies      
17. They will sue me for (sexual) harassment at work
MentatOnMelange 18 ago 1 reply      
Long time lurker on this site and think I finally have something worth conributing. As someone who has struggled with social anxiety, a big ommission from that list is "their reaction will provide emprical proof that insert negative self perception isn't just in my head*.

At which point the hypothetical doomsday scenario extends beyond just the immediate scenario. For whatever reason age does help though. Part of me wonders if modern marketing plays a minor role. The younger you are, the more advertising pushes the need to 'fit in' or impress peers as the primary appeal of a product.

protomyth 23 ago 1 reply      
Just ask about the damn coffee and quit stressing thinking your decision will set in motion events that will change society.

Save the stress for the relatives and in-laws, those will kill you.

blackflame7000 16 ago 0 replies      
The main source of this anxiety comes from the flaw in the phrasing, "There are plenty of fish in the sea."

That simply is not true. People are so unique and diverse that while there are other members of the opposite sex out there, to find someone truly compatible is rare, hence why divorce rates are so high. For me, the anxiety comes when I have found someone who I know is extremely compatible but am afraid I won't attract them.

Not every member of the opposite sex makes me feel the same way certain ones do. Saying there are plenty of fish in the sea is like saying there are plenty of baseballs to hit in batting practice. While true, it elides the fact that it's only real at bats that matter. Once you strike out, you're stuck waiting for your next at bat which may never come.

manarth 21 ago 2 replies      
"The coffee will be bitter/weak/mediocre/disappointing."

Closely followed by:

"I'll be branded a coffee snob for criticising the coffee."

Who am I trying to kid, I clearly am a coffee snob.

StillBored 18 ago 0 replies      
Nah, the worse part is sitting at home, alone, kicking yourself for not asking her out.
893helios 23 ago 0 replies      
Having any of these things happen to you would be great, even the bad stuff. Action is better than nothing.
notadoc 18 ago 0 replies      
Amusing article.

Isn't this anxiety? Couldn't this apply to any situation that is anxiety inducing?

I suspect it's all originating from the same irrational feelings.

intrasight 21 ago 0 replies      
Get married, have kids, have them move off. Now you're just not going to have any of these hangups or concerns. I'm fine asking anyone to coffee.
IncRnd 18 ago 0 replies      
Clearly the author is too nervous, likely from drinking too much coffee.
avindroth 11 ago 0 replies      
Our biological OS is built to worry about every one of these things.

Our bodies are still Windows 98.

jimmywanger 21 ago 3 replies      
I don't understand why this article is so long.

It can be boiled down to "I am afraid of rejection, and I don't want to put myself out there."

Rejections suck. We've all been there, and nobody else out there is responsible for your mental well being.

Why are you afraid of a simple "no" or a response out of an obligation? Maybe the person you're asking out to coffee is a Mormon or has high blood pressure and can't drink caffeine, or he/she doesn't like the taste.

You miss 100% of the shots you don't take. I know that I personally regret everytime I wanted to ask somebody out or even ask for a raise at work, but didn't, cause "that would be weird".

ryandrake 16 ago 1 reply      
Wow, the article reads like a Seinfeld episode, full of neurotic self-absorption. Almost every sentence includes either "me" or "I". If you over-dramatize going to get coffee with someone, making it all about you and your feelings, I'd hate to see how you handle signing a mortgage or picking a career to commit to.
AnimalMuppet 14 ago 0 replies      
You forgot about meeting a lion in the street (Proverbs 26:13).
nxzero 23 ago 1 reply      
Once got a coffee a day with someone new for six months; it was fun, but then again, really didn't have any expectations.
myflash13 16 ago 0 replies      
17. Being stuck with the nagging question afterwards: Was that a date?
jt2190 22 ago 1 reply      
Everyone knows that "Daily Shouts" is The New Yorker's humor column, right?
navyad 9 ago 0 replies      
Awesome read.
paul_milovanov 23 ago 0 replies      
Snakes?!! Velociraptors!!!!
EGreg 23 ago 0 replies      
How about the other consequence: getting addicted to coffee? I am already hooked on some degree of sugar and cellphone use.
huangc10 21 ago 0 replies      
Another cliche quote to add to all this: "Your own worst enemy is...yourself".
cloudjacker 1 ago 0 replies      
Accept the consequences
jflowers45 23 ago 1 reply      
could use a (2015) in the title
amyjess 21 ago 1 reply      
> Theyll think I invited them to coffee because I have a crush on them.

This is so true. I'm an aromantic lesbian, so even though I'm attracted to women, I have no desire to date anyone. But I'm worried people won't know that I'm aromantic and will think I'm hitting on them when I just want to hang out and maybe become friends.

A few years ago, I pretty much botched any chance I had to be friends with one of my female coworkers, who was a really cool person I'd have liked to hang out with more. Unfortunately, I was afraid to talk to her, ask her to hang out after work, etc., because I was terrified she'd think I have a crush on her and was hitting on her. It doesn't help that I'm transgender and I knew her before my transition, so I was always afraid she'd think of me as just being another guy trying to get into her pants. Oh, and it didn't help that I was jealous of her, because she looks pretty much exactly how I want to look but will never be able to (dysphoria is a powerful thing).

As such, I was terribly awkward around her, and every time I wanted to hang out with her or anything, I either decided against it or was so nervous and awkward that anyone around me could tell.

I'm a little better at it now, thankfully.

And, yes, I'm the kind of awkward person who over-analyzes every conversation I've ever had with anyone.

4ndrewl 22 ago 4 replies      
Must be a US English <-> British English thing, but I thought this was about asking a new acquaintance to buy some coffee for me.

"Everything I am afraid might happen if I asked a new acquaintance to meet for coffee."

ZoF 1 ago 3 replies      
A year of Rust and DNS bluejekyll.github.io
362 points by bluejekyll  1 ago   101 comments top 18
SomeCallMeTim 1 ago 11 replies      
Love the idea of reimplementing DNS in Rust. Would love to see more efforts like this so that we have secure-by-design language implementation of core security services.

But BIND isn't just failing because "it's written in C", it's failing because it's written in terrible C. That said, "terrible C" is probably most every C routine written by someone with less than 10 years of solid low level experience, so "writing good C code" is not very scalable. There are active, solid projects with very few security exploits that are written entirely in C. Nginx comes to mind.

The article makes a brief reference to DJBDNS, which is written in C, but has suffered zero security exploits [1], and is extremely performant. And it is being used in production, so presumably if it had any exploits they would have been exposed by now.

But...DJB's code is (usually?) released under a rather unfriendly (though mostly open) license, and DJBDNS hasn't been updated in some time, so a more modern project isn't a bad idea. And Rust developers can probably write solid code with only a few years of programming experience, which makes it easier to extend without adding security holes on a weekly basis (coughBINDcoughOPENSSLcough)...

[1] http://cr.yp.to/djbdns/guarantee.html

jcranmer 1 ago 1 reply      
I don't know if you're aware of this or not, but when implementing standards with dozens of haphazard extensions, often the best way to keep track of all of the extensions is via the IANA assignments (see http://www.iana.org/protocols). For DNS, the main page seems to be here: http://www.iana.org/assignments/dns-parameters/dns-parameter...
kenOfYugen 1 ago 1 reply      
An inspirational side-project.

> I want to get a DNS fuzzer running against it to really pound on it, and then get some benchmark and comparison tests against other servers.

Even if the benchmarks turn out to be not in favor of Trust-DNS, the added safety overhead (if any) should be worth it.

> (Ill try to post more regularly on progress)

Please do. I thoroughly enjoyed the content as well as the writing style!

dochtman 1 ago 1 reply      
As a ring [1] contributor, would be curious to hear if it could satisfy your crypto requirements. Using it could maybe help with your "(oh-my-dear-god I can not unsee what I saw in there, the C, not Rust)" experience.

[1] https://github.com/briansmith/ring

viraptor 1 ago 1 reply      
bluejekyll, have you thought about splitting the server into separate crates? (for example the parsing, the dnssec validation, the forwarding/resolving part, etc.) I'm planning to write a purely forwarding/reporting DNS server with white/black-lists and it looks like I could reuse 99% of code from you. But unless I'm missing something, the whole of the server is in one crate/repo. I did see that it's split into one lib and one bin, but haven't looked into how reusable that lib is yet.
arunmu 1 ago 1 reply      
Would love to repeat this sometime using pure modern C++.
voltagex_ 1 ago 1 reply      
Great article.

>Implementing rfc1035 was deceivingly easy

This stuck out to me - RFC1035 is the first RFC I've read all the way through and had a shot at implementing. Then you realise it's RFCs all the way down and there's no easy way to navigate through all the overridden / deprecated parts.

DigitalJack 1 ago 2 replies      
Just curious. Is this the normal way:

 let recursion_available = (0b1000_0000 & r_z_ad_cd_rcod) == 0b1000_0000
That just seems redundant. If you declared the var as a Boolean, would the == 0bxxxx part have been necessary?

signa11 1 ago 3 replies      
with erlang you can parse the dns-header like so:


| %% extract dns-header fields from a raw packet.

| parse_dns_header(raw_packet) ->

| <<

| ID:16,

| QR:1, OPCODE:4, AA:1, TC:1, RD:1, RA:1, Z:3, RCODE:4,




| Tail/binary

| >> = raw_packet,


| {#dns_header_record {

| id = ID,

| qr = QR, opcode = OPCODE, aa = AA, tc = TC, rd = RD, ra = RA, z = Z, rcode = RCODE,

| qdcount = QDCOUNT,

| ancount = ANCOUNT,

| nscount = NSCOUNT,

| arcount = ARCOUNT

| }, Tail}.



almost verbatim 'transliteration' of sec:4.1.1 of 1035 :)

nialv7 1 ago 3 replies      
Nit: Rust doesn't prevent memory leak.
jedisct1 18 ago 0 replies      
Another DNS-related project in Rust: EdgeDNS https://github.com/jedisct1/edgedns
pjmlp 1 ago 1 reply      
Any benchmarks against the MirageOS TCP/IP stack?

It would be interesting to see how the way Rust manages memory and its current optimizer match against OCaml in a production TCP/IP stack.

nickpsecurity 1 ago 2 replies      
With Ironsides, that makes at least two of you using safest languages you can find to try to improve DNS. Good goal and tool to pick. :)


"Take a look at the full list sometime, I think roughly 50% of those could have been avoided by using (safe) Rust."

This is true for most vulnerabilities I see in C-related apps. We also know there's techniques to prevent that with acceptable performance. So, outside hobbyist or non-critical stuff, I tell people to use a different language to get the baseline of quality/security up. DNS is kind of example that makes it more true.

Now, I haven't learned Rust yet, so I won't be able to fully appreciate the article or spot coding ideas until I do. What I did spot was that you slogged through all the RFC's implementing and testing what you thought should be in the DNS system. Aside from a DNS server/client, I think one of most valuable things you could do in this project is create a single specification of various things in DNS that cites parts of RFC's or advice from real-world implementations to justify each part. Kind of a walkthrough for other people creating DNS's that gives them high-level view and/or drill-down into details of something.

"While in the pit of dispair" "Then I climbed saviors peak"

Haha. Good job getting through all the work as most people quit that I can tell.

"Its currently not used in production (as far as I know). Ive put a lot of work into validating correctness of what is going on"

It's good you have gone through the specs and have plenty of features. You did hit on a risk area that might need to be in this section: difference between the specs and real-world implementations that you have to work with. The differences could cause your server to fail. If those exist in DNS, too, there might already be write-ups on common ones out there. Biggest way to find them, though, is running combo's of popular clients and servers against each other in testing. So, that's a possible consideration for the future.

bsder 1 ago 1 reply      
The only technical comment I have is that it continues to be amazing to me that destructuring binary data is so stupidly verbose in so many languages. I think C and Erlang are the only two languages that got this right.
masgui 1 ago 0 replies      
colemickens 1 ago 0 replies      
Great post, great library. I used it in a CloudFlare dynamic dns client I wrote to play around with Rust. I'm still very new to Rust so these sorts of post are really helpful. Thank you bluejekyll!
killbrad 1 ago 2 replies      
Isn't trying to write a secure implementation of DNS as it is today, like adding a chain to your door? Mostly for show?
The Internet of Poorly Working Things mondaynote.com
306 points by kawera  1 ago   300 comments top 29
binwiederhier 1 ago 8 replies      
Thank you for that article! I have been feeling like that for many months, and whenever I point it out to co-workers or friends, they mention the Apple-universe as the ultimate achievement of interconnectivity. And in a one-vendor universe, the world may be okay right now. Apple did a fine job there.

However, interconnectivity between devices (and software) of different vendors seems to get worse and worse; standards seem to have become irrelevant. Time to market is the only thing that matters and long-term customer satisfaction and durability seem to be of no importance any more. They just don't care about integration with other vendors any more.

When I saw Minority Report (2002) a few years ago, I thought dragging windows and applications across devices with a gesture would be a possibility in the not-too-distant future. Now, in 2016, this not-too-hard-to-develop feature seems almost impossible to imagine. Sharing content between devices is utterly painful or even impossible: copying large files between computers in the same Wifi without going through the Internet; playing a video from your Android phone on a Samsung TV; moving application state from your laptop to your desktop PC when you leave work; playing music from your Android phone in a brand new Audi via Bluetooth ... All of these things are absolutely achievable if vendors worked together or standards were to be developed/followed. Right now, though, it just looks like technology fragmentation is getting worse every day.

m_mueller 1 ago 18 replies      
IMO there is a very annoying trend in household appliances towards worse and worse UI.

Basically, it started with light switches already decades ago, at least in Europe. In the past, you had switches that themselves indicated what state they are in, so if you cluster them on a board it's incredibly easy to find the one switched on at the moment [1].

Then, some moron came up with switches that don't show anything anymore [2].

Nowadays you're lucky if you get switches on anything at all. This [3] is how a standard stove looks like in new Swiss apartments nowadays. Good luck explaining this to your grandma. You idiots, it has one job, getting more or less hot!

Recently I took a residential elevator that just had an empty touch field when you came in. No indication of what you could do whatsoever. This immediately gave me anxiety, and I'm just 31 goddamnit. Anyways, what happened was that once the elevator door closed it gave me a selection of floors to go. [4] facepalm

Please, for the love of what's holy, stop improving what doesn't need improvement! In german we have a word for this: "Verschlimmbessern". (a combination of verschlimmern='making it worse' and bessern=improving)

[1] https://adventurelightingblog.files.wordpress.com/2010/02/li...

[2] http://www.schulungshandbuch.de/WebRoot/Store22/Shops/627847...

[3] http://media3.siemens-home.com/Product_Shots/915x515/MCSA006...

[4] https://www.google.co.jp/search?tbm=isch&q=schindler+touch&t...

wtbob 1 ago 3 replies      
> In less than two years, the CPU inside the TV quickly becomes obsolete and cant be upgraded while the display itself easily lasts a decade, and software updates are persphinctery, if they happen at all.

That's odd the word 'persphinctery' seems to appear only in Monday Note articles. Is this some sort of trap street[1] for medium-form articles?

[1] https://en.wikipedia.org/wiki/Trap_street

GuiA 1 ago 4 replies      
Any smartphone from the past few years + an internet connection lets people file their taxes, watch videos, communicate by text/audio/video with pretty much anyone in the world, read online encyclopedia, order pretty much any physical good to their door, listen to music, translate languages, read books, take high resolution videos, and so much more.

How do you even beat that? And what do you offer beyond that?

It feels like we're reaching the flat part of the logarithmic progress curve (a much more appropriate curve for progress than the hockey stick curve) when it comes to what personal computing is going to bring to the daily lives of consumers.

Of course, there are still many areas not explored by computing. Computer aided medical procedures and diagnoses, monitoring and upkeep of crops, and so many more fields will grow in the years and decades to come. And improvements to personal transportation, through i.e. self driving cars, is arguably a consumer technology.

But the whole IoT movement is just a parody of itself. The vast majority of people do not want internet connected water cups or juicers or microwave ovens; and those who do soon get frustrated by the real world logistics that come with these things (higher costs, more frequent failures, lack of interoperability, etc.). I had Philips Hue bulbs for a while, and the girlfriend I lived with at the time hated them with a passion - for understandable reasons. When it comes to turning lights on and off, you can't beat a light switch, and the same logic applies to every single item we interact with daily. For instance, the Nest we had in our apartment would randomly start on and off, or suddenly stop being visible to the app, etc.

wangchow 1 ago 0 replies      
A lot has to do with the fact that most of these devices are novelties. Save for smart thermostats (can save money on heating bill), or simple efficiency-boosting products, life is generally easier without the added complexity of an interconnected-everything. A fridge is a fridge and its proven to work well for over 100 years. A toaster makes toast. The dishwasher washes dishes.

Once something more useful comes around companies will invest more resources but I mean who wants a smart toaster? You pop some toast in there and let it do it's thing. Simple.

We need some major breakthroughs in, for example, 3d printed gourmet meals or something.

DiabloD3 1 ago 3 replies      
> In less than two years, the CPU inside the TV quickly becomes obsolete and cant be upgraded while the display itself easily lasts a decade, and software updates are persphinctery, if they happen at all.

I refuse to buy a smart TV, and when my TV finally dies, I will probably be forced to buy one, and the first thing I will do is either disable, or just refuse to setup and use, the smart part.

I will then plug a Chromecast in. That cost me $35. That actually works with the services I pay for correctly.

Even those Android TV-based smart TVs are useless, because Google does not control them and cannot force the OEM to push Android updates.... although, Android TV does let you Chromecast to it, which is probably what I would use it for.

Although, if I wanted Android TV, I'd buy an Nvidia Shield TV and use that instead, since it actually has a reasonable amount of horsepower, supports H265 Main10 and Rec2020 colorspace (in the hardware, AndroidTV support of it itself is upcoming), thus, true 4k support (not merely 2160p support using 8-bit Rec709/sRGB, which a lot of so called 4k devices and TVs are; for a historical perspective, see all the TVs that can only do 720p and 1080i but not 1080p, thus are not actually HDTV/Bluray compatible at all, this is the same thing al over again).

Technology advances too quickly for a TV to ever be smart. I'd pay more for a dumb TV that has two more HDMI ports instead, ripping out any smart TV SoC, and ripping out the (extremely useless) cable tuner.

Side note: the cable tuner is useless on cable, due to all cable companies moving to encrypting all channels, or moving to IPTV platforms entirely, thus always requiring a box (CableCard is a dead standard and was a mistake, similarly to how smart TV is a mistake and should be just as dead); the cable tuner is always useless on satellite; and if you're trying to do OTA, many TVs, even ones produced today, do not have sufficient sensitivity to dial into channels for many reasons (distance, obscured line of sight, reflections), and even with large enough antennas, you'll get better performance (and sometimes, the ONLY performance) out of a dedicated OTA box.

As of which OTA box, if you need signal performance for extreme OTA situations, Channel Master's tuners, as a set top box, kind of suck, but often are the only ones that can coherently decode a signal.

Mister_Snuggles 1 ago 2 replies      
I've got a bunch of WiFi cameras that aren't able to access the internet[0], so really it's an "Intranet of Things". But they sometimes drop their connection, which is expected because it's WiFi, but a lot of the time they won't reconnect properly.

I figure that if basic infrastructure stuff, like automatically reconnecting to WiFi, doesn't even work, what chance do we have of harder things like security working properly?

[0] https://news.ycombinator.com/item?id=11933851

meddlepal 1 ago 1 reply      
Worked on industrial IoT for five years from devices to backends. It's all garbage. Nothing works, everything is wrapped in marketing double speak, security is a Shit show and UX is virtually non-existent.
flyinghamster 1 ago 1 reply      
Vendor lock-in gets short shrift in this article. Let's not forget Philips attempting to lock out competitor's bulbs from their Hue bridges.
sangnoir 1 ago 1 reply      
I just had a chuckle: the Chrome tab I have open now (for this HN story) reads "The Internet of Poo" with the rest of the sentence faded out & cropped. How fitting
walterbell 1 ago 8 replies      
> A look at the so-called Smart TV reinforces the observation about CE culture. In less than two years, the CPU inside the TV quickly becomes obsolete and cant be upgraded while the display itself easily lasts a decade, and software updates are persphinctery, if they happen at all.

Evocative term ("persphinctery"), what does it mean? A web search mostly references mondaynote.

Animats 1 ago 1 reply      
There are two basic problems. 1) getting the devices to find each other and communicate in some reasonably secure way, and 2) controlling them in some user-friendly manner.

1) is still a mess. Things that are hooked to line power ought to talk over the power line. There are lots of standards for that, from the old X10 (1980s, low bandwidth, poor noise immunity, no security), bidirectional X10, Echelon (1990s, low bandwidth, very good noise immunity, some security), HomePlug (2000s, high bandwidth, some security), plus some proprietary systems. X10 refuses to die, and HomePlug's bandwidth is overkill for lighting. Echelon mostly gave up on the home and went on to become the standard for subway and rail automation (signs, lighting, HVAC, doors, etc.) because of the good noise immunity. They're working on a new approach to lighting, where LED lights run on 48VDC. This is a bit radical for home automation.

If you have any of these, it's useful to have a whole-house RF filter where power enters the house or apartment to isolate your network segment from everybody else on the same pole transformer. These are cheap ($6 or so) but have to be installed by an electrician. This is a big obstacle to power line networking.

Then there are the RF-based networks. WiFi, Zigbee, etc. These have range limitations and may not work through walls. Despite all the headaches of RF networking, most of the IoT vendors are going that way because they get to dump the range problem on the user.

supergeek133 23 ago 0 replies      
Working in consumer IoT on a regular basis.. here is the main problem with adoption:Every device has it's own app!

Let's say eventually I want to use Echo, or SmartThings, etc. I can buy 20 devices but half of them have their own app that requires registration before I can connect it to my Echo or SmartThings hub.

The reason people hold up Apple as a potential solution is you are seeing hints of them starting to steal registration tasks of even cloud connected devices.

So you might ask "Why" does each device have it's own app? Because all of these companies want to "own" the customer and the associated data.

The funny part to me, is I know damn well some of them don't want any part of maintaining an application and the team associated with it and the infrastructure.

If someone came up with a platform that was easy to integrate, managed an app, and just shared that data for free or a nominal fee, I believe they would win.

ss9445 18 ago 0 replies      
It's true that progress is slow, partly because not enough people are tinkering and innovating in this space. We're trying to fix that by getting affordable starter kits into the hands of more developers.We saw a big gap in the market in terms of innovation at the edge especially using cellular technologies. Technologies like CAT-M and Narrowband should make this gap even smaller.
gabomagno03 22 ago 0 replies      
Even though I compleatly agree with everything being said in the article ,it highly understimate how hard it actually is to build a semi-complex IoT product. Our startup realized this after 2.5 years in the making of a Smart Things competitor. For example, to debug something you need to check a larger set of things (cliente, backend, hardware, embedd software, network, etc). It will all get better but as of today it really is hard to achieve.
sickbeard 1 ago 0 replies      
The internet of low margins and devices of dubious functionality. I'm truly scared that the next dotcom crash will come from iot not panning out for companies jumping into it without any forethought
Scea91 1 ago 1 reply      
Regarding the mentioned Moore's law, do we really have 1000 times more computational power on consumer devices than we had in 1996?
dredmorbius 1 ago 1 reply      
JLG raises some excellent points and points to some excellent critics, including the Internet of Shit twitter account.

Some elements to highlight:

We had this problem in consumer electric goods, starting slightly over a century ago. Poorly-made devices could electrocute users, burn down homes (or offices or factories), would fail to work as advertised, fail to work, or otherwise disappoint. The result was that, at the instigation of insurance companies, an independent testing and certification service was created, Underwriters Laboratories. UL are actually looking at entering the IoT/IoS morass: http://www.csmonitor.com/World/Passcode/2016/0405/Can-testin...

More complex products make for more complex assessments. One of my most useful and oldest household purchases is also one of my oldest: a cast-iron frying pan bought 30 years ago. It functions as it did when I first bought it, I've used it daily for decades, and its function was immediately evident. Contrast the tablet and keyboard on which I'm typing this, neither of which have lived up to expectations (nor the manufacturers to their warranty obligations -- Samsung and Logitec respectively). With hardware, control interfaces, communications, server dependencies, bugs, security vulnerabilities, and more, IoT devices are vastly more complex than what they replace.

Poor margins make for poor products. This seems far less well understood than it ought, particularly among the HN crowd. An undercapitalised company, or one operating on a burn-rate and prayer, or some overseas firm you've never heard of, trying to crack your local market, may well choose to skimp on quality options. Vendors who care nothing other than order fulfillment and logistics (Amazon, WalMart, Best Buy) have no vested interest in product quality. Quality itself is a difficult metric to assess, particularly lifetime quality.

Product / vendor lock-in is a consequence of this. Rather than make money on initial purchase, various forms of subscription-based services are offered, or interconnectivity is provided ... as an ad-on, additional-cost, feature.

Unintended consequences are a real bitch. This is an area of market failure I've been exploring which seems highly underconsidered in contemporary economics. While Akerloff's "Market for Lemons" gave us a fuller awareness of information asymmetries, I'm not aware of a general treatment for a simple inability to know the salient future outcomes of current actions. Consider Thomas Midgley, Jr.'s contributions of chlorofluorocarbons and tetraethyl lead, used in leaded gasoline. He's done more than any other engineer to put all of humanity at risk. And while lead was a known pollutant by the 1920s, the effects of CFCs on the ozone layer wasn't discovered for another 50 years. Long-lingering systemic effects are particularly pernicious.

"Smart" products only offer so much additional capability. The ability of added logic or communications to a device to increase value is limited by the mode through which that object operates. Take Google's recently announced energy savings AI applications. The multi-millionfold increase in computer processing capabilities, combined with a decade's hard work at improving AI ... can deliver a ~15% energy efficiency improvement. In a Moore's Law scenario, this is about a four-month advance in processor capabilities, and given interactions of the Jevons Paradox, the end result is likely to be increasing energy utilisation, not decreasing it. Similar results are seen for automotive and aircraft control systems. Net increases in efficiency from millionfold processer capability improvements are, at best, a factor-of-two doubling, and often far less.

There are examples of marked improvements, with the multiple factors compounding in shipping markets a good example: containerisation, standardisation, improved dockside handling, intermodal rail and trucking, invoice and manifest controls, etc., have increased the efficiencies (and decreased the costs) of shipping markedly. Of course, the result is that far more goods are shipped, and labour rates in advanced countries have fallen tremendously with falling negotiating advantage.

Complexity increases failure modes. The more interconnections and mutual dependencies a thing has, the more ways it can fail. Any individual or combination of possible interactions needs be considered. Multiple systems with complex interactions compound this. The alternative is fully modularised and independent systems.

E.g., rather than a fully integrated "IoT" refrigerator, specific sensing, control, and communications modules might be provided to monitor power, temperature, contents, and communicate these.

Taking my tablet example -- apparently cases designed to specific device dimensions make the interchangeability of what ought be completely separate systems complex.

davidgerard 1 ago 0 replies      
Whenever you see "Internet of Things", think "Unfixable Heartbleed Everywhere Forever".
DyslexicAtheist 1 ago 1 reply      
Technology is no longer driven (=funded) by innovation but by the ability to produce "recurring revenue". Selling software and some support around it is no longer a viable business model. It's unicorns or nothing.

Even Microsoft has seen that coming (and responded by open sourcing gradually ... from C#, ... MS Azure, ... the current Windows-10 beta that runs a Linux kernel, to PowerShell that runs on Linux). Has hell frozen over? No. But we think the only way to make money is with reselling user-data. And the VC industry reflects that. How many tech businesses that made it big today aren't built on central data harvesting of their users in exchange for "free" stuff?

We've pushed all our computing to the cloud and think it's the panacea to everything. Do we really expect to apply the same business model and technology principles from virtual world to our physical devices and get away with it? Reminds me of the old phrase, "When your only tool is a hammer, everything looks like a nail".

Problem isn't that Security is so much worse in IoT than in your typical web application stack. The problem is that it isn't any better than web-security! We have XSS and SQL injection in IoT, crypto built on shitty javascript, we have MiTM attacks, lack of authentication, ... worse, we can re-use the same exploits (shellshock, heartbleed, ...), and nearly identical attack-vectors! In the age of Shodan and MASSCAN we won't get away with that [1]. (ProTip: (from Gartner, I think): you can send payloads with MASSCAN to a gazillion connected devices by 2020 ;-))

Take a look at the issues of trust on the web! How many signing countries can we trust in our certificate chain? How confident are we that our HTTPS connection is safe (cloudflare is known to MiTM[0] half the internet, so the silly green browser padlock doesn't mean anything) Yet we expect to use the same flawed trust-model with IoT (where bugs hit us in our physical face) as we use on the web. We want to protect all this with Tor browsers, even more centralization and vendor-lock-in or even creating dystopian proposals like the ones from many EU based countries (hello Germany) that propose that the data will never leave the EU.

The vendors response?

I, and many others who stepped forward to report critical bugs to IoT companies are ignored, accused, blocked. I started IoT Security[5], on LinkedIn some years ago. LinkedIn is a joke of a platform I agree, but it also allows us to put all this shit right into the face of these crappy vendors. It's the perfect melting pot for marketeers and engineers ;-))

Privacy and Security have gotten worse over the years. not just for IoT but on the web in general. We point the finger to the "Internet of Shit", but do we expect if we constantly use the wrong tools for the job?

We're looking to the industry to solve this for us. An industry which hasn't figured out yet how to monetize their technology without selling our private data. VC's are as much part of the problem as tech companies. What could go wrong? It's like asking an addict to seek cure by discussing it with their dealer.

There is too much technical debt in terms of privacy & security that all these gadgets are either going to kill us, and as consequence will be regulated like anything else that has killed regularly in the past. If not now then as soon as the first connected car is weaponized, or the first smart-home kills someone.

Nation states and their armies have become dependent on the Internet to keep us safe. They're doing a good job reminding us that the cyber-threat is real. They're right about the threat. What we're wrong about though is believing that global data harvesting by shady intelligence services will keep us safe. But they have their own agenda[4].

The way forward? Certainly not more centralization or cloud.

PS: If you're also sick of all this shit, please do find me ;) there is lot's to talk and little time.

[0] https://scotthelme.co.uk/tls-conundrum-and-leaving-cloudflar...

[1] https://media.defcon.org/DEF%20CON%2024/DEF%20CON%2024%20pre...

[2] https://twitter.com/ValbonneConsult/status/74958341227350835...

[3] https://twitter.com/ValbonneConsult/status/74962796346176307...

[4] https://blog.valbonne-consulting.com/2016/08/06/smartcitiess...

ebbv 1 ago 3 replies      
All that build up and it ends up basically an ad for Amazon Echo? Disappointing. Amazon Echo is of very borderline usefulness at best. On its own it does almost nothing that the digital assistant on your phone can't already do, and a lot that it can't just because your phone is so already entrenched in your life.
bingobob 22 ago 0 replies      
waiting to see what comes out of Google IoT project like Brillo, Weave it's been kicking around a Google for a while now seems long before the 2015 I/O announcement. maybe Fuchsia has something to do with it who knows.
Aelinsaar 1 ago 1 reply      
Nothing about the impending IoT seems planned, standardized, and remotely designed with the consumer's best interest in mind. That being said, consumers for their part seem almost eager to throw their money away for truly dubious improvements. What isn't a bad idea from the get go is often a good idea ruined with careless implementation.
jwatte 1 ago 0 replies      
And then the cloud service for your door lock, light bulb or security system gets shuttered.


seiferteric 1 ago 0 replies      
My neato botvac seems to forget its wifi settings every couple weeks :(
dredmorbius 1 ago 4 replies      
This isn't the first time gratuitous "technologisation of stuff" has been proposed, though it seems to be proceeding rather apace.

From 1922, "Radio All the Things!": http://i.imgur.com/TSJnicdl.jpg

(Via: http://www.darkroastedblend.com/2015/01/videophones-from-fut... and origionally: http://wi.mobilities.ca/grant-wythoff-aerophone-telephot-hyp...)

Or the original book if you'd prefer: Hugo Gernsback, Radio for All (1922): https://archive.org/details/radioforall00gerniala

I think I know what a radio phone might do. Radio clocks even exist (though a networked NTP timekeeper is more useful). I'm not sure I'd care to fly in a radio-controlled airplane. And I'm utterly perplexed at what a radio heater might be -- unless it's a microwave space heater, or an early instance of Nest.

api 1 ago 2 replies      
I just call it IOJ: Internet of Junk.
chaostheory 1 ago 0 replies      
This post would be more accurate two years ago. Today IOT is usable with much less work and headaches. Sure there are still bugs, but it's a big improvement from a $40 brick. Also platforms like IFTTT, Alexa, and Homekit make it more practical as well.
July was the hottest month ever recorded, according to Nasa nytimes.com
268 points by MarkEthan  4 ago   307 comments top 26
harryh 2 ago 24 replies      
Personally I think that the idea that these trends can ever be revered by some sort of world wide agreement to lower consumption is a pipe dream. People want to eat meat and drive cars and fly on airplanes. As long as they're rich enough to do so you'll never stop them.

The only solution to this problem is technology. Some combination of green energy sources plus probably very large scale terraforming.

codecamper 2 ago 3 replies      
Solar panels now sell for about 45c a watt out of China. Cost to create is around 36c / watt.

This allows deployments such as in Chile & Dubai where the cost/kwh is 3 cents & below.

This now cheaper than coal, without subsidy.

Remove the subsidies that oil, gas & coal receive, and you will witness a new day.

Only problem for solar right now is over supply caused by a sudden drop in demand. China reduced its subsidy a bit. Also the US extended subsidy for 5 years, which is great, but it has removed the urgency to complete utility scale solar plants before subsidy cutoff, causing the utilities to sit it out & wait.

Better act soon! (250 years to clear CO2 from atmosphere, feedback loops only now being discovered.)

bjourne 2 ago 1 reply      
Anyone else who feel that the climate-related news are getting very scary? If for every month for the last year, there was a new sprinter breaking the last months world Record in 100 meters you'd know something was fishy. Like some kind of incredibly potent anabolic steroid runners were ingesting. With new and improved versions of that steroid being released every few months...
sevenless 3 ago 6 replies      
These maps are based on emission trends continuing, assuming no political action will be taken. But wouldn't a hotter world tend to exacerbate emission trends, with human needs for AC and desalination rising, plus forest fires? Moreover, there might be some tipping point for large scale methane release. The 'tail risk' to global warming has appreciable density a long way out.
triangleman 1 ago 0 replies      
El Nino + urban heat island effect. According to satellites, July 2016 is the 2nd hottest July since the 70's when satellite records began. The hottest is July 1998, also El Nino.

Take a look at weather stations that have been situated in the same place for 100 or more years, far from cities:http://www.john-daly.com/stations/stations.htm

mfer 10 ago 0 replies      
Not to negate the issue at hand, which is important, there there's an interesting note on perception here.

"ever recorded" is used but how long have we been recording? The last study I checked on that made a claim like this was about 100 years. "ever recorded" sounds more sensational.

In any case, how we treat our planet... our ecosystem... is important. Seeing it go downhill is saddening and motivating. Glad to see things pointing out the change, its direction, and possible impact.

throwanem 2 ago 3 replies      
It never ceases to bewilder me that nuclear power isn't taken seriously as a replacement for fossil-burning baseload generation.
davidf18 1 ago 0 replies      
There is much we can do in the US to decrease air pollution within a very short time. These reductions also contribute to lowering asthmatic episodes and hospitalizations and various forms of heart disease in the elderly, especially women.

1. As of a few years ago, 10% of the 450 coal powered electric power plants produced half the air pollution by coal powered plants (according to an Environmental Pollution Agency report). Take the top 45 polluters off line. The Obama administration is already to shut down some of these plants.

2. Many buildings in the Northeast and Midwest burn old #6 and #4 fuel oils for heat during the winter which are very polluting. New York City, where I live, has now banned the dirtier #6 oils, but politics has intervened and significantly delayed the banning of #4 oil. But other cities should ban the burning of these dirtier fuel oils.

The shutting down of the top 45 polluting coal powered electric plants and the banning of #6 and #4 fuel oils for heating buildings would not be hard to do and would make a significant difference to reducing air pollution.

AdmiralAsshat 42 ago 0 replies      
Two weeks from now: "August was hottest month ever recorded, according to NASA."
arethuza 3 ago 0 replies      
Found this link with similar maps for Europe:


NB from 2012.

Interesting that it actually predicts that the UK will have a distinct benefit from warming when it comes to agricultural productivity.

aangjie 3 ago 0 replies      
This graphic could've used a Brett victor style tangle.js based simulation model. Then we could play around with it assuming future emission rates going down or up etc.. http://worrydream.com/Tangle/
cconcepts 3 ago 2 replies      
I initially thought this was a NYTimes self promotion piece. As in "Think NYTimes is hot right now? Just Wait".
Karellen 3 ago 0 replies      
Phil Plait of Bad Astronomy also has a good (if now depressingly repetetive) take on it[0].

[0] http://www.slate.com/blogs/bad_astronomy/2016/08/16/july_201...

artursapek 3 ago 3 replies      
I didn't know if it was just because I now have a 2 year old, but New York summers have definitely seemed harder to deal with each year I've lived here.

Is there a commonly accepted minimum temperature difference threshold that is perceivable by humans? E.g., if summer in 2016 was hotter than 2015 by X, I will notice?

elcct 40 ago 0 replies      
I planted palm trees and bananas in my London garden to reflect that :)
novalis78 2 ago 0 replies      
and here we are, still without any thorium reactor anywhere on the planet in operation.
return0 1 ago 0 replies      
As someone who does not tolerate heat well, i can confirm. And it isn't over yet.
kelvin0 1 ago 0 replies      
Well clearly it has nothing to do with man-made pollution, it's nature taking it's course and these rampant wildfires. Oh wait, what causes the wildfires you ask? :-)
tombert 3 ago 4 replies      
The part I find especially frustrating is that, on average, a New Yorker has a lower carbon footprint than the typical American, probably due to the heavy use of the train.

So even when I do the (I think) eco-friendly thing by not having a car and using public transit, I'm still being punished for what the rest of the US (and the rest of the world) does.

misja111 3 ago 4 replies      
So 14 of the 15 hottest years ever have taken place since 2000. I'm wondering, if this is the result of global heating due to emissions, why do we see so many new record highs only now?The industrial revolution has been going on already for over 150 years. Why were there so few records broken in, say, 1980 - 1996, and so many in 2000 - 2016?
chukye 2 ago 4 replies      
OK, but... any ideas how can we solve that?
bnolsen 2 ago 3 replies      
if almost half you input data is made up" and not raw observations then you can falsely come to this conclusion. The real data shows spiking in the mid 1930s far above current levels.
gadders 1 ago 0 replies      
It looks like the actual increase was minimal and within the margin of error:

NASA and the National Oceanic and Atmospheric Administration (NOAA) announced this week that according to their calculations, July 2015 was the hottest month since instrumental records began in 1880. NOAA says that the record was set by eight one-hundredths of a degree Celsius over that set in July 1998. NASA calculates that July 2015 beat what they assert was the previous warmest month (July 2011) by two one-hundredths of a degree.But government spokespeople rarely mention the inconvenient fact that these records are being set by less than the uncertainty in the statistics. NOAA claims an uncertainty of 14 one-hundredths of a degree in its temperature averages, or near twice the amount by which they say the record was set. NASA says that their data is typically accurate to one tenth of a degree, five times the amount by which their new record was set.


factorialboy 3 ago 4 replies      
pedro2 3 ago 0 replies      
staticelf 3 ago 2 replies      
I am lucky I live in the northen parts of Europe. I do not wish to live in America in ~50-100 years from now since there is so much guns and other weapons easily available.

People who care about their childrens future should start planning now. I think when the shit hits the fan it will hit hard and fast.

How to be mediocre and be happy with yourself bbc.co.uk
331 points by sjcsjc  1 ago   199 comments top 30
CM30 1 ago 8 replies      
I think the key point here is this one:

> social media ensures we're constantly exposed to the highlight reel of people's lives and that's leaving some feeling like they're not quite making the most of their time on this mortal coil.

Social media and the internet has made it incredibly easy to see the various people out there who are in the top of their league at any particular hobby, vocation or activity, and made it seem like that's the 'norm'.

If you're an entrepreneur, it can feel like everyone's making millions off their startup ideas and that anyone who isn't is a failure. If you're a web developer, it can seem like the majority of developers are experts in every one of the latest trendy languages and frameworks and that you suck for not being like that. If you're making a game, you might end up judging your worth in comparison to say, the makers of Minecraft or Pokemon GO.

But keep in mind that these are outliers by default. The top percentage of people in a field are overrepresented online and even then, they mostly only post about the things going well in their lives.

Most people are average, and for any one thing you're interested in, there will almost always be many millions of others who are either better or worse than you at it.

And hey, you don't have to be 'great' or even 'good' at something to be successful in it. Many people who were 'average' in a field ended up doing really well in it regardless. Maybe they had a good team, maybe they put in a stupid amount of time, maybe they simply had the right idea at the right point in time. You can just as easily be an expert or prodigy toiling away in obscurity as you can a celebrity with few skills to speak of.

In other words, don't worry too much about it.

SCdF 1 ago 2 replies      
In my mid-twenties I had this peak of "I'm going to make the next Facebook for cats and it's going to be amazing". After repeatedly trying to have an enjoyable life and make the aforementioned Catbook* I realised that I'm actually pretty OK with not being super duper rich and famous. I quite like being out to chill out and watch some dumb youtube crap. Or actually spend time with my partner. Or put my health (running, sleeping a good amount, not using a computer too much etc) above some violent need to succeed.

In my early thirties I've now taken two pay cuts to move to jobs that I thought would be more enjoyable, as opposed to more important / prestigious / success signifying.

I still tinker around and would like to produce something all on my own, but I don't really care if I don't. What I'm doing now for someone else is important enough.

*Not actually it, obviously I'm not a complete failure

ThomPete 1 ago 2 replies      
Normally when people ask me why us Danes are considered the happiest people in the world I always answer the same way.

We have no aspirations and are totally fine with living a mediocre life. The entire society is based on that (with the high level of wealth redistribution)

Good enough is almost the definition of Danish working culture.

It's not for everyone (I moved) but it sure is a good quality of life.

doc_holliday 1 ago 1 reply      
It's absolutely important to your well-being and happyness to understand when something is enough and to be at ease with what you have.

Hedonistic tendancies teaches you to want more and more, you never appreciate what you have because once you have got it you want the next thing. It becomes a zero sum game and you've probably just sacrificed a whole heap of your precious time and health to achieve it.

Many people fall trap to trying to beat others in what they have / what they do.

They see their friend has a bigger house, they want a bigger house.

They see their friend had a big weddding, they want a bigger wedding.

They see their friend earns X a year, they want to earn X + Y a year.

Remember, there are 7 billion people on the planet, you will probably not be the richest ever, so learn to be happy with what you have.

This is not to say do not strive for improvement, just allow yourself to be happy.

SmellTheGlove 1 ago 1 reply      
Mediocre, or just average? Mediocre implies something, like you're lazy, not trying your best or otherwise leaving something on the table that you should grab. The outcome is being average, and there's nothing wrong with that at all. I have for sure been gunning for the next promotion, but I've started to question my motives there - I'm director level now, and is the personal sacrifice really worth it to keep moving up?

I started to do the math for us. Public school starts in 2 years, there's $1000/mo back in our bank account (private pre-k/preschool now). My federal student loans can "only" go another 18 years max. Mortgage will stick around, but isn't terrible. Cars are paid off. And my wife stays at home, which was a goal for us (she was a director too, that was a fun adjustment financially). Right now I spend every dollar that comes through the door on mostly non-optional things. Some months a little more. But I'm realizing that in terms of financial obligation, I may be at or near my high water mark. I never realized that before. If I never get another promotion, we'd likely be just fine, even improving financially over the next decade or two.

So the question becomes, do I really need to hit VP or whatever other level? It's not to say I don't work hard now, but I mostly leave the office to get home by 5. I take all of my vacation time and wish I had more. Maybe I have the potential to keep moving up, maybe I don't, but the question is really becoming more whether I'm willing to put in more time and energy here, rather than outside of work. I think I'm pretty good at what I do, but does that obligate me to go as hard as I can at that thing I do pretty well, and subjugate everything else just a little more? I don't think it does.

ioda 1 ago 2 replies      
"Everybody is a Genius. But If You Judge a Fish by Its Ability to Climb a Tree, It Will Live Its Whole Life Believing that It is Stupid."- Einstein

"Mediocrity" arises only when compared.

And comparison is plain stupid. My genes are different. The life circumstance through which I have gone through are different. My responses to various stimuli are different. My pleasure points are different. My pain points are different. My memory is different. The things that I consider important in my life are different.Every single thing about me is different from that of anybody else in this world. And yet, if I want to compare myself to some other person on a specific domain, I should be plain stupid.

Instead I would happily compare myself with myself. Am I giving my personal best? Am I getting paid in 'currencies that are important to me'?

Life is not single subject course. It is a multi disciplinary course. One may have ideas about becoming the greatest entrepreneur, the greatest artist or the greatest politician. But not many people talk about being a great child, being a great brother, being a great husband, being a great friend, being a great father or being a great grand father.

You may think that you got an "A' in a particular subject. But you may be a complete failure in other. If you are OK with it, no issues. But make sure that you scored 'A' in subjects that you thought important, and not someone else thought important.

I would love to believe that there is no such thing as 'Mediocrity'. The greatest tragedy is to not have lived the life that you wanted to live.

jrs235 23 ago 5 replies      
Seriously, everyone should read The Underachiever's Manifesto: The Guide to Accomplishing Little and Feeling Great[1] (after they turn 18, in hopes it doesn't encourage them to drop out :). I'd recommend getting a used hard copy and placing it on your coffee table. It's a quick and humorous read that helps bring the lighter side of things back into perspective.

[1] http://amzn.to/2bAi7dN

teekert 1 ago 0 replies      
I love this post: http://waitbutwhy.com/2013/09/why-generation-y-yuppies-are-u...

Hits home for me. Life expectations people. You're not great, your 35 and your farther is still not driving your Porche as he always wanted. But hey, you have a house, income and food every day. How beautiful is that?

Try to long for the things you already have (as it taught in this "always popular on HN" -book http://www.goodreads.com/book/show/5617966-a-guide-to-the-go...)

wazoox 1 ago 2 replies      
There is a serious error in this article: by definition and etymology, "mediocre" means "in the middle, average, ordinary", absolutely not "the worst possible quality".

The "tyranny of excellence" undeservedly pushed "mediocre" meaning for many people down towards "bad", but "mediocre" isn't really bad; it's just barely average.

katzgrau 1 ago 1 reply      
Mediocre or not, my personal view is that everyone should envision themselves as the person they want to be.

Being mediocre may or may not prevent you from achieving your highest goals, but believing in your mediocrity will prevent you from setting out to achieve those goals in the first place.

Tell yourself what a badass you are every day, surround yourself by the kind of people you want to be, and keep persisting until you get what you want or are simply satisfied by the effort you put it.

There are plenty of roadblocks between you and your dreams that are completely out of your control - and the question of whether you believe in yourself doesn't have to be one of them.

igf 1 ago 3 replies      
The vast majority of people on the planet are mediocre, and many people seem happy about it; or rather, if they're unhappy then they're unhappy about other things rather than their own mediocrity.

But then you've got the interviewee for this article, who is probably not happy with her own mediocrity. You can tell this by how hard she insists that she is. People who are genuinely happy with their mediocrity don't go around blogging about it -- in fact, they don't think about it.

tibbetts 1 ago 1 reply      
The article gets it at the end even if the author doesn't appear to: in order to talk about mediocrity on an absolute scale you have to be thinking of achievement on an absolute scale, and that a flawed approach no matter what percentile you are at. Achievement, and happiness, and other metrics of utilitarian outcomes, should be measured in context, against your inclinations and potential and situation. Trying to grade yourself against an average or aggregate scale will not only be disappointing, it won't even help you accomplish the right things for yourself.
Artoemius 2 ago 0 replies      
I'm surprised that not a single comment here suggested that mediocrity is boring.

Boredom is of course a subjective state of mind, but I have a feeling that the average and the boring cannot be entirely statistically unrelated in humans.

lambdacomplete 1 ago 1 reply      
There has been a scientific study about what makes people "happy" [1]. It's even something we can easily test personally: relationships. No matter what, relationships create that emotional swing that makes our lives interesting (assuming a psychologically "healthy" person, by today's standards).

That said, how many people actively pursue relationships? To me a person who has tons of friends (the kind you spend time with on trips etc. not the kind you text every once in a while to see how they are doing) but works a frustrating 9-5 job at a bank is definitely not someone I'd look up to. On the other hand a person who is extremely successful in his field, wins the most prestigious award in that field, but does not (again, by today's standards) live a healthy life does not set a good example either [2]. So what's the optimal situation?

And appearances don't help. I have no idea whether Elon Musk (since he was mentioned in another comment) is happy. I just know he looks successful. In my mind he's the kind of guy that enters a room and automatically and instantly gets the respect and admiration of the "smart" people in there. Does he even care about that? Am I being tricked into seeing Elon Musk as a status symbol like I'm tricked into seeing the iPhone as one of the best smartphones out there?

Happiness is definitely more complex than accepting what you do as "special". Accepting your current situation is a great way to start clearing up the cloud of things you consider important but if that was really the way to be happy why would we even bother improving ourselves or society? I hate to say this but I almost feel like this is the classic story of the fox and the grapes. When the fox can't reach the grapes says they are not ripe.

What if happiness was about pursuing something, regardless of the end result?


[1] https://www.ted.com/talks/robert_waldinger_what_makes_a_good...

[2] http://www.nature.com/news/2006/060821/full/news060821-5.htm...

JMCQ87 1 ago 0 replies      
>I put it to her that all of that doesn't sound very average to me.>She pauses and laughs sheepishly. "I guess it just depends on who you're comparing me to."

That's the key thing. If I compare myself to the average person, I'm obviously not "mediocre". However, compared to some people who (I think) have had similar opportunities and potential, I probably am.

jokoon 6 ago 0 replies      
Mediocre falls along the "good versus bad" metric, which honestly is not the most valid.

It's true, it's easy to feel like society and capitalism in general are becoming a rat race. Excellency is neither subjective or objective, it's just a made up social norm that people create in their head for competitive purposes.

What do you want millions of dollars for? To travel the world for all the rest of your life? I see so many people fighting during their vacations.

If you're the best at your job, will the world really be improved by your contribution?

Innovator tend to explore beyond what society wants, and when it works, it works great.

cko 1 ago 2 replies      
I really have to internalize this message. I always feel like I have to be better and smarter than everyone around me just to feel adequate. I guess that's one characteristic of insecurity.

Is this why romantic relationships seem to kill ambition? I've seen this in myself and some of my friends.

khaledh 1 ago 0 replies      

How Will You Measure Your Life? Clay Christensen at TEDxBoston

This is a very inspiring presentation, especially towards the end. The take away is that it's not important how high you went up a certain hierarchy or how much money you have in the bank. The important question is what did you do with your life to make other people's lives better? When you're put in certain situations that touch other people's lives, it's for a reason, and your life will be measured by how you did in those situations.

isuckatcoding 23 ago 0 replies      
I think this is getting a little too much into semantics but I see people here using "mediocre" as being synonymous with being "average". What if I'm feeling BELOW average?


advertising 23 ago 0 replies      
Had to be a CEO and rich until I tried being both of those things and realized it wasn't as important as I thought. Pursuit of those things was quite an unhappy pursuit. But I needed to know I had tried before I was comfortable not being those things.

Maybe instead of being average, focus on your average of the 5 people you spend the most time with. :D

DrNuke 1 ago 0 replies      
Social gamification is almost impossible to evade but you can still disconnect and carry on with your own local life.
sixhobbits 1 ago 4 replies      
"Mediocrity" is such an ill-defined concept. As you advance out of being mediocre, the pool of people you compare yourself to gets smaller, and you still end up feeling mediocre. When you get to Elon Musk levels, I would assume you switch from comparing yourself to those you know and instead compare yourself to historical figures (e.g. Henry Ford) and perhaps still feel mediocre.

Then add in the Dunning-Kruger effect (the real one, not the popular misinterpretation of "you're so dumb that you think you're smart") and judging what exactly mediocre means becomes even more difficult.

But in the end, sure you can convince yourself that the negative feeling of not achieving as much as you might have been able to isn't worth losing sleep over, and you might end up 'happier'. You can also try to leverage the feeling to stop yourself from wasting those 50+ hours rewatching Game of Thrones and instead putting that time into becoming more like someone who you perceive as being less mediocre than yourself.

throwaway991199 1 ago 7 replies      
This sort of article should really serve as a wake up call to this community.

There was another article here recently: https://news.ycombinator.com/item?id=12329255, where some comments mentioned that people should retrain once jobs become available for consistent automation.

Lets take the simple example of the machine that picks apples: https://www.youtube.com/watch?v=TBcWZcjXr-I

Typically immigrants do this work, low educated immigrants. They do it because they can't get any other work in higher skilled professions. Sure they could be a driver, oh wait soon that will be autonomous. They could work in a warehouse, oh wait Amazon is doing it's best to disrupt that. They could be a cleaner? Oh wait, companies like Roomba and Dyson are working to disrupt that.

The point I'm trying to make here, is for all the poorly/low educated people and lets be really, seriously honest are in the tens of millions. What are they going to do?

I've traveled all over the world, all the continents. There are segments of the population that can't read, can't write, can't even grasp basic maths. They are the ones who depend on these low end jobs.

Are you going to tell me with a straight face they they can re-train, go back to school and work in STEM? It's just not feasible. Also, who is going to pay for all these millions to retrain for years. Remember, they'll probably have to restart their education, basic maths, basic science, then college, then university. That's what 7 years? Who will pay for their living expenses for them and their family?

There is a ticking time-bomb coming soon. Where we'll have an OMEGA man type of situation. All sorts of jobs will be automated and people won't have anything to do.

What are the solutions?

1) Do we implement 1 child per couple policy? To lessen the burden on the state?2) Do we provide free schooling with a zero-tolerance on NO child left behind? So that they can go on to STEM fields?3) Are there enough places in STEM fields for those who do retrain to move into? Is this another thing for government to throw money at?4) Does society move from a capitalist to a socialist/communist system? But what happens when government runs out of money?

What are we going to do?

Just saying people will retrain is just utter folly.

There is a time-bomb ticking and some of you just don't realise it.

Want to know the result of no jobs, low educated populous, government with no money, socialism failed. Oh yeah. Greece. How's that doing for the last 10 years? It will be like that for another 25.

Kenji 1 ago 0 replies      
I have the simplest tastes. I am always satisfied with the best. - Oscar Wilde
guard-of-terra 1 ago 1 reply      
How to be reasonably good and productive while having a life and not cannibalizing your time with work? (or anything else for that matter)

That's what should have been asked.

scythe 1 ago 0 replies      
The dominant ideology of Western aesthetics in life is existentialism: the idea of excelling by your own definition. But in order for this to be meaningful, "your definition" has to be socially informed, and yet by construction, it's not part of the theory. In some sense there's an essential tension between the need to be free and the need to not be insane. Unfortunately, succeeding according to a socially-informed but ill-defined set of criteria basically always reduces to being socially successful in one way or another, and it might be cheeky, but it's true, to point out that this is a perfectly reasonable metric of success... for philosophers. However, the fact is that everyone can't be socially successful, and the result is that many people who are motivated by existentialism in its popular form end up unsuccessful and unhappy.

Note that "social success" does not mean socializing; Paul Dirac was extremely socially successful, yet nearly incapable of socializing. And despite this it strikes me that he would have been less happy if he had not been one of the greatest physicists in history: introversion or even (possibly) autism is no antidote to vanity.

In light of this people point the blame at social media, but don't ideas matter?

ObeyTheGuts 23 ago 0 replies      
This read was 100% onion article
andrewclunn 1 ago 1 reply      
I was going to write a lengthy in depth comment, but this is good enough.
touristtam 1 ago 2 replies      
Why is this on HN?
amelius 20 ago 1 reply      
HoloLens secret sauce: A 28nm customized 24-core DSP engine built by TSMC theregister.co.uk
245 points by runesoerensen  14 ago   110 comments top 10
ericseppanen 11 ago 10 replies      
For those who haven't had the pleasure: developing on Tensilica Xtensa cores generally means living within 128-256KB of directly-accessible memory; a windowed register file that makes writing your own exception handlers "interesting"; a 6-year-old GCC bolted to a proprietary backend; per-seat licensing fees to use the compiler; and a corporate owner that's only halfway interested in the ecosystem they now control.

So yeah, kind of wishing it would just die and let ARM take over the embedded space.

AndrewKemendo 13 ago 4 replies      
Holographic Processing Unit (HPU) chip used in its virtual reality HoloLens specs

It seems to be insurmountably hard for press to understand that virtual reality and augmented reality are related but distinct concepts.

Beyond that, interesting teardown. The 10w number is great, but my guess is a lot of the high power processing happens in the sensor suite because it's using 4 repurposed IR sensor receiver combos to relay depth data in a highly structured way. That means this processor is the glue between the IMU and RGBD camera combo. I think in the end this can't scale down to consumer side with this approach, not to mention the other hindrances to scaling down with the protection system.

markingram 11 ago 2 replies      
HoloLens 3D miniMap of House with Synced Ladies +1 https://www.youtube.com/watch?v=T-JvTZjbwNsGG:)
ex3ndr 2 ago 0 replies      
Well this explains a lot. In my university (Russia, Mathematic-Mechanic faculty of SPBu) there are a lot of investments in computer vision stuff and it it almost impossible to build fast and with low TDP software for CV without SDP/FPGA/etc stuff.
eganist 13 ago 6 replies      
Lay speculation here:

If the first run is on a 28nm process, does this suggest the second generation on e.g. Intel's 14nm process might yield a drastically more powerful HoloLens model in the current form factor, or at least a more compact one capable of all the same (assuming the optics were also compacted) for revision 2?

There's a whole ton of stuff I'm not taking into account such as the physical size of the current HPU, but my guess is that this is the largest point of improvement just going by the process alone.

oaf357 13 ago 4 replies      
Only 10W for all that? While I'm a little sceptical that is an impressive number.
cloudjacker 13 ago 0 replies      
I love to see when vendors actually modify a chip instead of using just off the shelf parts.

But even then, low bar....

ioquatix 12 ago 0 replies      
TLDR; It's running Windows 10 :)
frik 9 ago 7 replies      
Several independent review article about Hololens mention the limited view (relatively small view angle), underwhelming and off putting. (You can immediately spot a sponsored article if it doesn't even mention that issue at all.) iPhone 6 and high end Android smartphones already have augment reality apps with SLAM technology that is far more impressive than what MS PR is suggestion Hololens might one day be able to offer - the E3 presentation in 2015 and 2016 were faked as we know in the meantime.
velox_io 13 ago 5 replies      
This is a GPU rather the a DSP (the tricky part is rendering not filtering). Core count is a bit of a pointless metric unless you know how big/ powerful a core is.

I like the idea of a self contained VR headset with no wires, but I don't think going to see on for ab a decade, there's just too much processing power needed for a realistic experience. I hope I'm wrong!

Edit: It appears this is in fact DSP (digital signal processor). That's a huge amount of power for dedicated signal processing, I'm intrigued to know what can be done with it.

A parallel implementation of gzip for modern multi-processor multi-core machines github.com
276 points by odeke-em  2 ago   64 comments top 23
dmourati 2 ago 1 reply      
I learned about pigz in the High Performance MySQL O'Reilly book appendix. I used it, and other techniques there to improve our MySQL backup/restore time by 7x. This in turn won first place at a company hackathon.
maxpert 2 ago 3 replies      
I love it! I just wish somebody can make a GPU based compression library (doesn't have to be gzip or bzip), every mobile device today is shipping with a GPU, there are some techniques out there like this (http://on-demand.gputechconf.com/gtc/2014/presentations/S445...), but I am still waiting for a solid implementation.
Rapzid 2 ago 0 replies      
Used extensively while a system engineer at a hosting company; 10/10, would use again. Excellent utility if you have the cpu cycles to spare and need to cut time; gzip is almost always your bottleneck. Didn't seem to quite scale linearly, but what does.

--rsyncable support too.

Twirrim 2 ago 4 replies      
pigz is extremely fast and very capable, plus it's packed up and provided for almost every mainstream linux distribution, and can act as a drop in replacement for gzip as it supports the same flag syntax.

On the bzip2 side there is pbzip2, which is also a drop in replacement, http://compression.ca/pbzip2/

mgerdts 2 ago 0 replies      
pigz parallelizes compression. I've made changes that make it so that it can parallelize uncompression as well.


This feature is present in Solaris 11.3 and later. Actually, it is in some later patches to 11.2 as well. I added it to speed up suspend and resume of kernel zones.

spullara 2 ago 3 replies      
My guess is that this compresses less efficiently as you would have to shard the dictionaries. Might be close though for large files. I was surprised that there were no speed or efficiency comparisons in the README.
tobias3 2 ago 3 replies      
And the same for LZMA: https://github.com/vasi/pixz

(it's relatively easy to remember those commands)

bpchaps 2 ago 0 replies      
Why doesn't parallelized gzip get more attention? For larger files, it can be quite a pain to see a multicore machine sit mostly idle. Something like this works pretty well, but I've never seen it done outside of my own stuffs:

split -l1000000 --filter='gzip > $FILE.gz' <(zcat large_fille.txt.gz)

lateralux 2 ago 1 reply      
I'm using pbzip2 to solve this problem. http://compression.ca/pbzip2/
dorfsmay 2 ago 1 reply      
If you know that what you compress is alphabetised text, and space is more important to you than time, then please use lzips over gzips. For a parallel version:


axelfontaine 2 ago 0 replies      
Previous discussion 6 years ago: https://news.ycombinator.com/item?id=1233317
discreditable 2 ago 0 replies      
You can also use 7-zip for multithreaded gz/xz/bz2/7z/zip compression.
pselbert 2 ago 0 replies      
It's amazing to see what somebody like Mark Adler can achieve when they focus on a specific niche. You often hear about the importance of specialization when positioning a business, but it isn't mentioned as much for individuals. His work serves as an ideal case of specializationif I wanted to ask an expert for advice on compression, or was looking for an outside expert on compression that is immediately who I'd think of.
sshaginyan 2 ago 1 reply      
Why not just use GNU parallel?

`parallel gzip ::: file1 file2`

I wish there was a standard for this type of stuff. So that an app will check for existing child spawns and act accordingly (IPC).

teej 2 ago 0 replies      
I use this extensively in schlepping data around for analytics work. Any data that moves over the network going into or out of a data store gets compressed with pigz.
haughter 2 ago 0 replies      
How does the strategy for parallelization of gzip in this project differ from the strategy used for LZMA2 parallelism as implemented by 7z?
slashcom 2 ago 0 replies      
http://lbzip2.org/ also exists for bzip2. It works really well, especially since bz2 files are split into discreet blocks which can be un/compressed independently.
axelfontaine 2 ago 0 replies      
The big challenge seems to be parallel gunzip as that requires a special stream with an index to work, with no general purpose solution available so far.
lmeyerov 2 ago 0 replies      
We use this in production at Graphistry, super useful for latency-sensitive dynamic media applications! (In this case, we needed to add an auto-tuner + node bindings.)
noipv4 2 ago 0 replies      
For bzip2 lovers the tool is pbzip2 ;)
abhishivsaxena 2 ago 1 reply      
Any ideas how this would compare to gzip while on a Microserver? I'm thinking of Atom C2750 bare metal from packet.
Hydraulix989 2 ago 1 reply      
Probably useful for servers more than anything where perf really does matter at scale and content type is gzip.
donatj 2 ago 0 replies      
Huh. I was living under the incorrect assumption that gzip was inherently single threaded
Taking the final wrapper off of Android 7.0 Nougat android-developers.blogspot.com
282 points by raptaml  20 ago   230 comments top 18
ymse 20 ago 3 replies      
Just a shout-out to the fine folks over at CopperheadOS[0]who have been doing a lot of behind-the-scenes hardening in Android over the last year[1].

If you have a Nexus device and don't care much for the Play Store applications (F-Droid is included) give it a go for a true AOSP experience.

The only apps I'm really missing are Tasker and Signal, but I have a second phone for those.

0: https://copperhead.co/android/

1: https://copperhead.co/android/docs/technical_overview

primo44 19 ago 12 replies      
No Android 7 on my Nexus 5 might just be what pushes me to iOS. I've gone from being an Android evangelist to being an Android apologist ("Yeah, battery life sucks. Google has failed to fix it after X years") to now being, frankly, pissed off that my expensive phone is going to be left behind.
jacquesc 19 ago 0 replies      
Using the beta for the last couple weeks (easy to install on nexus via android.com/beta), I really like the notification improvements in 7.0. They are grouped by app now, which makes things easier to navigate, and the new clean styling of the notification content makes it more pleasant to read.
velox_io 14 ago 0 replies      
1171.5MB for the download alone!

It is a shame about the Nexus5 not getting the update, it's still a pretty decent phone and not slow by any means.

I broke the screen on my Nexus5 recently so I brought a Nexus6, so far the battery seems to be worse than the N5 (I miss the wireless charging), so I'm looking forward to the battery improvements!

Edit: The split screen is a great feature, and not finicky at all. Being able to play videos in the background is going to be one of those features that you cannot live without (OK, it's a first world problem).

frik 9 ago 0 replies      
Global Smartphone Marketshare Q2/2016

 Android ... 86.2 iOS ... 12.9 Windows ... 0.6 Blackberry ... 0.1 Other ... 0.2

qznc 20 ago 4 replies      
Personally, I don't care for any [0] of the new features. This is the greatest feature, because it means Android is mature now.

[0] Sure, better performance is always nice, but I don't care for Vulkan, VR Mode, or Multi-Window.

SadWebDeveloper 20 ago 3 replies      
No nexus 5 (not 5x) OTA Update ='(
cpprototypes 18 ago 2 replies      
Does anyone know how strict is the Google two year update policy for Nexus? It's clear they won't do major releases (such as Android 7.0) for old phones. However, what about critical security updates? For example, if something as bad as Stagefright was discovered, would Google do nothing for all the older Nexus phones out there? Or would something like that be a special exception to the rule?
UncleChis 18 ago 2 replies      
Did I read it right: Nexus Player is in the list of OTA update? I thought the device is dead? Anyone has a link of Nougat for Android TV?
dang 20 ago 0 replies      
Also https://news.ycombinator.com/item?id=12337538 and https://news.ycombinator.com/item?id=12338381. If there's a clearly better URL, we can consider swapping the current one out.
ashitlerferad 7 ago 1 reply      
Why was it wrapped in the first place?Why can't Google do development in the open?
roschdal 19 ago 2 replies      
Is there a way to force update to Android 7 on Nexus 6P now?
PaulHoule 19 ago 2 replies      
Some of those screenshots make my eyes bleed, particularly the one with the multiple-windows with one of the windows having the painful app selector that pretends to be a window selector.

When will we see Android 7 on a third party device?

therealmarv 15 ago 0 replies      
Why did they removed the color correction slider from dev previews? My Nexus 5X has a light yellow tint and the cool color switch is way too cool. Three sliders for RGB would be ideal. Now I still need to flash custom kernels to adjust RGB :|
0134340 17 ago 1 reply      
Works well on my Nexus Player and happy to see Google is still supporting it.
digi_owl 18 ago 0 replies      
Looking forward to see AOSP reach Android-x86.
msoucy 19 ago 2 replies      
Meanwhile many phones (especially vendor-locked ones like the Droid Turbo...) are still stuck on Lollipop.
Ologn 20 ago 3 replies      
> Multi-Window support lets users run two apps at the same time

(click the Multi-Window support link)

> If your app targets API level 23 or lower and the user attempts to use the app in multi-window mode, the system forcibly resizes the app unless the app declares a fixed orientation.

> If your app does not declare a fixed orientation, you should launch your app on a device running Android 7.0 or higher and attempt to put the app in split-screen mode. Verify that the user experience is acceptable when the app is forcibly resized.

Great, more nonsense being shoved down Android developer's throats. They still claim that newer versions and new features are not breaking their API - if this is the case, why would an app on Google Play from a year ago need the developer to "verify that the user experience is acceptable" just because they published a new version with a new feature?

It's been almost nine years since Android released its initial SDK and they're still playing cowboy stuff like this. This feature should be turned on for apps targeting Nougat or later, you shouldn't make every Android developer check every app ever made because three people on the Android developer team made a new feature and can't wait for its use to become more widespread, so they turn it on by default for legacy (pre-Nougat targeted) apps.

Data points that Facebook uses to target ads washingtonpost.com
236 points by suprgeek  3 ago   131 comments top 17
erikb 3 ago 17 replies      
"perfectly targeted ads"? Never had that experience.

Even if I summarize all the analytics solutions out there, like ads, Facebook stream, Youtube, Amazon, Netflix. What they usually do is they figure out what I did in recent history or in general and present me more of that. I don't know about other people, but that's absolutely not what I want. That's boring, or maybe even unnecessary.

Like, if I just bought a book about Linux system administration the only reason I would buy a second one would be because the first one sucked (and Amazon knows the ratings of the books). A reasonable suggestion after that would be something people do with Linux, like hosting a Wordpress blog. Honestly I might not even know what I would do with my newly acquired skill once I succeeded with that topic. Also it's quite tiresome to learn. Maybe after reading 30% of it I would like to insert a simple but interesting novel or comic book. Why not suggest something like that to me?

Or I just watched an action movie on Netflix. Then I don't need other action movie suggestions. After that maybe I want to watch a character focussed tv show.

What we need is not a "I know what you did and here is more of that" suggestion engine. What we need is a "oh that's getting boring, here is other awesome stuff that may surprise you" suggestion engine. Figure out the stuff I don't know I want and then suggest that.

And honestly, not a single of these billion-dollar-engines out there do that for me at the moment.

nsxwolf 3 ago 4 replies      
I like when they show me ads for the expensive thing I already bought, as if I'm going to buy it again. Had they predicted I wanted to buy that thing, and shown me ads for things like it first, that might have been useful.
frik 3 ago 0 replies      
You wouldn't see such an article on WashingtonPost before 2013. The former owner of the Post Donald Edward Graham was the mentor of Zuckerberg and Mr Graham is the lead independent director of Facebook's board of directors.
newsat13 3 ago 7 replies      
Does anyone know of any startups or upcoming projects that are trying to compete with Facebook (i.e building a social network?).

I still haven't managed to wrap my head around how exactly facebook makes money. Do so many people actually click on ads? And this somehow makes billions? I find it very hard to imagine (for myself).

zappo2938 3 ago 6 replies      
When people purchase GQ or Vogue, they are buying it as much for advertisements as for editorial content. When someone flips through one of those magazines, it's like they appreciate the designers hiring models, taking pictures, and sharing it in the magazine even though both they and the reader paid. Take the ads out of GQ or Vogue and there is almost nothing there.

If I create an ad on Facebook and people click on it, leave comments, and share it, Facebook will mark it as quality content and discounts the price per impression. It seems that Facebook wants to create the ad experience of GQ or Vogue and gives money incentive for advertisers to do it.

Facebook knows I'm into technology and web development. There are lots of great products for web developers created every month. Facebook is a way for companies to find people who are interested in their products. It helps people learn about new products and it helps companies target people who are interested in their new products.

There was a fundraising concert last year for a local charity. I volunteered $300 and time to do Facebook advertising. It yielded several thousands of dollars in ticket sales. For $300 Facebook let me communicate to people who really cared about either the bands who were playing or to people who really cared about the project what was happening. Facebook let me target people who like Candelbox within 40 miles of the venue.

This is good. I don't understand why people hate on it so much.

jypepin 3 ago 0 replies      
I definitely notice facebook ads being more and more interesting to me. And even more interesting is that most ads are of the "need creation" type. Things I didn't really google anything about, or I never really thought as a product I would buy that show up and are actually relevant to me.

Last one I can remember is from yesterday a new startup doing good, cheap luggage (both carry on and bigger ones) which had me go to their website, spend some time looking at some of their stuff and made me think that I new a new carry on since then and I'm very close in puttin $250 in a carry on...

I guess I travel pretty frequently I don't know if facebook is able to say that I travel a lot by different locations I check in, or from my use of google flights and booking.com etc. but yeah, I never googled anything about luggage or buying anything else related to travelling (except hotel and plane tickets). So pretty interesting to see

clumsysmurf 3 ago 0 replies      
Yet another book out recently which explores this topic:

"Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy"


jonathankoren 3 ago 1 reply      
Like everyone, we get a quite a bit of junk mail at our house. Some of it comes to "current resident", others target us by name. Broadly speaking it breaks into two groups. Things that are either geotargeted. (In my neighborhood, that would include ads such as "Why don't you lease a car through Uber and pay for it by driving!") and those that are more personally targeted (think credit card offers).

Today we got an interesting one. It was an offer to try Blue Apron. If we lived in Palo Alto, I'd chalk it up to geotargeting rich neighborhoods, but we don't live in one. Also, it came to wife who posts recipes on Facebook. It's enough to make us wonder if Blue Apron is somehow linking Facebook targeting data with real world names and addresses. Now I don't know if it's true, but if it is, that's creepy as fuck. That's autodoxing.

chinathrow 3 ago 0 replies      
"While youre logged onto Facebook, for instance, the network can see virtually every other website you visit. Even when youre logged off, Facebook knows much of your browsing: Its alerted every time you load a page with a Like or share button, or an advertisement sourced from its Atlas network."

If you want to enable sharing of your content on FB and other platforms easily but you do not want to support the tracking madness, you might want to use Shariff. It's setup to not send any tracking information to FB etc unless the user wants to share something.


wtracy 3 ago 2 replies      
Any ideas how Facebook acquires information about things like home ownership? I don't recall filling that out as part of my profile.

Do they perhaps match users' identities to public records? Is it possible to look up home ownership given just a name, demographic information, and contact information?

chirau 3 ago 13 replies      
Contrary to popular opinion, people want to see ads.

A good ad is very much appreciated. This is why magazines still make money. This is why the SuperBowl is popular amongst non-football fans. Unfortunately, most ads are spammy which is why internet ads have earned this terrible reputation.

Silhouette 3 ago 0 replies      
Perhaps the most disturbing thing about this article is how much data from other sources Facebook apparently has access to now. How do these "data brokers" have such detailed information about things like purchasing power and spending habits to share with Facebook in the first place?

It seems to me that a lot of that must come from either retailers or financial services, and in either case questions should be asked about how much people understand about what data is being collected for purposes other than simply making the intended purchase and what is done with that data.

et-al 3 ago 0 replies      
Well now they can definitely compete with eHarmony's 29 points of compatibility.
ddmma 3 ago 0 replies      
I guess there must be a fine line between data with personal character like social security number for instance and private information as your current location.

Every coin had two faces, wonder how many faces bitcoin will have in the future

sp527 2 ago 0 replies      
Why did the submission title change? It was the article title before and I didn't find that remotely controversial. Did FB PR get in touch or something?
astazangasta 3 ago 1 reply      
I don't understand how people can build such a system. Yes, you. I don't understand how people in the thread are debating the merits of advertisement, as if that is what matters here. 1984 is here, and we all love Big Brother.

I mean this 100%: if you are participating in ad tech, collecting data on grandmothers in large databases, quit your job now. You are doing harm.

JustUhThought 3 ago 0 replies      
So I guess we've moved past the part of the conversatio where we ask if this tracking is ok?
The Long, Remarkable History of the GIF popularmechanics.com
244 points by pmcpinto  1 ago   104 comments top 22
SquareWheel 1 ago 17 replies      
It's honestly something I've become quite bitter about. I find the resurgence of the gif has significantly reduced my enjoyment of browsing websites like reddit. Even small subs which used to be the place to go for deeper discussion have largely devolved into image boards targeting the lowest common denominator.

When it's appropriate I have no problem with a video. But if posting a video it should really be in its original form. Uploading to Youtube or even just using a <video> tag lets the viewer decide if they want to listen to audio, and what resolution to watch in. It's pretty rare that I'd choose to watch in no sound and at the lowest resolution possible. Not to mention the absurd amount of data gifs download.

The recent trend of converting videos into gifs back into videos (websites like gfycat) make even less sense. All you're doing is damaging the quality of the video. To what end?

Even articles that are posted are now scattered with gifs. Why? I guess because it's trendy. But it makes it incredibly difficult to focus on the text. The article above is a good (and likely self-aware) example, but this one from Atom also stuck out for me.


Terribly nauseating.

So that's my rant. Frankly I'd welcome a browser extension to block animated gifs online and hide reddit posts than only link to them.

ianamartin 1 ago 3 replies      
I'm pretty sure everyone is missing the point here.

GIF isn't about optimization. It isn't about the best technology. It isn't about anything that we care about as engineers.

It's about what is easy for users to create.

That's it. That's all there is to it.

GIF is easy.

It doesn't matter if it sucks. It doesn't matter if it's not optimal. It's easy.

Gankro 1 ago 1 reply      
Great read, but the initial description of the inefficiencies in gif are off. They touch on it a bit at the end but make it vague so I'll explain it here (because I love gif because it's a hillarious mess).

Gif is actually a mosaic format. It defines a canvas on which you can render a series of images. By adding a (per image) delay between images, you can make an animation. All this data comes as a stream of blocks. Most are images, but some are extensions. The "NETSCAPE 2.0" extension is what makes gifs loop.

Nothing at all says an image needs to fill the whole canvas, and the format for rendering a new image specifies what should happen to the existing pixels. It also can specify a colour to be interpretted as transparent. Gif is also a palletized format, and you can actually abuse this to get more than 256 colours -- each subimage can have its own palette.

By picking "leave all the pixels there" as the transition format, you can do non-iframes by overlaying only pixels that have changed. You can also apply many small deltas "at once" with 0-second delays to optimize this further (think: only changing the corners of the image). You can then of course "stick" pixels to a single color of they don't change by much to avoid re-encoding them.

The end result of all these tricks is a pretty dramatic compression as one would expect from a "real" video format. It's not great but it's a lot better than the naive encoding described at the start of the article.

If you find a real gif that's "live action" and decode the individual frames, you will very likely see all these optimizations in action!

Note however that much of this is underspecified! The only reason your animated gifs work is because everyone vaguely agrees on an interpretation of certain things which are honestly quite ambiguous. Firefox's testsuite has a pile of ambiguous gifs which will render messed up in different image viewer applications!

Edit: this one is super cool to watch in delta-only mode (don't have that version on hand anymore): https://github.com/Gankro/gif-rs/blob/master/data/cat-jump.g...

fsiefken 1 ago 7 replies      
In the article Battilana is quoted as saying: "Even browsers today don't have a good alternative to GIF unless you want a very sophisticated and still patent-covered video like MPEG-2 or MPEG-4."

But we have WebM, works on Chrome, Firefox, Opera and on IE with components https://www.webmproject.org/ie/ - is that not good enough?

No mention of the Animated extention to PNG, which Firefox, Safari and Opera do support. Chrome has an add-on for it but I don't know about IE. You wonder if the image is smaller when it's a silent video or when it's a APNG, probably it's the same with animated GIF, the movie format is more efficient then the animated image format.

Perhaps someone can make a proxy plugin that converts all animated gifs on the fly to webm and all gifs to png. You never have to see these GIFs again, another SaaS subscription model

xvilka 1 ago 3 replies      
This is web browsers developers fault: Chrome developers actively resisting APNG support [1], while Firefox developers actively resisting WebP support for years [2] and [3].

[1] https://bugs.chromium.org/p/chromium/issues/detail?id=1171

[2] https://bugzilla.mozilla.org/show_bug.cgi?id=600919

[3] https://bugzilla.mozilla.org/show_bug.cgi?id=856375

danso 1 ago 1 reply      
That was a nice lengthy read. While the GIF format itself is a bit outdated, I hope the artform of excerpting video/animation into a silent, short-loop never goes out of style.

Is it impossible to turn back GIF adoption? What if someone makes the equivalent of Steve Jobs's declaration against Flash? To me, Flash was far more vital to the very operation of web services. Then again, processing GIFs doesn't require near the same complexity or security risks as Flash did.

garaetjjte 1 ago 2 replies      
Actually animated PNGs exist: https://upload.wikimedia.org/wikipedia/commons/1/14/Animated...(though it maybe works only in firefox, idk)
runaway 1 ago 0 replies      
The author is wrong about gifs encoding full frames. Properly encoded gifs only store the pixels that change in each frame. His own example with the dog shows this http://i.imgur.com/wISu3An.png
hartror 1 ago 0 replies      
The limitations of a medium drive creativity, and there is an expectation when publishing a video of a certain quality. Not so with gifs, the format lowers the bar for people to be creative overcoming many expectations of quality.
mxfh 1 ago 2 replies      
While this article excells on the technical aspects of the GIF format, it misses out on the significance of the GIF art renaissance of ca 2011 to 2014, which was most likely initiated by tumblr's arbitrary file size limits.

Here is a primer on that:


Waterluvian 1 ago 1 reply      
Can someone point me to how I show full quality animations on an iOS device? I've seen some canvas style spritr sheet solutions but those feel bulky for when I just want a looping animation.

WebM has been awesome for animations on desktop but they don't work properly on iOS: won't animate inline.

_fs 1 ago 1 reply      
I know we are supposed to be commenting on the article, but why is this headline "* is dead, long live *" so overused[1] on hackernews. It is so tiring.

[1] https://hn.algolia.com/?query=%22is%20dead%20Long%20live%22&...

digi_owl 1 ago 0 replies      
Never liked gifs.

In particular those "joke"/shock gifs that pretend to be a still image.

One of the first things i go looking for in a new browser install is some way to stop gifs from animating.

ausjke 1 ago 0 replies      
It's painful to read the article with fonts/colors/videos came out everywhere. Is there a better and practical format on the way to replace gif since it's dead?

I use gif once a while and feel it does its job pretty well.

rangersanger 1 ago 0 replies      
I had an incredibly hard time concentrating on the article with all the GIF's (or videos, I suppose) looping on the page. Down with the GIF!
ppod 1 ago 0 replies      
Great article. Gets the tone and level of detail exactly right for talking about something like this to a wide audience.
kozak 1 ago 0 replies      
GIF is an excellent example of how constraints facilitate creativity.
4684499 1 ago 0 replies      
I can google image search gifs, but can't google image search webms.
witty_username 1 ago 0 replies      
To me, GIF = slow loading animation.
EGreg 1 ago 0 replies      
cocktailpeanuts 1 ago 3 replies      
Please somebody provide a TLDR.

The content is too long and the title too click-baity for me to invest time in reading without what I'm getting myself into.

Google Intrusion Detection Problems fredtrotter.com
362 points by K0nserv  1 ago   124 comments top 23
colinbartlett 20 ago 2 replies      
If Google is ever going to fix their cloud market share problem[1], they need to reverse the deeply engrained perception that they don't support or maintain their services for the long term.

The majority of Hacker News pundits will all have the same gut reaction to these kinds of anecdotes: complete and total lack of surprise. And although not all of us are in a position to control the vast budgets of the enterprises that drive much of this marketshare, the opinions of the rank and file do have a vast impact and I believe Google's unstable support and product commitment is one of the biggest things holding them back.

1. http://www.nytimes.com/2016/07/25/technology/google-races-to...

abstractbeliefs 22 ago 4 replies      
I don't understand why people are ever surprised about Google offering substandard support for really anything.

Time and time again Google products prove to make the happy path happier, but as soon as anything should go wrong, you're on your own with no explanation, and no support.

Additionally, and as an aside to the core issue here, I've found the recent Google Cloud UI updates to be a real pain. When I last used them, I did acknowledge that it was beta, but I seemed to lose an awful lot of oversight of an awful lot of things. It's put me seriously off it vs. AWS.

20years 20 ago 0 replies      
This is a perfect example of why I refuse to use Google Cloud. Google in the past has put me through similar automated loops with Adwords and Adsense. Even disabling an Adwords account for 2 weeks for reasons I still do not completely understand. After weeks of trying to get a hold of the right person and being pushed through a ton of different channels, it was finally re-instated with a generic apology and still no concrete reason as to why.

I have learned that it is way too risky relying on Google services for crucial business operations.

tscanausa 21 ago 17 replies      
My name is Terrance and I work on Google's Cloud Support Team. Our team mission is to Reduce Customer Anxiety. We take that very seriously, and Fred's experience obviously shows that we fell short this time. The process should have been an easy flow to follow, and it was not. We are reviewing this incident in detail to ensure that we make the process less error prone and quicker in the future.


nokya 21 ago 1 reply      
The Nth story about that guy who runs a business depending on Google services and suddenly starts telling everyone else they shouldn't run their business on Google services because Google services flagged them and they realized it was a bad idea to run a business based on Google services...
redstripe 21 ago 1 reply      
This is the google M.O. - optimize all processes through cheap machine learning and heuristics and accept that some people will get screwed along the way.

The number of people that can be served at very little cost without any human involvement is apparently very lucrative. Google just considers customer service as an archaic relic that predates the invention of behavioral algorithms.

paradox95 19 ago 1 reply      
Once had Google kill a Spark job after it was running for about 12 hours because they thought we were doing something nefarious. It cost us thousands of dollars in wasted time and Google spend because their automated system made a mistake. They never attempted to fix it, refund us or even seem concerned.
Animats 20 ago 1 reply      
Intrusion detection false alarms can be a problem if you run anything like a web crawler. I have one running at "sitetruth.com", which is a site rating system hosted on a leased server. (Not a cloud service, a leased rackmount server). One of the things it does is to find the home page of a site by trying "example.com" and "www.example.com", with and without HTTPS. Some sites will block access for 30-45 seconds if those requests are made too fast.

About once a year, there's a serious intrusion complaint, as the crawler, which obeys robots.txt, examines about 20 pages on a site in a few seconds. No more than three connections at once, but some sites are touchy. The server leasing company sends me a warning letter, I reply and call tech support, and there's no big problem.

I can recommend leasing servers from Codero as an alternative to dealing with the Borg of Mountain View. I've been a customer for five years, and nothing bad has happened. They now have "cloud services" too, but I haven't used them.

76bd 56 ago 0 replies      
The situation of having a project fully shut down without adequate information or process by which to correct it is clearly unacceptable. That said, I was involved in recently migrating to GCP and it has worked wonderfully for us thus far. We run a fairly standard java/MySQL web app, with paid silver support. Questions are answered promptly in our experience. We found compute engine VMs to be faster (and cheaper) than EC2, the developer console easier to use, and better quality documentation/APIs.
CaptSpify 21 ago 1 reply      

Sorry to be so harsh, but Google has always been this way.

Why do people keep putting essential stuff in someone else's sandbox? Your effectivly adding a SPoF, that you have no control over

aluminussoma 21 ago 1 reply      
Reaching a human being is a common problem with all Internet businesses. I've had similar frustrations with Google competitors. As an industry, we have to do much better with customer service. Our users are real human beings.
ne01 9 ago 1 reply      
I hate that Google don't give a damn about its users. To them interacting with humans is a waste of time and money... They prefer to automate everything and manage servers, robots and programs instead.

The same thing happened to us in 2013 luckily had backup servers at Linode.. took them 2 weeks to solve the problem it was a bug in their billing system.

Google gave us $8k credit and I gave them another try and convinced myself that GCE was very young in 2013.

Now I'm just worried! It's 2016 and our entire business depends on their platform and their support sucks! Even the $400/month version!!

Google, why can't you have support like DigitalOcean and Linode?

pellej_s 21 ago 7 replies      
So, everyone's basically saying:

> Google sucks

> Don't rely on Google if you want your business to succeed

> This is old news! Happens all the time

> Google = SPoF

...you're all looking like trolls to me. Please, do tell how Spotify manages to service 60+ markets with some core pieces of infrastructure on Google's Cloud offerings.

pjjw 18 ago 0 replies      
Dollars to donuts these guys got hacked. Still pretty shitty the appeal process was busted.
damm 12 ago 0 replies      
I admit to having a gmail.com account


I really don't understand how anyone can use Google for production. If you don't pay for support to have a way to actually get a hold of Google...

... Someday you may be in a world of hurt and have absolutely no way to get ahold of anyone. You can post a story and hope and pray that Google might read it and have someone reach out and remedy the situation.

zippy786 18 ago 1 reply      
Google, said to have very high hiring bar, yet I've seen more than a few times now that their product lines are sub-par. I'm sure the explanation they give is that we are very big and we can ignore the 1% corner cases and have the mentality that "we are so big we don't really care about smaller parts of the internet".

Also, the hypocrisy in using very logical questions to hire people but not sticking to similar logic in the product line is laughable.

yladiz 12 ago 0 replies      
This makes me wonder, which do people dislike more: not having access to support in almost any capacity except "help desk tickets" which might not get answered for days, e.g. how Google handles non-free support tiers, or horribly long wait times on phones? My gut says that the former is worse, especially in this case, because you can just turn your phone on speaker and wait for the queue to finish up (or with some systems, have them call you back when the queue is to you).

I almost can't believe that Google doesn't have a support line for its account holders for Google Cloud, when you have companies like Paypal and United Airlines, which have many more users (paying or not) and have support lines.

martinald 14 ago 0 replies      
We got the 100k credit which is nice. Noticed the actual technical side is excellent - vastly better than azure and AWS (AWS especially - I find the whole UI and way it works so clunky).

However, despite using our credit to buy gold support when it came to using it I couldn't work their tickets UI at all. Turns out you need to change products from your Google apps support to cloud platform by some tiny link which is not at all obvious.

I had to phone up and it was obvious people hit it all the time as the operator instantly knew what the issue was.

Also related: azure by default didnt renew your free trial despite an active cc being on file. It just shuts everything off with no obvious warning. What other service does that?

Honestly think these companies need to do some proper usability testing on their flows because there is so much clunky weirdness in cloud providers right now.

danpalmer 21 ago 2 replies      
Of course it's easy to pattern match this to previous stories of advertisers and publishers being banned from Google's ad products, people losing their email access on Gmail, etc, but this is an entirely different area of the company, and one where account managers are very much a thing. It strikes me as odd that one couldn't just email or phone their account manager to ask for clarity on a situation like this?
jeffmould 21 ago 1 reply      
Not sure if he missed it on the support page, or simply is not willing to pay the added cost, but Google does offer a paid support option with both phone and email support options. Obviously the more you pay the quicker the response and more access you have to those options.
yclept 11 ago 0 replies      
This sounds like a nightmare. One that I can't imagine happening at Rackspace Cloud...
oconnore 18 ago 0 replies      
Just wait until you can buy one of their self driving cars.
Safety1stClyde 17 ago 0 replies      
Google = rain man.
Neural network spotted deep inside Samsung's Galaxy S7 silicon brain theregister.co.uk
243 points by dragonbonheur  13 ago   92 comments top 9
hcs 13 ago 3 replies      
My Computer Architecture professor Daniel Jimenez worked on (invented?) something like this:

"Dynamic Branch Prediction with Perceptrons" (PDF)


sehugg 10 ago 1 reply      
I don't see why branching is so hard -- it's two cycles to test the branch, one more if the branch is taken, one more if it crosses a page boundary! /6502joke
dingo_bat 13 ago 7 replies      
This is literally insane. I've never heard of a neural net branch predictor.

Edit: my layman knowledge about branch prediction and neural networks is showing its gross inadequacy :/

pepijndevos 4 ago 0 replies      
The latter half of the article makes me wonder to what extend the cycle count of instructions is specified, or up to a specific implementation.
Jerry2 11 ago 9 replies      
Even with all those neural net advantages, dual-core, 2GB RAM iPhone 6s beats octa-core, 4GB RAM Galaxy Note 7:


Video (3:29 sec): https://www.youtube.com/watch?v=3-61FFoJFy0

PS: I highly recommend you check out the video... iPhone completely obliterated Note 7.

AstralStorm 7 ago 1 reply      
This is not new, AMD has been using perceptron-based predictors probably even before Bulldozer. They are good, but harder to optimise for than standard rule based heuristic ones.
mrfusion 11 ago 2 replies      
Hmm now I'm wondering if the brain does something like this? Might the brain need branch prediction? Interesting line of thought.
goombastic 11 ago 1 reply      
Why is it still slower than oneplus3 and htc 10?
ForFreedom 8 ago 1 reply      
iPhone excel because the OS and HW is designed by one company unlike Samsung where the HW and OS are from different companies, add to android Samsung bloatware.

The iOS works well with the iPhone simply because it is fine tuned for the product. On the androids side the OS is tuned to add bloatware of the respective HW manufactures. To add the so called speed the HW manufactures put in more RAM.

An American Doctor Experiences the NHS Again drjengunter.wordpress.com
244 points by petepete  1 ago   334 comments top 21
codegeek 1 ago 10 replies      
The article is nice but a specific comment caught my attention which was "I honestly live in fear of having an accident or getting sick here in America."

This. No matter what we claim about American healthcare, this points resonates so well with most of us in America. It absolutely is scary to imagine being sick even with insurance. Without insurance, forget about it. Then comes the maze of crazy terms like co-pay, co-insurance, deductible, pre-authorization, in-network, out-network, Specialist referral and god knows what else.

Now may I rant about the Insurance companies if I may ? Obamacare was supposed to fix healthcare but it added to the red tape further. No real solutions. Insurance companies still have too much power and they call the shots, not the doctors/hospitals. Good luck fighting a denied claim unless you can shell good amount of lawyer money.

I don't know the solution though. America still has one of the best facilities in the world and even though the process is too bureaucratic and downright idiotic in many cases, the care that is actually provided is still pretty good. And I say this as someone who is aware of medical tourism and countries such as India.

One solution could be to get rid of insurance scam (yes I call it a scam) and then lets hospitals/doctors compete with transparent pricing in a free market. For catastrophic events, let insurance companies exist. I hate it that to even get an X-Ray, I need insurance and without it, they will look at you as if you came from some strange land. And if you do end up getting a service without insurance, good luck looking at the bill when it finally comes. You can be a crypto expert but won't be able to figure that one out.

grecy 1 ago 2 replies      
> To receive this care all my cousin had to do was provide her name and birthdate. No copayments, no preauthorizations, no concerns about the radiologist or orthopedic surgeon being out of network

For the vast majority of people in the developed world, that's how heath care works.

brightshiny 1 ago 13 replies      
I've worked in the NHS, for several different organisations at different levels of the hierarchy (PCTs, SHAs, CCGs etc), as a contractor, over several years.

I would privatise it tomorrow. The amount of waste in terms of time and money were eye-watering, every time. The amount of politicking was immense. A project that should take a few weeks could easily expand into a couple of years. I resigned from my last contract because it actually felt immoral to be taking a large amount of cash to either sit around twiddling my thumbs, fight with people, or do work at the slowest possible pace in the most ineffective way. I'd rather do something else. Basically, I've never worked for a private company that was anything like that, even the worst ones, and I find it hard to believe any private company would survive very long working that way.

I'll add, my Japanese girlfriend and her friends are utterly unimpressed with the level of service. As was my Spanish girlfriend, who also worked on a hospital's grounds as a medical researcher.

I'm not saying we should copy the Americans, no way, but the Japanese system is private, the German system is private, the Dutch and French systems are also private. Going private does not mean chucking the idea of universal healthcare on the fire, it means properly separating the regulator from the provider and removing the kind of conflict of interest that led to the cover up of high mortality rates, among other things. There is more than one way to skin a cat.

And no, I'm not a Tory. Try picking up a copy of Private Eye if you want to divest yourself of the notion that the NHS is a saintly institution. Or work there.

Normal_gaussian 1 ago 3 replies      
> Maybe teach kids in school how to use the health care system (hey, why not NHS ed alongside drivers ed or sex ed?).

We should probably break the news to her that there is no "drivers ed" in the UK.

But she is right. The NHS can be great, but the way the government is fucking with it at the moment will prove disastrous. It is education, not privatisation, those are the answers to the problems of the NHS.

privateersman 1 ago 4 replies      
She had two options: go through the GP-based system where treatment can take anything from days to months, or take the initiative by jumping the line and go to A&E.

Going to A&E is like having your tech support handled by engineers. Going through the GP system is like calling a support line. She got a positive outcome because she went with the option that costs the most.

I'm pointing this out as someone who has gone through both methods. Don't believe the hype that you read about the NHS. It's good to have, but it's not amazing and if everyone went through A&E, the system would probably collapse.

lanestp 1 ago 4 replies      
This kind of article is a nonsense puff piece. Giving an example of receiving good or bad care in a health system is of no value. I have had incredibly positive and negative experiences with US hospitals and doctors but my experience is simply a data point.

As for the NHS the mother of a friend died due to negligence while being transported home. So it's not all good, which is why the system cannot be judged this way.

andreasklinger 1 ago 3 replies      
As an european now living in America:

The reason the EU model works is that people treat their problems early.

You have a concern - you go to the doctor.

NHS - with all it's short comes - has the same principle

Waterluvian 1 ago 4 replies      
I wish Canada's system was that good. I've been to the ER twice. 6-8 hour wait for things that weren't life threatening but definitely required an ER visit. Ever waited in an ER for 8 hours while sick as a dog? It's awful.

But then I think about how often I intend on being at the hospital and will gladly pay 8 hours every time if it means nobody has to find a co-pay to learn if their cancer is back.

madaxe_again 1 ago 2 replies      
The NHS is great at emergency care. I came down with a severe infection a few months ago and landed in Glan Clwyd. I was practically grabbed as I staggered through the door, and spent a few days in a private room (!!!) white they figured out whether what I had was infectious. Then a few days on a general ward, then discharged, well enough to finish sorting myself out at home. The staff were lovely, friendly, genuinely caring people - although I worry for their futures, given that most were EU nationals, and the only UK natives were agency staff. In my less delirious moments we chatted about the impending brexit vote - very worried faces.

When it comes to long term or weird and wonderful stuff - not so great. I've been bouncing around referrals for five years, and the lack of digital records means that they have to mail 30lbs of paper to each other before I show up. If the records didn't arrive, I have to dictate my case. Again, and again, and again. Still undiagnosed. One particularly spectacular SNAFU ended up with me having unnecessary surgery, because someone read the wrong notes.

Their core infrastructure is in dire need of modernisation. The government seem to think the answer is creeping privatisation. If they hadn't wasted 14bn on lining their chums pockets, the NHS would probably be pretty unassailably brilliant by now. https://en.m.wikipedia.org/wiki/NHS_Connecting_for_Health

DanBC 1 ago 0 replies      
Those hospital wheelchairs are difficult to use, but it's possible to safely push them.

> The only real issue was people who show up for care that is clearly not even semi urgent never mind emergent.

People do love the NHS. And they trust A&E. Companies would kill to get that kind of brand loyalty. So the answer is to co-locate walk-in doctors and minor injuries with A&E departments, and have fierce triage on entry to divert the people who don't need it away from A&E.

UK-AL 1 ago 0 replies      
I find the NHS is good for the little things, gets worse as you go for the more complicated things.

My dad had pancreatic cancer and went to the doctors, did every sort of cheap test they could. They didn't believe him, and said it was psychological and sent him to a psychiatric hospital instead. Again nothing worked

After constant complaining, eventually they went for the expensive scan, and found that it was pancreatic cancer wasting several months.


Ericson2314 1 ago 0 replies      
I know rhis isn't the main impediment to policy change, but I kinda want to require that all doctors and nurses study/be a resident abroad, just so most people in any US hospital know what we're missing.
greggman 1 ago 0 replies      
Well... Sadly I've had bad experiences

I was in London last summer. Got sick. Waited about 9 days for it to get better. No signs of change so went to NHS. Wait one hour. Was led into consultation room and was told I could wait another 90 minutes if I wanted but that I could only see a nurse and regardless of what the nurse found she could not prescribe any medicine. Or, I was told by the same NHS person I could go about a 10 minute walk to a private clinic and see a doctor probably immediately and as a doctor they could prescribe medicine if needed.

So ... Yea, wasn't a good first impression of NHS.

Not saying USA's system is better.

In Japan it's easy to see a doctor but at least in my experience about 30% of them are quacks.

Also if you need surgery and you want a good doctor you need to bribe them. This is so common it's a normal part of most medical dramas.

I'm scared of getting the giant bill in the USA. But I'm scared of getting hurt by the doctor in Japan if I need any surgery. I don't know the solution.

anexprogrammer 1 ago 0 replies      
We get a lot for the money, compared to the cost of other systems. I absolutely don't want to see more privatisation of the system despite most governments nudging towards this. All is not perfect though. Every government wants to be seen to be "doing something", so fiddles. Constantly. Staff is overworked and mostly underpaid.

Emergency and cancer care seems to be very good. I'd recommend not needing a non-emergency or mental health treatment in some of the poorer regions of the country though.

For instance. If you need Aspergers help for your child, or referral to some CBT for depression in some places you can wait 12 months. Sometimes more.

Even so I prefer it to anything like the US system as you can;t fall through the gaps entirely for being broke. I'm glad to pay more tax as a result, rather than an insurance premium or in-work benefit.

WoodenChair 1 ago 2 replies      
The problem with socialized medicine is not with regards to the routine (broken ankle, high cholesterol) but instead with regards to the even slightly unusual. The economic incentive is not there for highly trained specialists.

Nothing about this article is surprising, despite the author's tone. Any modern industrialized country should be able to quickly treat a routine broken ankle. Where that's not possible in the US it's a travesty. Now compare the care of extremely rare diseases in the US and UK and you have something more interesting to talk about.

shanacarp 1 ago 0 replies      
I actually watch this.

I'm in a private group for what is collequally called previvors (specifically Young previvors - though some people have had/current do have breast/ovarian cancer, including a few who are dying of it). I need specialized DNA testing for complicated reasons in my family history for risk assessment. It isn't covered. We have UK and European Members. I see who their doctors ares, what research they publish. I know what they pay. ($0) I know that for patients like me, European doctors are way more proactive, even for joining studies, because they can afford to be.

I know factually that if I were in europe that test would have been done LAST YEAR. Instead I'm still looking around for one doctor outside of my regular provider to do this one DNA test under research protocols.

Doing this one test could potentially save the medical system thousands of dollars. That's what baffles me the most: The insurance companies wants me to use a certain amont of services, but only so much, in the US

It is like they actually don't want me to be healthy - and I am not sure why.

neom 1 ago 1 reply      
Grew up in the UK, went to college In Canada, US for the last 6 years.

NHS was excellent growing up. OHIP/Ontario Health Care was excellent. The US is contrived and awful.

dzhiurgis 1 ago 1 reply      
Had 3 visits so far in London hospitals, regarding recurring pain around heart area. Doctor solution was making me keep pain diary as NHS has no money for diagnostics...

I did get some basic blood checks though.

ZenoArrow 1 ago 3 replies      
Many of the problems with the NHS derive from the UK government's attempts to privatise it. If you're short on time but want to understand more, look at the mess with PFI used to fund the building of new hospitals.

EDIT: If you've got more time/interest I'd recommend the documentary Sell-Off:


jkot 1 ago 2 replies      
My friend had diabetes, her GP could not even diagnose that from basic symptoms, several visits and blood test.

I strongly recommend to anyone with 'mysterious' long term illness to visit doctor in other country.

maxerickson 1 ago 1 reply      
For lots of Americans, the copay on that CT scan is 100%.

I guess if they are getting over cancer they might be lucky enough to have met their out of pocket maximum for the year.

edit: Why is this controversial? I literally paid out of pocket for 2 CT scans last year. They both applied against my deductible (so they were 'medically justified' or whatever, not done at my insistence or something). Is being sarcastic about the high cost of healthcare in the US so absurd?

Penny Auctions curiousgnu.com
337 points by tonteldoos  1 ago   152 comments top 26
tominous 1 ago 3 replies      
This is like the old trick of auctioning off a $20 note with the twist that the two highest bidders must both pay even if they end up losing. I saw this play out in a Negotiation 101 tutorial once, for real money, and the bid got up to $40 or so before one person finally bailed (i.e. around $80 spent on a $20 note).

It's a game of chicken. The earlier bids are a sunk cost. You hope you can spend an extra $1 to win the $20.

Sounds insane to even start bidding, right? But then you see things like this play out all the time in the market. For example, the war between Blu-Ray and HD-DVD had a very similar dynamic.

EDIT: See https://en.wikipedia.org/wiki/Dollar_auction

dividuum 1 ago 0 replies      
Story time: A couple of friends and me played on a competitive Minecraft server for a while. You could form factions, build protected bases and compete on various player vs player events. The last part didn't interest us at all, since we were all a bit older and went there to relax after work. So we did all kinds of other creative things like automated shops using redstone. The server also had an in-game currency which you could only obtain by mining. One day we got the idea to do a penny auction to finance the expansion of our protected area. So I quickly put together a Flask based site that parsed the Minecraft client chat log in realtime. Other players would send me in-game money which would result in a chat message my locally running python script would pick up. Everyone could then see the auction happening in realtime on the website (Example screenshot: http://i.imgur.com/yg2V3iS.png). It was really successful, everyone had fun and hopefully learned that you should never bid on penny actions :-)
legohead 23 ago 2 replies      
I worked for one of the biggest penny auction sites. I'll address some things in this thread.

One key thing the author left out was "free bids". You can win auctions that give you bids. So, for instance, you spend 10 bids and get lucky and win a 2500 bid pack. When bidding on the auction, an outside observer can't tell if you're using a bid you paid for or a "free bid" you won on the site or gained in some other mean (some auctions may be a product + free bids). So while it looked like the guy spent $3,500, he may have only spent $100 of his own real money.

Some sites even stop you from spending over the retail value of the site. The one I worked for let you buy the item if you spent bids that equaled the retail value. And we were fair with the retail value of items. We had people whose job it was to source products so we could afford to sell items at retail. If you lost the auction, you could also pay the difference between the value of your bids and the retail price and buy the item that way. So if the item was $150, and you spent $100 in bids but lost, you could pay $50 and get the item anyway.

Yes, some of these sites use bots. You can buy pre-built penny auctions sites and you will see they contain bots. The site I worked for never used bots while I worked there. I'd be surprised if any of the bigger sites use bots, as it's not needed and can be pretty obvious to detect with basic statistics. And you definitely have a certain type of person on this site who is running statistics and trying to beat the game.

These sites can actually be used to find a good deal, if they offer the retail option I mentioned above. Imagine you plan on buying a 65" TV. You try out a penny auction site and win it for $100, or end up having to pay retail for it anyway. The main loss would be paying shipping, which admittedly for a TV would be a lot. Personally, I never used one of these sites. It's just too stressful and slow for me (auctions can last for hours).

air7 1 ago 1 reply      
Similarly, I once also looked into such a site, collecting data from on-going auctions. While the interface showed very little information about the bidders, the underlying JSON data carrying the real-time bids had addition fields such as user ID.I remember that while my user ID, along with other players that seemed human (i.e few and far between bids) had user ids that were 2000+, there were several users, that bid thousands of times, sometimes for 16 hours straight (there was no auto-bid option), all of which had a user ID of 1000-1010...
grantcox 1 ago 6 replies      
If one bidder spent more than $3500 in bids for that tablet, it's either a shill account or a credit card scammer. Neither one is particularly surprising with sites like this.
riceslush 1 ago 1 reply      
Any sufficiently advanced technology involving money is indistinguishable from gambling.
zekevermillion 1 ago 3 replies      
I remember a game circa 1980s called Inside Trader, a stock market "simulation". There were a bunch of fake stocks that fluctuated in value randomly, and one could choose to trade honestly or based on tips. Tips carried the risk of penalties and jail (losing the game). I eventually figured out that while the prices did go up and down, they never went below zero. Therefore, one could trivially beat the game by purchasing penny stocks. Unfortunately, the real market does not have a meaningful lower bound on losses.
lubos 1 ago 0 replies      
Relevant: Profitable Until Deemed Illegal (2008)


wiredfool 1 ago 1 reply      
It's entirely possible that it's scams all the way down.

In recent memory, there was one penny auction site that was essentially a justification for a ponzi scheme, where people were buying thousands of dollars of "bids" (not using cc, but cashier's checks and money orders. i.e. non-reversable funds) because there would be riches of payouts on the other side.

chii 1 ago 1 reply      
It would be interesting if someone created a site that helps users of these penny auction sites collude. Imagine, if the site queued up each user as they registered, and the user as the head of the queue gets a guaranteed win on the item because everyone else stops and don't bid... So if you wait in the queue long enough, you'll get your turn and get the item really cheap.
foota 1 ago 1 reply      
Interesting it seems that running for a political office is a type of penny auction.
pkamb 21 ago 3 replies      
> each bid adds ten seconds to the countdown which gives other bidders the time to counter your bid.

Why has eBay never implemented this behavior?

metafunctor 1 ago 5 replies      
In game theoretic terms, is there a best strategy for a game like this, other than not playing?
rtpg 1 ago 9 replies      
What's the argument that this is gambling? Or rather, that this is gambling while eBay is not?

I understand on a rational level that this is less fair to bidders and a bad model if you want to just buy things, but I don't see where this is a game of chance instead of a consequence of who is participating.

lrem 1 ago 4 replies      
How does one person spend 3500$ on a 180$ tablet: write a bot. You tell it to "buy any item at 10% retail price if the remaining time is 1s". Sounds reasonable, even if you pay a few cents for the losing bids, right? ;)
barrkel 1 ago 1 reply      
I feel like there's a bit of explanation missing in this article. It only seems to make sense if bids cost a penny at the time they are made, rather than needing to pay at the time the auction is won. Is that the case?

Does the winner have to pay the existing bid total too, or do they benefit from the spending if other bidders?

Wikipedia has a better and more complete explanation: everyone pays, both for bids and the total:https://en.wikipedia.org/wiki/Bidding_fee_auction

sailing 20 ago 1 reply      
Theory 1: "Arsenic" spent $3500 to lose an auction for a $180 tablet.

Theory 2: "Arsenic" laundered $3500.

I'd wager a penny it's #2.

jrs235 1 ago 0 replies      
The other hidden scam in this is that [some/many] these sites will give you X number of free credits when you first sign up. Not only is this to get you hooked and using the site but it cheats people who are already hooked and paid for their credits because it allows others to bid and "take away the winning bid(s) of actually paying customers) for free!
ryporter 1 ago 3 replies      
While I certainly think that Penny Auctions are less than useless, I really don't see how they can be classified as gambling. I think that the salient point here is that there is no element of chance. Sure, you don't know how your opponents will behave, but you also don't know how your opponent in a chess game will behave. The nondeterministic behavior is not endogenous to the game itself.
thomasrossi 1 ago 0 replies      
In some country that kind of auction is considered gamble indeed and it's against the law (most EU as far as I know). From a Mechanism Design point of view, they are maximizing their own profit and they also expect their players to be not very rational entities and so they don't nee to use a truthful auction.
grenoire 1 ago 2 replies      
Having seen all the gambling site scams for Valve games (see CS:GO streamers faking wins on their affiliated websites), it would come to me as no surprise that they are abusively pushing users to bidding wars by counterbidding 'fake money.' If they win the bet themselves, they can always reauction the item at a later date.
philliphaydon 1 ago 0 replies      
There are also reverse Dutch auctions which you bid to drive the price down. The winner ends up paying the remaining cost of the item. You buy a bid for 50 cents and the bid drives down the price by 2 cents. If the remaining item is $15 for a $2000 item. You pay $15.

It's all a scam tho imo.

gedrap 1 ago 1 reply      
It's an interesting topic but I unexpected some analyses or conclusions at the end :( maybe the OP is going to release the raw data for the interested? Quite curious how would that turn out.
DoubleGlazing 1 ago 0 replies      
Seems like a good way to launder money.
chinathrow 1 ago 0 replies      
Wow, I would love to see someone filing suit and seeing them subpoenad to discover the shill accounts linked to it.

There is just no way that Arsenic is legitimate.

thalesfc 1 ago 1 reply      
Excellent content. I was surfing your web page and it seems ridiculous interesting. Tomorrow I will read everything, do you make all your source code available ?
Computers do better than pathologists in predicting lung cancer type, severity stanford.edu
195 points by rch  22 ago   56 comments top 13
Strilanc 22 ago 2 replies      
Yet another press release that fails to reference the study it's talking about. Not to mention giving no quantitative details about its lead ('Trounced'? By how much?).

The study is "Predicting non-small cell lung cancer prognosis by fully automated microscopic pathology image features" by Kun-Hsing Yu et al. It's open access: http://www.nature.com/ncomms/2016/160816/ncomms12474/full/nc...

aschampion 21 ago 3 replies      
I worked on a similar pipeline for renal cell carcinoma a few years ago [1], although we only published a small subset of results since parts of the pipeline (e.g., finding representative tiles, survival prediction) had better results being produced elsewhere in the lab.

Regarding the hook in the headline -- computers surpassing pathologists -- it's a bit like automated driving in that even if true the immediate problem is the social and economic system. That is, we're not going to be removing the pathologist from the diagnostic and prognostic process anytime soon for many reasons, so how instead do we leverage machine learning in concert with the human observer to improve the diagnostic system? For that reason, decision introspection may be as valuable a topic of research as improving classification accuracy: justifying a particular automated classification to the pathologist, directing them to representative regions, and describing regions of feature space in biological terms.

[1] http://ieeexplore.ieee.org/xpls/abs_all.jsp?arnumber=6945104

clumsysmurf 22 ago 2 replies      
My friend worked in a pathology lab, run by a recognized world expert in the field (wrote college texts, gave lectures, etc) yet the performance was 'mediocre' due to reoccurring mistakes made by humans:

* Doctors disagreed with each other frequently. Most senior won.

* Results from one patient were given to another patient

* ... or results were lost entirely

* Lab destroyed samples before examination or stained incorrectly

* Samples never received, lost in mail or never picked up from airport

These folks did have a tough job, looking into microscopes for many hours per day. At some point I imagine things just start looking the same as fatigue sets in.

muxxa 21 ago 3 replies      
I wonder how the pathologists would fare if they were also put through the same process as the ANN, i.e. given training data along with immediate feedback on whether their prognosis was right or wrong, then tested on the reserve data. Pathologists give daily prognosises but only get feedback, if at all, many years later.
yread 20 ago 0 replies      
It's nice that the article is open access and even includes the R code use to perform the analysis

[1] http://www.nature.com/ncomms/2016/160816/ncomms12474/full/nc...

[2] http://www.nature.com/article-assets/npg/ncomms/2016/160816/...

Fede_V 22 ago 4 replies      
I am surprised they only used ~2000 samples. That's a very tiny amount of data to train a decently sized CNN, even with data augmentation.

I presume using a pretrained net probably won't help you since the features found in pathology slides are so unlike those of natural images.

charlesism 21 ago 1 reply      
The world is full of constant hype about "breakthrough" studies, and cures for diseases that are "coming soon". Somehow the cures seldom materialize, and too often, if anyone bothers to check, they can't replicate the studies.

I wonder if the secret sauce that makes science effective isn't so much the scientific method, as the spirit of skepticism behind it. Without that, it's easy to make errors. And wishful thinking is pervasive because there are strong incentives: career advancement, corporate influence, ego, and so on.

My view, assuming AI doesn't kill us, is that it's going to save all of our asses (from ourselves).

WhitneyLand 21 ago 2 replies      
Why isn't this already is practice given ML advances in recent years?

Why are mammograms still being subjectively interpreted?

Are there not already companies today where you can upload medical imaging results and get back results that beat humans?

danielrm26 20 ago 0 replies      
Human error comes from a lot more than just pure assessment of whether something is cancer or not.

Fatigue, willpower drain, emotion, and many other factors can make a top-tier human expert into a mediocre one (or worse) in a way that is very hard to detect.

Even if computers aren't perfectly matching the ideal human (which they will inevitably pass, of course) they're still massively valuable in that they can maintain their level of expertise when humans cannot.

tvladeck 12 ago 1 reply      
The article mentioned they only had a dataset of 2,186 slides. Isn't this a _tiny_ dataset to do any learning on images? They also mentioned they blew up the predictors to over 10,000 features. This sounds like a recipe for overfitting to me. I can't access the actual paper (getting a 500 error), but does anyone know how they built these algorithms?
njharman 21 ago 2 replies      
Um, computers do better than humans in the vast majority of applications to which they are programmed for.

By which I mean, the news is not that computer > human, but that medical industry/profession/jobs are taking first baby steps at being automated away.

nikcheerla 13 ago 0 replies      
There's a LOT of biological data out there. With a little bit of effort, researchers can design AI systems that beat doctors at most specialized tasks, in both computer vision and genetic data analysis. The question is how such systems can actually be used in practice.
shazam 19 ago 2 replies      
Does anyone know about how to get involved in work like this? I'm outcome-oriented so disease diagnosis would be motivating for me. I'm looking to help out on an open-source project along these lines.
Kubernetes the Hard Way github.com
236 points by tantalor  3 ago   77 comments top 12
anemic 3 ago 9 replies      
Lots of good information here but it is still not enough for a production setup IMHO.

There is a great need of good source of production setups. In open source software this seems to be the secret that no one is willing to reveal. I tried to setup a kubernetes cluster from scratch a while ago and soon I was browsing the source code for answers. Openstack is the same, you need to understand a lot about the inner workings before you can even attempt to setup something for production.

There is always a simple "this shell script starts your own <name your tech here> cluster in vagrant" but it is still not a production setup.

And even if this article is the "hard way" it describes:

> This is being done for ease of use. In production you should strongly consider generating individual TLS certificates for each component."

And it does not mention that the crucial part is the common name field in the certificate maps to the user name that is the magic information that I once needed.

I sincerely appreciate this article but production setup is still a long learning experience away.

mattikus 3 ago 3 replies      
Just saw Kelsey give a talk at Abstractions about more advanced patterns in Kubernetes and he mentioned this repo. Looks like a fantastic tutorial and his talk was very informative.

Highly recommend watching the video when it's released if you weren't able to attend.

marcoceppi 3 ago 0 replies      
This is a great starting point. We're been running Kubernetes in production alongside an OpenStack for a while and charm'd up the deployment: https://jujucharms.com/kubernetes-core. The majority of the information here (and more) seems to already be encapsulated: `juju deploy kubernetes-core`. Since we need things like logging and monitoring, we bolted the elastic stack on the side and called it observable kubernetes: `juju deploy observable-kubernetes`.

While one-liners are typically pretty limited, the charms come with quite a few knobs to help tweak for deployments.


There's still room to improve, but we've been happy with the cluster so far. Considering Juju and charms are open source. Eitherway, great guide for those getting started.

jondubois 3 ago 1 reply      
The article mentioned Google Container Engine as one of the 'easy ways' but it didn't mention Rancher http://rancher.com/ - This is not quite as easy as GCE but I found it pretty easy.
mmagin 3 ago 0 replies      
So, I see something like this, assume they mean "from scratch", read a little way down the README and it says "The following labs assume you have a working Google Cloud Platform account and a recent version of the Google Cloud SDK (116.0.0+) installed."

The irony.

nathan_f77 3 ago 0 replies      
Here is Kubernetes the easy, production-ready way: https://github.com/kz8s/tack

I've spun up a few clusters using this, and I absolutely prefer learning this way. I love to get something running before I dive in and look at all the pieces and individual options. I also really love "convention over configuration". I want to study a production-ready reference implementation that simply works, with sane defaults.

I've spent a lot of time banging my head against the wall while I try to follow some complicated tutorial that doesn't work with the latest versions. Maybe this approach works for others, but it's not for me. I like to stand up a cluster, deploy a database and a real application, try to scale it, set up some test DNS records, do some load testing. Figure out the pain points and learn as I go. If something just works perfectly fine behind the scenes, then I probably don't need to learn that much about it (or I have enough experience that I already know what it does and how it works.)

This might not be a suitable learning style for a beginner, but I think I have enough experience working and experimenting with AWS, Saltstack, Chef, Puppet, Ansible, OpenStack, Deis, Flynn, Terraform, Docker, CoreOS, etc. etc.So at this point, I just prefer to evaluate new technologies by spinning them up and diving straight in.

philips 3 ago 0 replies      
For Kubernetes the easy way checkout this blog post on the work going on upstream for "self-hosted" Kubernetes: https://coreos.com/blog/self-hosted-kubernetes.html
readams 3 ago 1 reply      
It would be nice if all the Kubernetes tutorials didn't assume you're using GCE.
felixgallo 3 ago 0 replies      
Kelsey is a national treasure. Kubernetes is getting pretty close to being ready, if it can avoid the same fate as OpenStack and the like. Interesting times.
bdcravens 3 ago 1 reply      
What's nice about this is it seems to teach you without taking a GKE-first approach, like some of the other tutorials out there.
user5994461 3 ago 1 reply      
Looked at the documentation and quickly gave up on kubernetes. It's nice and it solves real problem but the barrier to entry is INSANE.

And it's lacking wayyy to much documentation deploy in production on own cluster. It's probably gonna take years to improve.

I'm wondering. Did anyone tried the kubernetes on GCE?

If Google can handle all the annoying setup and makes a mostly managed service, that would be extremely attractive. Actually, that's probably the only way k8s would be achievable in production, i.e. have someone else do it.

ldehaan 3 ago 0 replies      
this seems interesting.I have to say I have heard a lot of people talking about kubernetes but few actually using it in production.

for your CI woes there is a system that isn't hard to setup and is actually really easy to use:Mesos and Marathon with weave (without the plugin, for now) and docker.

Your biggest challenge is learning zookeeper, but really if you're dealing with large scale deployments, you're probably already using it or something like it.

there are puppet/chef/ansible modules for installing and configuring mesos and zookeeper.

toss in gluster as a storage driver in docker and you're pretty much ready to go for most types of application deployments using docker.

heck, there's even kubernetes integration if you're really hung up on it ;)

EpiPen Price Rise Sparks Concern for Allergy Sufferers well.blogs.nytimes.com
215 points by helloworld  15 ago   293 comments top 23
grandalf 1 ago 5 replies      
Someone might reasonably wonder why a federal task force is required to investigate "price gouging" for an item that contains under $10 of materials and is not mechanically complex.

Why wouldn't an entrepreneur simply introduce a competing epi-pen design and bring it to market for $300 (for example) and undercut the price being charged by Mylan?

We're told to hate the Martin Shkrelis of the world, and to resend the Mylans, but the truth is that our system prevents price competition because it is designed to protect established firms. This is why many established firms and "industry leaders" strongly supported the affordable care act.

ascorbic 4 ago 5 replies      
Meanwhile, in single-payer Britain the NHS pays 26 ($35).


gmisra 14 ago 1 reply      
One of the early on-line responses was "maybe we can figure out how to 3D print these cases". It turns out that an Epi-pen is just a plastic safety housing around a normal syringe.

Some discussions:



kabdib 13 ago 2 replies      
If your kid is allergic and needs an EpiPen (or something like it), most school districts will not allow the child to attend class unless they have one at the school. Maybe two.

And of course you have to carry them. There's another two.

So that's $2400. Out of pocket if you don't have insurance, or if your insurance doesn't pay.

The EpiPen manufacturers are abusing their monopoly.

invaliddata 12 ago 3 replies      
The adrenaclick generic is much cheaper than the epipen, with or without insurance. Although technically different epinephrine auto injectors cannot be substituted (you can only be prescribed the one written by your doctor, or its generic equivalent), practically speaking the only difference between the epipen and the adrenaclick design is the instructions for use. I would hope there isn't a case of doctors being unaware of the alternatives.
BryanBigs 1 ago 1 reply      
Since Sanofi previously had an equivalent product on the market for at least 3 years (called Auvi-Q), I'm really curious - why were people paying up for an Epi-pen? Was it the brand name? Do schools mandate it's an Epi-pen?I know Sanofi pulled the product in October of last year, but it seems like Mylan has been raising prices during that time.
sevenless 14 ago 3 replies      
How many people will this price hike kill? I'm sure that's estimable.

What would be morally justified, to stop it?

grandalf 1 ago 0 replies      
A friend of mine has a child with an allergy and was visiting Turkey. A family member of his (in Turkey) is a physician, and said that in Turkey nobody uses epi pens and allergies are generally never the cause of visits to the ER.

Is this true? What's going on?

xnor 13 ago 2 replies      
If people don't want to buy two epipen there's always that neat trick for multiple doses in precarious locations far from hospitals.

It is a well known hack taught in wilderness medicine training.


eliben 14 ago 1 reply      
If the product is decades-old, why haven't generic variants appeared?

Something seems to be missing from this article.

hoodoof 14 ago 0 replies      
Mylan refers the press to their local politicians "go talk to them about it, we pay our lobbying fees to get whoever we want elected specifically so we can charge any price we want, you can't do a damn thing - it's a free country".
helloworld 15 ago 8 replies      
Why haven't competitors entered the market with cheaper products?
kuon 5 ago 1 reply      
Is this only in the US?

I'm in Switzerland, and I bought some for my kid, I paid around 80$ per piece.


You have the price at the far bottom of the page, around 76CHF.

I'm a bit concerned if the price is going to be x10.

samgranieri 1 ago 0 replies      
Oy. As someone who carries two of those damn things (grumble grumble Peanut allergy grumble grumble) and is a very conservative eater, this is not good.

Luckily, I haven't had to use one of them in like 8 months.

ohitsdom 13 ago 0 replies      
The petition to Congress mentioned in the article:


gkanai 13 ago 2 replies      
This is a simple example of how health care systems should be nationalized. Or that the profit motive in health care is corrosive to health.
Mandatum 14 ago 1 reply      
They don't produce EpiPen's as a generic in India because the Indian medical community does not recommend self-administration of epinephrine.

If you're that much in need though, you can buy pre-filled epinephrine syringes for intramuscular injection - just keep in mind the ease-of-use of an EpiPen won't be available. You have to take the time to take off the cap, insert and inject the medicine.

shanacarp 14 ago 2 replies      
This is one of those devices that also should be low cost because businesses, locations, ect should have them as well as individuals (many people can have first time deadly allergic reactions, including as adults)

That the price for the case is expensive just rose as random is serious business. There really is no need for it to be so expensive

nilved 13 ago 3 replies      
This seems less damming to me considering the medicine naturally occurs inside your body. What reason is there that nobody else can make a competing product for cheaper?
serge2k 14 ago 1 reply      
Well you know, they need to recoup the R&D costs to develop the drug.

> Mylan, the pharmaceutical company, acquired the decades-old product in 2007, when pharmacies paid less than $100 for a two-pen set


This is why US health care is broken people.

tomohawk 14 ago 6 replies      
The US is subsidizing the rest of the world. When Canada decrees that the price of something will by X, then guess who pays?
glusterfuck 10 ago 1 reply      
These are $125 for two pens or thereabouts in the UK, same device. Presumably import is regulated?


The Elegance of Deflate codersnotes.com
246 points by ingve  1 ago   82 comments top 11
beagle3 1 ago 4 replies      
There were a lot of encoders at the time using this general scheme (a few more values to indicate match length or distances). PKZIP won at the time because it was faster, and PK had the name recognition from his PKARC, which was a superfast implementation of SEA ARC (the dominant archiver on PCs at the time).

PK had to stop working on PKARC because of a C&D request from SEA. He wrote the first algos of PKZIP, which were on par with SEA ARC on compression (and with PKARC on speed), but weren't much better. (And have been deprecated since the 1990 if I'm not mistaken).

Then, the Japanese encoders started to rule the compression ratio (and had comparably reasonable compression times) - LHArc, LZAri, don't remember the rest of the names. LHArc or LHA (don't remember which), had basically the same scheme that PKZIP converged on, except it used adaptive arithmetic coding. PK replaced that with static huffman coding, trading a little compression for a lot of speed, and the format we now know and love as "zlib compression" was born (and quickly took the world by storm, being in a sweet spot of compression and speed).

There's another non-trivial thing that PKZIP had going for it - it put the directory at the end, which meant you could see the list of files in the archive without reading the entire archive! This sounds simple, but back then everyone adopted the tar-style "file header then data" appendable style, which meant that just listing the files inside a 300KB zip file (almost a whole floppy disk!) meant reading that entire floppy (30 seconds at least). PKZIP could do it in about 2 seconds.

sebular 1 ago 10 replies      
Really enjoyed reading this, I think I'd like to look more closely at an implementation to understand it further.

Reading the article, I was reminded of a nagging question I've had in the back of my mind for a bit.

The ASCII table was invented in the '70s, when the cost of disk storage was much higher than it is today. But it's a shared dictionary, a standard that we can all agree upon, something that's already on every computer.

The thing I've wondered about is whether there could be any advantage to creating new generations of shared dictionaries that are domain-specific, and much larger.

For example, in the specific (and over-simplified) case of transmitting large quantities of English text from a server to a client, you could reduce the amount of data sent over the wire if both parties shared a lookup table that contained the entire English dictionary. In that case, you wouldn't transmit a book as a collection of characters, but rather as a collection of words.

Furthermore, it would seem like you could apply the same traditional compression methods like what's described in the article to furthermore reduce the amount of data being sent. Rather than identifying repeating patterns of letters, you would identify repeating patterns of words.

Of course, the obvious drawback is that an English lookup table is useless for transmitting anything other than English text. But again, disk storage being as cheap as it is, I wonder if it wouldn't be such a monumental problem to store many domain-specific dictionaries.

Of course, you'd always want to keep the ASCII table as a fallback. Much in the same way that a conversation in sign language (ASL, specifically) is largely composed of hand gestures referring to words or concepts, but the language still includes the complete alphabet as a fallback.

The thing I don't understand well enough is whether modern compression already incorporates concepts that are similar enough such that a shared-word dictionary would be useless. It seems like the "LZXX" style of compression essentially includes a similar but dynamically-generated sort of dictionary at the beginning of the transmission, and subsequently refers to that in order to express the message itself. Would the gains of having enormous shared dictionaries cancel out the advantage of that approach, only in a much more "wasteful" way?

Dylan16807 1 ago 0 replies      
It's a good system. Though the full DEFLATE spec has quirks like storing literal blocks prefixed by their length and then the ones complement of their length.

The next step in improving a simple compression algorithm is to replace Huffman encoding with arithmetic/range encoding. Huffman encoding proceeds one bit at a time, treating everything as having a probability that's a power of 1/2. Arithmetic/range encoding uses more accurate probabilities and then packs them together, letting you save a fraction of a bit with every symbol. As an analogy, consider how to encode 3 decimal digits. You could spend 4 bits on each digit, and it would take 12 bits total. Or you could combine them into a single 10 bit number.

brian-armstrong 1 ago 2 replies      
I really enjoyed this article. It's just enough detail to leave you thinking, "Hey, I should write my own DEFLATE library"
xxs 1 ago 2 replies      
The article mentions LZW. In terms of algorithm elegance LZW should be head and shoulders above 'deflate'. LZW got patented which (aside gifs) screwed up its popularity. 'defalte' allows custom dictionaries that adds some extra complexity.

I'd strongly disagree with In many people's eyes compression was still synonymous with run-length encoding as arj was extremely popular to the point it was used to cure full stealth viruses (v512 for example). hard to provide references now but wiling to search some old bbs

Other than that - fairy nice/nostalgic article.

jamesfisher 23 ago 2 replies      
> This paragraph has five-hundred and sixty-one lower-case letters in

It must have taken some trial-and-error to make this sentence correct, since the number itself adds some letters.

shenberg 1 ago 1 reply      
The most clever part of deflate is actually how they store the huffman trees without storing the frequencies but rather the length in bits of each symbol. Combining the length of each symbol with its lexicographical ordering is enough to uniquely specify the actual tree.

Ok, some research showed me that the idea can probably be traced back to Schwartz & Kallick 1964 (can't find the text), popularized by TAOCP vol. 1...

Jasper_ 1 ago 1 reply      
I wouldn't say that entropy coding is "separate" from dictionary / window compression. DEFLATE surely isn't the first algorithm to combine LZ77 and some form of entropy coding, and surely won't be the last.

I highly recommend that everybody go out and build an INFLATE implementation; it's a lot of fun.

Mine can be found at https://github.com/magcius/libt2/blob/master/t2_inflate.h

flohofwoe 1 ago 6 replies      
Anyone know what's the history behind that setjmp/longjmp as 'poor man's exception' error handling? Since C works perfectly fine without exceptions, why add such a hack? I've seen this in libpng and I think libjpeg has it too (one of many reasons I'm avoiding those libs).
banachtarski 1 ago 3 replies      
Using additional bits to encode lengths isn't really that different from having a marker indicating if the payload is a match vector or not. In fact, it could be argued to be less efficient for certain types of data depending on the huffman distribution since all "words" are larger than a byte. It also prevents you from encoding longer matches. I treat the decision as a compromise between the two extremes that "worked" for the majority of types of data "good enough" for some definition of both quoted words. The entire idea works well enough but I wouldn't go out of the way to declare it a marvel of engineering or anything.
mozumder 1 ago 1 reply      
Deflate is so ubiquitous (being a web standard) that I'd like to see a hardware Deflate encoder/decoder.

It really needs to be in the core instruction set of x86 & ARM.

The Ethernet PAUSE frame jeffq.com
252 points by quandry  16 ago   44 comments top 12
francoisLabonte 11 ago 4 replies      
Ah... The problems of crappy consumer ethernet equipment ( I work at an ethernet switch vendor so excuse the rant )

What is likely happening is that your switch is configured by default to implement both rx and tx pause. What is happening is that your TV who's also erroneously ( in my opinion ) configured to transmit pause goes bonkers, starts sending pause to your switch. Your switch then starts buffering packets for your tv until the buffers are full and then starts transmitting pause to everyone else including ports. The switch must have some horrible buffering policy where one port ( the tv port ) can hog all the buffers and deprive every other ports of being able to send...

Now the kicker is that the way every endstation implements pause is this... Notice the pause quanta in the Pause packet is in units of 512 bits and in the packet you captured it is set to the maximal value of 65535 which is on a 100Mbps port ( presuming 100Mbps since the Mediatek has 4x100Mbps (Fast Ethernet) and 2x1Gbps ( Gigabit Ethernet ) that computes to >>> 512*65535/100.e6 = 0.3355392seconds

A normal Pause sends this packet periodically and once it has buffers to receive will send a pause with a quanta of 0 meaning cancel previous timer... but if it's malfunctioning who knows if it ever will...

The sad part is that I don't even know what to recommend for a good consumer level switch that has good defaults or configurable defaults and sane buffer config... Mine is a dinky one probably vulnerable to this problem as well... Need to do some research.

ChuckMcM 15 ago 1 reply      
Also one of the only ways to negotiate your way out of a spanning tree broadcast storm. Generally the firmware on the MAC will reflect a pause frame to the source when it's FIFO is full. That happens because the host is not pulling packets out of the FIFO fast enough, or the network has gone bonkers and is sending a gazillion packets per second.

The latter can happen when your misconfigured DHCP server gives out an address that other nodes on your network believe to be the broadcast address for the subnet. The device with that ill fated address will get deluged after every packet they send as people ack or nak or respond with queries. I saw that happen when a NetGear router had a netmask of which the user copied from the WAN config to the LAN config, but the DHCP server was told the netmask was Hilarity (not) ensued.

rdtsc 14 ago 2 replies      
Nobody knows about PAUSE frames until they bite you.

I found out about them when someone at a place I worked wanted to design a custom Ethernet driver for an embedded device. There was no good reason for it, could have run the regular one shipped for that device (it was an RPi equivalent kinda unit).

So there they went and months later, it emerged. Everyone was amazed: oh wow handcrafted Ethernet driver, impressive.

Except what ensued was months of debugging and wireshark captures. Not handling PAUSE frame and flooding the network with packets took a good chunk of that time. Of course it was blamed on stupid switches and broken protocol and not on the bad decision to re-write a known, well defined and stable protocol without a good reason to do so.

martyvis 14 ago 0 replies      
The PAUSE frame is meant to be sent by a station (host) to the switch (or vice versa) as a flow control mechanism, only for that port. Assuming the switch has at least some egress buffering, it shouldnt result in propagation away from that switch port, to say the switchs uplink port, unless the switch finds itself completely congested. Most hosts wont have flow control configured at layer 2, instead relying on TCP congestion control. It is only useful when you have non-TCP type traffic, for instance fibre channel over ethernet, and you want to avoid packet loss and prefer to try force buffering upstreamReply
balabaster 1 ago 0 replies      
Does anyone know if this is controllable by software?

i.e. is it something DD-WRT, Tomato et. al. can alleviate?

I have been suffering similar symptoms on my home network and I also have a Sony Android TV, though to be honest it hadn't occurred to me to bust out Wireshark to figure out what was going on. In hindsight, I guess this was a rookie mistake on my part.

drewg123 13 ago 0 replies      
The only "safe" way to configure traditional (not per-priority) pause frames is to configure the switch to ignore pause frames coming from hosts, and to configure hosts to obey pause frames coming from switches. With data center bridging and per-priority pause, some of that goes out the window.

In the early days of 10GbE, I did drivers for a NIC that had a very small rx fifo. In some cases we had to advise customers to enable pause frames, otherwise the NIC would be subject to tail drops when the switch burst traffic to us. I still feel kind of bad about giving out that advise.

0x0 15 ago 3 replies      
That's some great detective work.

But why is the Android TV spamming these pause frames at all?

dfc 2 ago 0 replies      
I always thought you were not supposed to enable flow control on a network with mixed 100 and gigabit devices. From the list of things OP has hooked up to network I would be surprised if they are all operating at 100 megabits.
vardump 8 ago 0 replies      
L2 pause frames are a necessity for small networks that don't have server-grade hardware, when your MAC receive buffers are measured in a few kilobytes. That includes most embedded devices (consumer devices, printers, etc., even industrial), consumer/small business switches, they have tiny buffers.

Your datacenter/server/workstation hardware is different. It deals with higher speeds and has appropriate buffering and control.

I wish people here would understand consumer/embedded space needs pause frames to function properly and that TCP congestion control will often significantly hurt performance otherwise. TCP can't magically know when some buffer is full. Without pause frames low level hardware will send at full throttle. It's not ok if every second frame is lost.

RKearney 15 ago 3 replies      
I wouldn't call this obscure by any means. You'll find Ethernet flow control enabled on just about every datacenter network, especially those that have a combined network and storage fabric.
signa11 11 ago 0 replies      
what if folks start sending PAUSE frames when on wifi ? :)
bogomipz 8 ago 2 replies      
"The very existence of Ethernet flow control may come as a shock, especially since protocols like TCP have explicit flow control"

Not at all. Ethernet is ancient and there are other transport protocols besides TCP that can and have used it in the past Apple Talk, IPX/SPX, DecNet to name a few. This is the beauty of the OSI model. Ethernet at layer 2 is independent of what rides it at layer 4.

Former employee sues startup for financials in order to value granted stock wsj.com
229 points by nstj  4 ago   66 comments top 5
chollida1 3 ago 6 replies      
The article links to the actual law cited, read it, it only takes a few minutes.


It does make it seem very cut and dry that any employee who has a single share is allowed to view not only the financials but also the cap table and shareholder list.

To be honest, if this is the case I'm very surprised that fund managers like T Rowe Price and Fidelity haven't been invoking it for as long as they've held shares to do their monthly marking of their private positions. I mean, doesn't it seem like they actually have a fiduciary duty to do this as it lets them most accurately value the holdings that they have?

If this is really the case then, unless the companies can force some sort of NDA on shareholders, then this opens the floodgates where any company that has vested shareholders, not just option holders, is going to have its books made public?

This seems like a positive thing for employees as each new round would be scrutinized looking for provisions in the new raise that could screw employee's ie ratchets or liquidation preferences.

will_brown 3 ago 0 replies      
Assuming the facts as true, the shareholder might also have a claim for retaliation against the company, specifically:

1. Pursuant to DE law shareholder requested the books be opened to value his shares;

2. His request was not just unlawfully denied, but company in writing explained, by law shareholder was not entitled to the requested documentation;

3. Then "a few days later" the company fired the shareholder in his capacity as employee.

Could be a great claim for retaliation, though not the federal title 7 type, but the DE state equivalent.[1] Note the citation includes whistle blower claims, which is similar/associated with retaliation claims but they are separate. Basically the DE retaliation claim is triggered by the employer taking action contrary to DE public policy (i.e. Being fired for trying to exercise a right as a shareholder)

[1] http://www.workplacefairness.org/whistleblower-retaliation-c...

gjem97 3 ago 1 reply      
"Mr. Biederman holds 64,166 Domo shares that would be worth $540,919 at the $8.43-per-share price where Domo sold stock to investors last year."

Is there any reason to believe that these are the same series of shares? Isn't it more likely that the investors bought preferred shares that are worth a lot more than the common shares that he probably has?

koolba 3 ago 1 reply      
So if one exercises even just one of their stock options would they have full access to the cap table for a standard Delaware C-corp anytime they want to value their shares?
justinzollars 3 ago 0 replies      
This is cute. Go to Domo website: "Information is empowering."
       cached 23 August 2016 15:11:02 GMT