hacker news with inline top comments    .. more ..    24 Jul 2016 Best
home   ask   best   3 years ago   
1
Why Im Suing the US Government bunniestudios.com
1803 points by ivank  2 days ago   299 comments top 36
1
DoubleGlazing 2 days ago 7 replies      
Good.

My wife is a speech therapist and uses a system that is designed to help people who have had strokes regain their voice.

It comprises a piece of software that comes with a "specially calibrated USB microphone". The microphone is actually a Samson laptop USB mic that had the voice improvement systems logo stuck on it.

The system came with lots of legal warnings about not copying, not telling unqualified people about how it worked and not to use an unapproved microphone. The DMCA was specifically mentioned.

One day the mic failed (the program requires patients to shout aggressively at the mic) so my wife went off looking for a replacement. We had a few USB mics that we tried and and the application refused to acknowledge their existence even though they showed up in Windows. It became obvious that the software was checking the USB device ID. My wife went to the company that ran the system to get a replacement, but they said she had to buy a new copy of the software as well - total cost $659. So we took a chance and ordered a new Samson USB mic from Amazon for 30.00, but when it arrived it didn't work. It was the same model, but was a few generations ahead and therefore had a different USB device ID. My wife has some colleagues with the same package so I tested their mics and they had different USB device IDs and it became obvious that when Samson released a revision of the mic the company offering the system simply recompiled the code with he new device ID baked in and then re-branded the mic.

So, not wanting to shell out $659 for a whole new package I took the old and new mics apart desoldered the cartridges from both mics and put the new one in the body of the failed mic. It worked! Now technically this would be a violation of 1201 in the sense that the individual copy of the software they sold you was tied to the specific mic they sold you at the same time - they said so in the EULA. But lets be honest that's just nonsense. They were simply trying to sell more stuff - a tactic that seems fairly common in various fields of professional therapy.

This is the sort of problem caused by 1201. If we lived in the US we would have been in breach of the DMCA even though we copied nothing.

Also, the software is as ugly as sin.

2
hlandau 2 days ago 2 replies      
This post about the damage inflicted by 1201 reminded me of another 1201: Halon 1201, banned because it depletes the ozone layer. A serendipitous coincidence, with this post talking about 1201 like an ecological threat.

More seriously, the GPLv3 contains an interesting provision. Search for "Anti-Circumvention" in this to find the section: https://www.gnu.org/licenses/gpl-3.0-standalone.html

The second paragraph is probably enforceable, but I'd be interested to hear from someone suitably informed whether the first paragraph has any basis. How far can it be taken?

For example, one of the most insidious things about the Blu-ray format is that unlike DVD and HD-DVD, commercially pressed video Blu-rays are obliged to use AACS. Theoretically non-AACS discs could be pressed and work, but the replication plants aren't _allowed_ to print non-AACS video Blu-rays. This has caused some consternation where people want to distribute Creative Commons/etc. video on optical media, more than can fit on a DVD. I think I recall Archive Team talking about just having to resort to putting video files on a data Blu-ray instead.

If someone made a film, put "Neither this work nor any derived work can constitute an effective technological measure for the purposes of the WIPO copyright treaty or any corresponding legislation" in the credits, and then someone else got AACS'd Blu-rays made of it, would 1201 thereby not prohibit breaking AACS specifically in the context of that Blu-ray? It seems rather dubious.

3
benmarks 2 days ago 0 replies      
4
onetwotree 2 days ago 3 replies      
Good luck!

What's kind of cool about this issue is that it attracts support from citizens of all political stripes - whether you're a farmer who just wants to be able to fix his own damn tractor, or a hacker who wants to futz with proprietary hardware, the law is patently bogus.

Unfortunately, farmers and hackers have far less political influence than corporations. Hopefully by pursuing this through the courts and with adequate resources from the EFF some progress can be made that couldn't be in congress.

5
rayiner 2 days ago 9 replies      
Circumvention by itself definitely shouldn't be illegal, and it's probably unconstitutional to make building and researching circumvention mechanisms illegal. But I don't buy Step 2.

> EFF is representing plaintiff Andrew bunnie Huang, a prominent computer scientist and inventor, and his company Alphamax LLC, where he is developing devices for editing digital video streams. Those products would enable people to make innovative uses of their paid video content, such as captioning a presidential debate with a running Twitter comment field or enabling remixes of high-definition video. But using or offering this technology could run afoul of Section 1201.

It definitely should be legal to build those products. Maybe it should be legal to distribute that captioned video as fair use. But why should Twitter profit from a user captioning a video CNN created?

That's the part I have trouble with here. Fair use is fine and good, but there is a large universe of very profitable companies that don't make content of their own, but profit from other peoples' content. Of course they have a huge interest in weakening copyright protections under the guise of promoting fair use.

6
unabst 2 days ago 2 replies      
What we need is the legal right to fork any IP. An open licensing model where no one needs permission. They just need to maybe pay an IP tax that trickles up to the previous contributors that helped produce what was forked.

IP is completely flawed because it grants a monopoly on the fruits of specific knowledge or a work as if they are static end products, whereas in reality anything that is not evolving is dying. So the law restricts progress to the owners of the IP even when we could all contribute. And when there is incompetence or negligence by the owners, we have a situation where something good is ruined or withheld, with anyone fixing it being illegal.

Removing IP is impossible because it's about profit, which is also a right. What we need is a new revenue system based on new principles of an expectation of progress and open contribution. Open source software and hardware is this, but just without any standard profit model backed by law.

7
ethanpil 2 days ago 0 replies      
If something isn't done about this very soon, people will never remember or know what used to be. Most (many?) of us here have used VCRs, tape recorders and CD burners, etc, and understand what he is talking about when we remember the days when we had freedom to own information.

Today's kids have been well trained by Apple, Google and Netflix and hardly even understand what we are talking about.

"Oh, you don't have an iPhone anymore? Just buy it on Google Play and you will have it again on your Galaxy." is a quote I have heard more than once...

8
dikaiosune 2 days ago 2 replies      
If you're in this thread to support this EFF-backed action, I would strongly consider donating to a cause you support:

https://supporters.eff.org/donate/

9
DanBlake 2 days ago 3 replies      
Unfortunately I believe that even if the suit was successful, we would just see more purchases become 'perpetual licenses', skirting the updated law. IIRC, Tesla was very heavily against letting anyone tinker and went to some extremes to stop it. It wouldn't surprise me in the least to see them make buyers sign a EULA in the future when you go to 'purchase' a vehicle.
10
forgotpwtomain 2 days ago 3 replies      
I am curious why, if they actually believe they have a good chance of success, this is only being filed now rather than in prior years? Has something changed?
11
mrmondo 2 days ago 1 reply      
I fully support your cause.

I'm not an American and do not live in America but the problems with American (copyright) laws unfortunately affect the world on a global scale. I sincerely wish you all the best in your efforts and hope that other organisations as well as the (fantastic) EFF back you.

I stand behind you.

12
filoeleven 2 days ago 0 replies      
A quick summary for those who don't want to click through without knowing what the lawsuit challenges:

Section 1201 contains the anti-circumvention and anti-trafficking provisions. These infringe upon fair use activities like format conversion, repairs, and security research.

13
dang 2 days ago 0 replies      
A related article by Matthew Green is at https://news.ycombinator.com/item?id=12137437, and by the EFF at https://news.ycombinator.com/item?id=12136682.
14
thinkMOAR 2 days ago 1 reply      
If only they were the bully on the school playground perhaps you could fight him. But they are the playground, i wish you the best of luck.
15
lifeisstillgood 2 days ago 0 replies      
The UK government is trying to push for OSS as the default for all government software. As a default for all "societally beneficial"'software is a better goal and one highlighted here.

Now my attempts so far are stymied by this weird half world. Most government contracts basically want either bums on seats contractors or to fundamentally hire "someone who has done it before" (effectively the same as wanting to buy off the shelf)

So there is almost no way to seed fund the initial OSS development.

Down thread people talk about a fund for starting OSS projects to provide things like this. Plover is an example of people trying it on their own - but a funded system that basically follows current gov work seems better.

16
SilasX 2 days ago 1 reply      
Can someone do a tl;dr? This is upvoted very highly but it's assuming a ton of context I don't have. All I get is that someone wants to be able to tinker, but today that necessitates breaking some legally-enforced protections on the product.

That's a valid point but I don't see how it's gotten to 1000 points, so I think I'm missing something. What's the lawsuit? What's the egregious use case?

17
markokrajnc 2 days ago 2 replies      
"Our children deserve better." If you take children - they indeed mix and remix without worrying about any (copy)rights...
18
tomc1985 2 days ago 0 replies      
Doesn't the US dismiss most lawsuits filed against it out-of-hand? Wasn't that why that class-action on behalf of the Japanese concentration camp survivors was such a landmark case?
19
ankurdhama 1 day ago 0 replies      
The problem is this new business model where they don't just sell you things/stuff, rather they also sell you "specific rights" along with the stuff. The usual things like you cannot do this or that with the stuff that you bought from us. The sole purpose is to keep earning money even after the one time deal of buying the stuff.
20
shmerl 2 days ago 0 replies      
Great. DMCA-1201 was always unconstitutional and was in practice used to stifle free speech. Good to see EFF actually bringing it to legal fight. It should be repealed completely.
21
LELISOSKA 2 days ago 0 replies      
This entire cause is a sham, beyond belief, a cause that seeks to degrade the value of creative thought and intellectual property.

Before we get into socioeconomic barrier discussions I am a former disabled homeless person who is how the founder of one of the most powerful environmental activism groups in the country. I started out with nothing and worked myself to where I am, using original and creative thought and at no time have I ever needed anyones intellectual property to build myself to where I am.

The Electronic Frontier Foundation, that supports this complete bullshit erosion of the rights of content creators everywhere, does nothing in this world but fight for causes that continually reduce the market value of original ideas.

They claim to fight for things like free speech but what they really fight for is the rights of anonymous hate groups to steal your photos and write nasty messages on them. They fight for the rights of the meek to inherit the Earth so they can then destroy it with their abject failures.

Look to the recent lawsuit Google v Oracle, where Oracle sued Google over the use of their software in Android. Google avoided billions in liability and it was all thanks to the work of the EFF, who suck off the teat of Silicon Valley and protect their billionaire buddies from financial liability, and then they support little guys like this so they can continue their 1% supporting ruse.

I look forward to watching this mad grab at free intellectual property get slapped down by Washington DC. This is not about fighting the government, this guy is a puppet being used by the power that be in Silicon Valley in order to allow companies like Google to continue to rob, loot, and pillage other peoples intellectual property without financial liability.

22
BenedictS 2 days ago 0 replies      
I've made an account just to wish you good luck! You're a great man for doing this and I'm glad EFF is on board.
23
maerF0x0 2 days ago 0 replies      
IME many US people do not resonate with the creativity arguments, but do with the freedoms. The land of the free lately doesnt feel like it and I think many US people are feeling it too. It may help to phrase your arguments in the wording that the constitution is meant to protect -- in terms of freedom.
24
chejazi 2 days ago 0 replies      
This reminds me of a new patent Apple filed to disable video recording on iphones. Would winning this suit prevent that from being enforceable?
25
amelius 2 days ago 1 reply      
I wonder how much he budgeted for this series of lawsuits.
26
hackaflocka 2 days ago 0 replies      
DJ Drama, the mixtape guy, was raided under the same law. It's an interesting story, google "dj drama raid"
27
spacemanmatt 2 days ago 0 replies      
The whole DMCA is a steaming pile, but I guess I'm ok with piecewise dismantling.
28
wonkyp2 2 days ago 0 replies      
I cackled at the former, homeless vegan (or thereabouts) who started a shitstorm in the comments.
29
known 2 days ago 0 replies      
I'll support;
30
reddytowns 1 day ago 0 replies      
You know, no one asked you tech people from getting involved in law making. Nowadays, a law maker can't seem to do anything at all without some techie crying foul. Their argument always is some nonsensical technobabble, which the courts can't really understand anyway, often giving in to their demands just to get them to go away.

And it's such a shame, too, since those laws were bought and paid for by lobbyists, and what does it say about the rest of the country if one can't expect to get what one pays for when lobbying at the highest level of government?

31
blastrat 2 days ago 2 replies      
yes I agree, and also, what? why should PP's question be downvoted-to-hell?: He's entitled to defend the other side here.

Not saying you did it, but I had to comment someplace.

32
paublyrne 2 days ago 3 replies      
Some people just, you know, read the article.
33
magice 2 days ago 8 replies      
I do appreciate the effort to protect everyone's constitutional right. I wish best of luck to the pursuit.

However, I feel like there is something very very wrong about method and intention of this type of actions/complains.

One thing always bugs me about Americans: despite the liberties that they enjoy, despite the very real capacity to impact change in their government and laws, they all hate "the Government." Who is "the Government"? Wait, ain't them the very candidates that you the people vote into offices?

Like this idea of "suing the US government." Who are you suing? The executive branch? Why are you suing them? This is over a law. It's a piece of legislation. The executive branch merely, you know, execute the laws. Why not sue Congress? Oh wait, why sue Congress when you can simply vote them out of office? Oh wait, why "stop enforcing" the laws when you can, you know, CHANGE the laws?

This kinda reminds me of the libertarians' ideas of obstruction of legislation so that "the government does not spend more." If not spending is the right thing to do, why not educate people that. Even if one believes that 47% of the population is "takers," 53% is still a majority. So teach, advocate, change minds. But no, they prefer to obstruct their country, risk the centuries of their national reputation, put t heir fellow citizens to starvation. You know, if this happens in schoolyards, we probably call it "bullying." But if a bunch of libertarians do it, it's "principles."

Obviously, I agree with the plaintiff here. However, the method is still wrong. And different from above, there are very few "takers" here. Mostly, it's faceless businesses that (let's be frank here) few people like. So why not take the high road? Why not educate your fellow citizens on the danger of the laws? Why not change minds? Why not raise money for candidates who will change the laws appropriately?

In short: why not be a citizen rather than a rebel? Why not change the system for the better rather than obstruct it? Why not make your society/country a better place rather than simply fight it?

34
ryanswapp 2 days ago 13 replies      
I studied section 1201 thoroughly during law school and I think this post doesn't give a fair characterization of it. The reason this statute exists is because companies were unable to devise protection for copyrighted works that hackers were not able to immediately circumvent. As a result, the government stepped in and created 1201 to make it illegal for someone to circumvent some form of access control that a company used to protect their copyrighted works. The purpose of the statute isn't to destroy <insert Internet activist claimed right> but is to make it much less expensive for a company to protect its products. I don't see anything wrong with that.
35
olympus 2 days ago 2 replies      
I think this is an important topic that needs to be addressed, but suing the government is doomed to fail. The federal government has sovereign immunity, and you can't sue them unless they decide that you can. They usually decide that you can't. Most laws aren't changed in the court unless someone is criminally prosecuted. Then your appeal case can move through the higher levels of the court until it reaches a level that the law can be struck down completely, or what usually happens is a legal precedent is set regarding a specific portion of the law.

So unless Bunnie has been prosecuted for breaking the DMCA, this is likely going to be an ineffective move.

If you want to change a law without breaking it first, the right way to go about it is petitioning Congress, the lawmaking part of the government.

36
6stringmerc 2 days ago 3 replies      
Let's take a quick look at the understanding of Copyright law that this litigant seems to possess:

>Before Section 1201, the ownership of ideas was tempered by constitutional protections. Under this law, we had the right to tinker with gadgets that we bought, we had the right to record TV shows on our VCRs, and we had the right to remix songs.

Wait, before the DMCA "we" had the right to remix songs? Okay so this case is going nowhere because the person filing really doesn't quite understand the mechanics of basic Copyright. Just kind of throwing out the concept of "remixes" does a dis-service for the real nuances of how the rights/permissions/compensation system works, has been tested in court, etc.

The subject of ownership and repair is extremely complex and this lawsuit is frivolous when the matter is being actively tested by John Deere and various farmers. Maybe this person could assist in funding that challenge to 1201. There are some glaring flaws in this whole approach, from what I understand about Copyright law and the DMCA.

Also, I don't know why the EFF continues to push erroneous information regarding how Copyright, the DMCA, and Fair Use actually work:

>This ban applies even where people want to make noninfringing fair uses of the materials they are accessing.

Fair Use always trumps the DMCA; the nature of Fair Use, however, is subject to four factor tests, if an IP owner should feel compelled to assert the Fair Use was not in the spirit and letter of the law. Sometimes it seems like the EFF and TechDirt try to claim things that aren't true just to make a point. It's something that bothers me routinely in this subject in particular.

2
Master Plan, Part Deux tesla.com
1823 points by arturogarrido  3 days ago   674 comments top 81
1
Animats 3 days ago 10 replies      
"A first principles physics analysis of automotive production suggests that somewhere between a 5 to 10 fold improvement is achievable by version 3 on a roughly 2 year iteration cycle. The first Model 3 factory machine should be thought of as version 0.5, with version 1.0 probably in 2018."

What that really means: Tesla is going to lose a ton of money per car on the Model 3, or raise the price, until at least 2022. That's realistic. His two top production guys quit when he announced 2018 as the delivery date for the Model 3. His new production head, from Audi, may have given Musk a reality check.

Tesla produced about 50,000 cars in 2015 with 13,000 employees, about 4 cars per employee. Ford produced 3.2 million cars in 2015 with 187,000 employees, about 17 cars per employee. Toyota produced about 9 million cars with 344,000 employees, about 26 cars per employee. So Tesla needs to get their productivity per employee up by 4x - 7x to play with the big guys. Clearly Musk has done the same calculation.

Now, though, he's admitting that they can't do it by 2018. This is prepping the stockholders for bad financial news. Tesla is going to burn a lot of cash through at least 2022.

There's no reason that Tesla can't get their productivity up to at least Ford levels in time. Ford has a much broader product line, and Tesla's car isn't that complicated mechanically. But it's not instant.

2
thucydides 3 days ago 12 replies      
> "When used correctly, [partially autonomous driving] is already significantly safer than a person driving by themselves"

If you're an American, you're twice as likely to die with a steering wheel in your hands as you are to die at the hands of a murderer. Human-driven vehicle deaths cause grave second-order suffering for families and friends - and hurt the economy.

A shift to technologies safer than human-driven cars would dramatically reduce human suffering and should be welcomed.

I do wonder, though, how this would reshape our cities - if we're not careful. Besides direct costs for the car, fuel, and maintenance, the main disincentive to driving is how damn boring it is. What happens when we turn fully-autonomous vehicles into luxury entertainment centers? I suspect that, if we're not smart about this shift, we could see wild sprawl on a scale that would dwarf the mid-20th century sprawl we saw in Los Angeles and elsewhere.

On the whole, though, it's a beautiful thing.

3
sidcool 2 days ago 4 replies      
A digression from the mainstream discussion.

I woke up this morning feeling sullen (many factors involved). I didn't feel like going to work. I could hardly get out of bed. I just sat for a few minutes staring in the vacuumn. Something told me to check Hacker News (I am trying to avoid it in morning), and the top link was this. I went through it twice. It instilled hope and enthusiasm in me. I woke up in an instant and rushed to work to do great stuff.

Thanks for the article I am typing this at work, else would have wasted the day filled with self-loathing and despair. Hang in there guys, it gets better. Do Great Stuff.

4
kybernetikos 2 days ago 4 replies      
> The most important reason is that, when used correctly, it is already significantly safer than a person driving by themselves and it would therefore be morally reprehensible to delay release simply for fear of bad press or some mercantile calculation of legal liability.

I don't think this is the correct comparison. A car used correctly is safe. We have huge numbers of road accidents because most people are unable to reliably use a car correctly. The value of an 'autopilot' functionality is that it should be much better at using the car correctly in the real world than a human.

What matters is not how many accidents result when using autopilot 'correctly', but how many accidents result from using autopilot in the real world.

Also, because autopilot is primarily used on particular road profiles, it's not fair to compare accidents per autopilot mile directly with accidents per human driver mile. You need to adjust for the fact that autopilot is not used during more complex driving anyway.

I'd be very interested to know what the statistics are for those, since the recent press has given me a (potentially incorrect) impression that autopilot has lead to a relatively large number of serious accidents compared to the number of cars deployed.

5
bane 3 days ago 2 replies      
I'm always surprised when we get insight into Musk's plan, not because they're complicated but because they always come across as "no duh, why aren't we already doing these things? weren't we already on track to do these things decades ago?"

As much as the Internet transformed society, I also can't help but feel like we were on the track to have achieved these things and got distracted by our global communications and selfie-cat picture delivery network and are only now starting to come to our senses as the ubiquity has occurred and the ecosystem of necessary applications has become fleshed out, matured and developed a commercial angle.

If you look at his pre-hardware days, he built basically a e-phonebook when paper phonebooks were still all the rage and a couple payment companies. Both no-duh companies in hindsight.

Musk's plans feel like he's taking a derailed train, applying some common sense grease (solar panels on electric cars? MADNESS! Reusable rocket stages instead of throwing away the entire ship? ~~CRAZY!~~) and getting our civilization going again.

He's also really really public about his plans and telegraphs his moves years in advance...and yet very few seem able to execute anywhere near his league.

I sometimes feel if things had shaken out differently and Steve Jobs was younger than Musk and was running a successful Apple, Musk might try to recruit him with a "do you want to sell cat picture delivery boxes for the rest of your life, or do you want to come with me and change the world?"

I don't know if Musk is going to succeed in the long run, and I hope serious competition finally shows up (because that makes each of his industries healthier), but seriously,

it's about fucking time.

6
aerovistae 3 days ago 6 replies      
To anyone who's paid close attention to Tesla and to Elon's various offhand remarks to the press and on Twitter, this was all easy to see coming, every bit of it.

But now that he's confirmed it all officially: this NUTS. This is so awesome. The press is going to go crazy with this.

I wonder what will happen to Uber...seems like it will be hard for them to compete with the rates of cars that don't have to pay their driver a living wage, nor pay for gas.

Electric semis-- THANK GOD. I live in Chicago and I can't tell you how sick I am of the massive exhaust plumes billowing over me as they pass by, and the roaring of their engines on the street outside my apartment.

7
ams6110 3 days ago 13 replies      
My points of skepticism:

1. Semis. A typical long haul semi gets well under 10MPG. In some cases not much more than half that. They are heavy and need a lot of energy to move. A Tesla Model S weighs 4650 lbs and has a range of a couple of hundred miles. A semi truck can weigh up to 80,000 lbs. That is a lot of weight to get rolling and a lot to pull up a grade. Semis spend a large part of their time driving at highway speeds where air resistance is at a maximum. To achieve useful performance an electric semi will need a lot of batteries which will reduce its cargo capacity (Federal law regulates the maximum gross weight), which reduces its value to freight companies.

2. Autonomy. I think this will take a lot longer to achieve than planned, both technically and socially.

3. Enable your car to make money for you. I don't want anyone using my car. Legal liability is one reason. As owner of the car, I am liable for damage it causes. So legal liability laws will have to change. If I need to go somewhere, and my car is not here, I don't want to wait for another one. I don't want to get my car back from another user and find food wrappers strewn about and used condoms under the seat. I feel that my car is an extension of my home. It's personal space that I don't want to share with random strangers.

YMMV.

8
ilamont 2 days ago 3 replies      
Once we get to the point where Autopilot is approximately 10 times safer than the US vehicle average, the beta label will be removed

Musk misreads the public's attitude about vehicle safety. Human error is understandable, mechanical failure is unacceptable. Society can live with 10 people driving themselves off a cliff (and blame the drivers, road conditions, or poor signage) but they will not accept a car driving its trusting passengers off a cliff.

9
greendestiny 3 days ago 1 reply      
"I should add a note here to explain why Tesla is deploying partial autonomy now, rather than waiting until some point in the future. The most important reason is that, when used correctly, it is already significantly safer than a person driving by themselves and it would therefore be morally reprehensible to delay release simply for fear of bad press or some mercantile calculation of legal liability."

That's a hell of a statement and I want to see much better stats than that. Just looking at the total distance per death in human driven cars and comparing it to the autopilot total distance is a gross simplification. At an absolute minimum you have to start by only comparing driving on similar roads. Tesla simply keeps hiding behind 'if used correctly' which includes the driver being alert and ready to take over - if we restrict human driving stats to similarly ideal conditions the accident rate will also drop. Additionally driver demographics is a big deal as is the safety features of the car itself.

10
pdq 3 days ago 8 replies      
I still don't understand the SolarCity part. Tesla is atop the best rated electric cars, and has a good trajectory toward that product line future, with lots of innovation ahead. Successful companies like Apple focus on best-in-class products, so Tesla is smart to continue focusing their resources into those product lines.

Meanwhile SolarCity has been burning cash on a consistent basis [1], and is sitting in a hyper competitive solar panel industry, where I don't see their competitive advantage. It seems foolish to bring that business inside of Tesla, as if it failed, the debt risk would now affect Tesla's future. As many others have mentioned, a long term licensing deal or partnership avoids those risks.

[1] http://www.msn.com/en-us/money/companies/musk-says-solarcity...

11
danhak 3 days ago 1 reply      
I'm on board with most of this. The goals here are obviously ambitious by any standard, and would seem totally absurd if put forth by anyone who didn't happen to berth a private spacecraft with the ISS earlier this morning.
12
rbosinger 2 days ago 0 replies      
I have stock in Solar City and I don't care if I lose it all. It's not my life savings though. I invested because I enjoy day dreaming a similar dream to what Musk must be dreaming. Like any great science we just have to try and be excited, together. We can all banter about economics, rationality and history but I'm stoked. Who cares. I don't see how Tesla or SolarCity failing would lead to mass starvation or anything so let's strap in and be pumped!
13
d_t_w 3 days ago 1 reply      
"Coal is the future" - Tony Abbott, Australian PM, 2014.

There are many things to take from Musk's master plan part deux, but the most important for me is the intent and aspiration.

I live in Australia, the leadership here is absolutely dire both political and economic. A relentless cycle of vested mining interests and climate change deniers espousing at length on the cattle exports to Asia suffering if marriage equality is passed.

Maybe Musk succeeds, maybe not, but here's someone with vision, a plan, and he's going to have a fair swing at it.

14
paulsutter 3 days ago 7 replies      
> We expect that worldwide regulatory approval will require something on the order of 6 billion miles (10 billion km). Current fleet learning is happening at just over 3 million miles (5 million km) per day.

This seems very significant for Tesla vs competitors. Yes Google has a strong technology lead today, but how long will that last when Tesla is collecting more miles of data every day than Google has collected in 5 years? (Sincere question) Not to mention Apple and existing car vendors, who each have 0 million miles of experience.

Tesla should reach 6 billion miles very quickly once the model 3 is out.

15
Evgeny 2 days ago 1 reply      
As of 2016, the number of American car companies that haven't gone bankrupt is a grand total of two: Ford and Tesla.

Also, four entities have launched rockets into space: the US, China, the Soviet Union (Russia) and Elon Musk.

This guy is thinking and planning on a scale I find it hard to even imagine, to fit in my brain.

16
mrfusion 3 days ago 4 replies      
I didn't understand what enables them to get rid of aisles inside buses? People still need to reach their seats right?
17
unabst 2 days ago 1 reply      
Tesla isn't a car company. It's building Rome. All great "startups" are not some good idea executed well. They are companies with a long term vision that generate ideas to execute that will get them there. Anyone might steal an idea or copy a product, but no one can steal a mission or a destination far in the future. Apple, Microsoft, Facebook, Yahoo, Google... all started as or at some point became "build Rome" companies.
18
_s 3 days ago 1 reply      
Uber is betting on car manufacturers to have autonomous driving in place, while it builds up a worldwide user base of logistics (moving people and goods from X to Y). It doesn't care if vehicles are driven by a horse or by electricity.

Tesla is building the vehicles and energy source for the vehicles, and building the autonomy in to them, but it's betting on a user-base acquisition via hardware (vehicle) ownership and/or eventually some form of subscription to the "Tesla" club.

Carbon-based fuel(s) will eventually run out. Tesla via SolarCity will be in an incredible position of offering energy, so I'll be looking at how well their plan of putting Solar on every roof works rather than autonomy / vehicle manufacturing / sales. I think this is likely going to be their make or break asset.

19
sp527 2 days ago 2 replies      
I'm genuinely surprised at the quality of Musk's writing and the presentation of certain ideas. He clearly insisted against copyediting, and that was probably a mistake judging by the output. I suspect that he was attempting to eschew the formality of typical press releases, but this 'Master Plan' (which is itself a somewhat juvenile moniker) feels like something that warranted rigor.
20
sna1l 3 days ago 1 reply      
> "Create a smoothly integrated and beautiful solar-roof-with-battery product that just works, empowering the individual as their own utility, and then scale that throughout the world. One ordering experience, one installation, one service contact, one phone app.

We can't do this well if Tesla and SolarCity are different companies, which is why we need to combine and break down the barriers inherent to being separate companies. That they are separate at all, despite similar origins and pursuit of the same overarching goal of sustainable energy, is largely an accident of history. Now that Tesla is ready to scale Powerwall and SolarCity is ready to provide highly differentiated solar, the time has come to bring them together."

I don't really see how this answers the question as to why they need to merge? Why can't their just be a partnership?

21
SigmundA 3 days ago 2 replies      
Anyone else think AbstractTelsaFactoryFactory?
22
stephenitis 3 days ago 5 replies      
I hear about bus drivers and truck drivers getting into accidents due to drowsiness enough that it's a constant worry whenever I'm on the highway next to one. I wonder if I'd feel safer if i saw a Tesla Semi knowing that were wasn't a human behind the wheel.

Should autonomous vehicles be identified as such (special lights or label) so that real humans can know not to be erratic around it?

23
stephenitis 3 days ago 1 reply      
"So, in short, Master Plan, Part Deux is:

- Create stunning solar roofs with seamlessly integrated battery storage- Expand the electric vehicle product line to address all major segments- Develop a self-driving capability that is 10X safer than manual via massive fleet learning- Enable your car to make money for you when you aren't using it"

I take this as...

Having solar powered superchargers power autonomous semitrucks transporting cargo across america.

Having solar powered superchargers power autonomous public buses transporting people around a city.

Have my car join a fleet of uber-like autonomous teslas while i'm not using it.

24
j0e1 3 days ago 1 reply      
Haven't seen such a clean, crisp plan that has been implemented flawlessly by a company. Makes me reconsider how I should think the next time I'm asked to write a vision/mission statement for anything-company/product.
25
Double_Cast 2 days ago 0 replies      
> A first principles physics analysis of automotive production suggests that somewhere between a 5 to 10 fold improvement is achievable by version 3 on a roughly 2 year iteration cycle.

How does one calculate this? Does there exist some canonical Productivity-Equation?

26
mshenfield 3 days ago 1 reply      
Less of a master plan, and more a list of goals. The original was awesome because it clearly laid what Tesla wanted to accomplish "Consumer electric vehicles" and how. This just lays out the what.
27
stcredzero 3 days ago 0 replies      
A first principles physics analysis of automotive production suggests...

This is the part I really want to see.

28
ejz 1 day ago 0 replies      
This isn't really that interesting. There's nothing here that hasn't been said already or at least very strongly suggested by Musk. The last master plan was interesting because no one had ever really made a successful electric car company, so no one imagined it could be anything but a rich person's toy, so Musk's claim that he could make a desirable mass production car was a huge shock.

It's also very sloppy; it's not an actual plan, with goals and steps that logically follow each other. The last master plan had a clear logic to it: you used the margin of each successive step to fund research and development further down in order to increase the use of electric cars and limit global warming. This is more like a wish list than a plan. "We want to make semis." "It'd be great if we also provided the solar part of the stack because it dovetails with this other initiative we're doing." "Once we have solar, we can do this new thing." Etc, etc. Unlike the first master plan, I can't gauge how long any of this will take or whether it is feasible. I can't gauge what the actual strategy is any better than I could yesterday. And isn't that the point of a Master Plan?

29
davnicwil 2 days ago 1 reply      
> increased passenger areal density [on buses] by eliminating the center aisle and putting seats where there are currently entryways

I can't picture the layout he's describing here - not sure if it's been discussed in more detail elsewhere - anyone got a better idea or a reference image?

30
Tiktaalik 2 days ago 1 reply      
The best part of this is automated semi trucks. I think that's the perfect sort of business for Tesla to be in.

I have a lot of trouble understanding the public transportation part. The ideas presented fall apart when you remove the baffling assumption that traffic congestion decreases with the introduction of autonomous vehicles. Autonomous vehicles will expand the possible set of drivers. That will dramatically increase the amount of vehicles on the road. If anything our future with autonomous vehicles will be unbearable gridlock.

31
lsllc 3 days ago 1 reply      
My favorite quote:

"Starting a car company is idiotic and an electric car company is idiocy squared."

32
mrfusion 3 days ago 1 reply      
Are there any more details on the factory factory? I didn't understand that part.
33
cpwright 3 days ago 0 replies      
I'm very curious what the pickup offering will be like, and compare to existing offerings from the big 3, since it is a "a new kind of pickup truck". I think the segment of the pickup market that is mostly an SUV, but occasionally needs to haul/tow stuff could be well served by Tesla. They also have plans for a semi, so maybe they'll actually be able to compete for actual work trucks that haul/tow on a regular basis too; but the energy density of the battery compared to gasoline/diesel makes me doubtful.
34
shasheene 2 days ago 1 reply      
Hopefully Tesla will be able to achieve this new plan. Looking back at the first 'master plan' from 2006, it's clear that it failed pretty badly as Tesla Motors wasn't close to being able to self fund its goals over that time.

Since the first master plan was published in 2006, Tesla Motors has raised money privately (during its near death experience in 2008), sold a 10% stake to Daimler (which was recently divested), went public which has a side effect of raising even more (though the main reason to IPO in most cases liquidity to existing investors), and since then have continually raised money from the public market every year or two. There's probably private and public capital raisings since 2006 that I'm forgetting too (and they raised other capital streams like debt, such as the DoE loan)

The very lofty stock price of Tesla in recent years has helped it fund Model S, Model X and Model 3 designs, development, manufacturing (at large scales) and delivery, as well as the building of a large battery factory which Tesla owns a stake in. This constant fund raising has kept Tesla alive and I don't argue that it was very good corporate governance by Elon Musk and team to get Tesla Motors to where it is now (approaching the delivery date for the first Model 3 shipments and having a huge capacity to manufacture battery packs).

However, it's still a failure in its attempt to bootstrap the funding of Model 3 based on sales of previous models.

Of course, Tesla and SpaceX has consistently ended up achieving great things, even if the timeline is optimistic and the budget ends up blowing out. But issuing stock and eventually debt can only stretch Tesla so far. Hopefully Tesla can become a more sustainable business before that happens.

35
dredmorbius 2 days ago 0 replies      
For rates of process and cost improvement with scale, look to J. Doyne Farmer's work, and particularly Wright's Law (Moore's Law is a special case, and less accurate), which looks at cost improvements with volume increase through learning functions.

http://spectrum.ieee.org/tech-talk/at-work/test-and-measurem...

http://www.eetimes.com/document.asp?doc_id=1263107

http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjourna...

http://www.santafe.edu/research/working-papers/abstract/0650...

36
dskloet 2 days ago 0 replies      
> The most important reason is that, when used correctly, it is already significantly safer than a person driving by themselves and it would therefore be morally reprehensible to delay release

Doesn't Tesla charge a large fee to have autopilot enabled on your car? Isn't that equally morally reprehensible?

37
Shivetya 2 days ago 0 replies      
Can we talk about what I see as the most important driving force behind acceptance of full or partial electrification of transportation? Heavy duty and transport. Specifically I think Tesla would be best off getting school buses to full EV or even partial EV capability.

Using a Cobb County Georgia as an example, stats posted awhile ago listed over a thousand school buses traveling almost seventy thousand miles a day. Seventy thousand miles a day! Since the buses have to load/unload at schools and such its easy to establish charging points to include fast top offs where five or six minutes of charging can extend enough to the next time. Then between major routes, elementary, middle, and high school, longer charge periods can be done.

Get kids and parents used to silent electric buses and you go a long way to establishing a generation on them. Get autopilot to work well in that environment and you get to sell them on two innovations at once

38
maxander 3 days ago 4 replies      
Elon Musk is like the cat who tried to jump from the sofa to the top of the bookcase, fell to the floor in a tangle of wildly gyrating limbs, and is now sitting there quietly licking its paw like it was intending to do that all along. Even with his brilliance he's fooling no one into thinking the SolarCity merger was actually in his plan- there are some synergies, sure, but they're easily outweighed by the added corporate complexity.

But I mean, he's Elon Musk. He could still pull it off.

His biggest problem by far (excepting, perhaps, Model 3 production targets) will be regulations. It makes a nice story when you talk about the relative risks rationally, but there's no chance whatsoever American politics will deal with the issue in a rational fashion. Autopilot may retroactively become illegal in places people currently get away with it; cars driving themselves around is a different and titanic can of worms. What if a terrorist gets their hands on a Tesla and stuffs it full of explosives?

39
OliverJones 2 days ago 0 replies      
Fleets, Mr. Musk. Centrally owned fleets of vehicles, where you can make your sales case based on total cost of system ownership rather than sex appeal.

It's a tall order, but can you set your sights on those "long life vehicles" presently used by the US Postal Service in urban and suburban areas, or maybe similar vehicles in Europe. Those machines return to base daily and usually are unused at least 8 hr/day. Massive buildings with large roofs.

Cop cars. Lots of slow speed cruising combined with a very occasional need for high speed and agility. Return to base every shift. Location awareness.

These sales cycles will be long, and probably a pain in the neck for your major account teams. But you're in it for the long haul.

From the owner of Model S #146761

40
bambax 1 day ago 0 replies      
> Enable your car to make money for you when you aren't using it

In cities, everyone needs their car at exactly the same time, that's why we have congestions. When I'm not using my car, no one else needs one (that's an exaggeration of course but not by much).

So in order to get to sustainability, we need to understand why remote working (for instance) hasn't happened yet.

41
back_beyond 3 days ago 1 reply      
Master Plan, Part Trois:

Solar-powered, autonomous spacecraft.

42
kilroy123 3 days ago 1 reply      
> We expect that worldwide regulatory approval will require something on the order of 6 billion miles (10 billion km). Current fleet learning is happening at just over 3 million miles (5 million km) per day.

So he is essentially say, in at most five in a half years, they'll be ready for fully self driving cars?

43
amluto 3 days ago 1 reply      
Master plan part one made sense: it all lead to Tesla as it is today.

But master plan part deux seems odd: what exactly does "One ordering experience, one installation, one service contact, one phone app" for solar have to do with the Tesla transportation part?

44
esusatyo 3 days ago 1 reply      
It's interesting that they portray self-driving capabilities as something that can be turned on or off, unlike Google's where it's just always on.

I think in the long run Google might be building the correct solution for greater number of people.

45
hackguru 3 days ago 3 replies      
I am in no position to question EM. But I was hoping he would give some good explanation for spending resources on SolarCity acquisition but nothing. Nothing in this master plan explains why SolarCity was bought other than some hand wavy explanation about inherent difficulties of two separate companies working together. It still doesn't seem like a good purchase for Tesla specially at the moment. Solar car and SolarCity seem to only have the word solar in common :) TBH I am still fuzzy how expensive purchase of SolarCity can benefit a solar car manufacturing even in long run.
46
thatfrenchguy 2 days ago 1 reply      
"In addition to consumer vehicles, there are two other types of electric vehicle needed: heavy-duty trucks and high passenger-density urban transport"

Where I'm from, we call that trains.

47
paul_milovanov 2 days ago 1 reply      
For a bit of a reality check, go take a look at Financial Times coverage of Tesla.

Tesla will require regular infusions of capital over, say, the next 5 years. The only source for that is more equity, and to do that you need to actually start meeting some of your self-declared profitability goals. Up to now, Tesla hasn't.

The tactic of diverting attention with "but look, here's this great awesome world-changing thing we'll do next" has worked so far but it's rapidly getting old. In general, the frequency with which sleights of hand are starting to be employed is concerning. Remember the "but don't just take my word on it I myself will be buying $20M of new stock!" thing? Sure you will to reassure investors, given that loss of confidence will cost you personally far more than $20M.

SolarCity? "If Musk thought Tesla really needs a solar company, he might as well buy a good one. But it doesn't" [FT Lex]. Given how important it is that they are able to keep raising capital through equity offerings, taking the risk of freaking the investors out with SolarCity acquisition (otherwise expected to go into bankruptcy protection by next year) makes sense only if letting SC fail presents a bigger risk of the same. The Musk fairytale would certainly take a hit from a SC bankruptcy.

And Musk setting these crazy numbers goals practically guarantees he's setting TSLA shareholders up for disappointment.

Non-profit-making Amazon has been raised as a counterargument in the comments on this thread; the amount of trust the market has extended to Bezos for the time that it has is practically unprecedented; and Bezos has worked hard to make that happen by making investment/direction choices & providing information to earn the trust of the market. Musk, to the contrary, is doing everything to the opposite.

Now, what's the likelihood of a macro downturn within the next 2-5 years? Massive. That might trip up the availability of capital a bitthose refundable $1000 deposits too but who cares about them (by the way, much of T's capital has been raised during the period of literally historically unprecedented low cost of capital)

I didn't even begin to talk about competition. Or that Panasonic, T's critical gigafactory partner, isn't just sitting around twiddling thumbs (or the Chinese).

So, it'd be prudent to curb your enthusiasm. There might not be a part trois.

48
freshyill 3 days ago 0 replies      
I feel like there must have been a few pages left off the beginning and this whole post was a summary of an article that doesn't exist. I guess that's just Elon Musk's train of thought.
49
rjdevereux 3 days ago 1 reply      
Why is residential solar important to the plan? Aren't large solar installations a more cost effective way of switching houses already on the grid to solar energy.
50
lazyjones 2 days ago 0 replies      
Battery-powered trucks and buses seem like a logical next step, since there's not much competition there yet (save some obscure small players) and weight is less of an issue.

[I'd still like to own a fully autonomous mobile home that drives me to work while I am eating breakfast in my bath robe or taking a shower and then moves me to a beach while I'm sleeping on Friday nights. Well, one can hope, right?]

51
plcancel 3 days ago 0 replies      
If this article is accurate, the Master Plan, Part Un narrative is a bit more nuanced.

http://www.slate.com/articles/business/moneybox/2013/05/tesl...

52
restalis 2 days ago 0 replies      
Hey, Musk, small nitpick: I think when you say inertial impedance you're actually referring to mechanical impedance: https://en.wikipedia.org/wiki/Mechanical_impedance
53
femto 3 days ago 1 reply      
> via massive fleet learning

Is Tesla going to make this data freely available, to accelerate the development of safer autonomous driving software? Given that it's "morally reprehensible to delay release" of autopilot, it is also morally reprehensible not to publicly release such data if more groups working on the task will lead to safer software.

54
DrNuke 3 days ago 0 replies      
Nice plan, good luck! If that happens, though, I can't see why owning a car at all, ten to twenty years from now: just make an enormous fleet of electric unmanned buses running up and down every major road on Earth, uh? No jams, no accidents, only a regular and pre-determined flow of vehicles. One planet, one network.
55
k__ 2 days ago 1 reply      
Somehow these discussions about human driven cars feel like the discussions about smoking or owning a gun to me.

The world could be a better place for so many people, but somehow a bunch of other people think it's okay to continue doing them until they die.

56
caf 3 days ago 0 replies      
I would have thought you'd want to go for delivery vans before semi-trailers.
57
helicon 3 days ago 3 replies      
Forgive me if this is a silly question but when he talks about "beautiful solar-roof-with-battery", is he talking about car roofs or roofs of buildings?

An electric car with a solar roof that charges all day would be pretty cool.

58
serge2k 3 days ago 0 replies      
> Traffic congestion would improve due to increased passenger areal density by eliminating the center aisle and putting seats where there are currently entryways

I don't see how automation reduces the need to get on the bus.

59
jzawodn 3 days ago 1 reply      
The most interesting bit, to me, was: "Enable your car to make money for you when you aren't using it"

Things are gonna get real interesting in the auto world in the next few years, aren't they?

60
Dowwie 2 days ago 0 replies      
Note his last point: Enable your car to make money for you when you aren't using it.

I suspect Tesla won't be alone in that space. Good luck to those who are.

61
orky56 3 days ago 2 replies      
I was hoping to see another zero to one approach with the master plan where he is taking on some other industry goal. I totally get that part deux is still very, very ambitious and one that no other company can truly realize. But if we're just talking about Musk's master plan, I'm curious why he didn't talk about the synergies with his other company, SpaceX. Tesla and SpaceX both rely on vehicles & transport while SpaceX and Solar City both revolve around innovation in energy/physics. I'm wondering if Tesla's mission/vision will dwarf SolarCity's when solar harvesting in space could be one of many opportunities to partner with SpaceX. /rant
62
lowglow 2 days ago 1 reply      
Last year I pitched Playa (http://getplaya.com/) at Launch conference and was laughed off stage by Jason Calacanis. My example use of the technology was that your autonomous vehicle would be able to contract itself out as an 'uber', earning you money while you slept. Today Tesla announced that this is part of its plans for the future and everyone is going wild.

We're now going a step further and building a next-gen interface for that autonomous future. #Asteria.

http://getasteria.com/

63
3327 2 days ago 0 replies      
So how do we go work for Elon if we have particular expertise on the Solar and storage part - particularly in the way that he has mentioned here?
64
untilHellbanned 3 days ago 0 replies      
Its so rubegoldbergtastic it puts all other successful entrepreneurs in history to shame, which is to say I'm incredibly bearish on it.
65
suprgeek 3 days ago 3 replies      
Obscured under " Develop a self-driving capability that is 10X safer than manual via massive fleet learning "is the cold hard truth that that learning will be paid for in lives - mostly of Tesla drivers possibly of others.

The rates of accidents in manual vs current version of autopilot may work out to be favorable - (and that is still under debate) - but there certainly will be people who will die (and have died) due to a premature roll-out of Autopilot and their trust of it. This is some bloody cold calculation

66
Tloewald 2 days ago 3 replies      
I think the fundamental error here is assuming the existing car ownership model. I think the future is autonomous taxis (or Ubers). This actually reduces congestion, eliminates the need for parking, and plays to the strengths of electric vehicles. Building a master plan based on families owning cars is, I think, skating to where the puck was ten years ago (car ownership is dropping in the Western world, especially among the young).
67
puranjay 2 days ago 0 replies      
This is just mind blowing to me.

This man is dreaming the future. Nay, he is building the future.

68
themark 3 days ago 2 replies      
"...transition the role of bus driver to that of fleet manager..."

Does he really think that ?

69
firewalkwithme 2 days ago 0 replies      
The footer is covering the entire page in my glorious IE11 browser
70
soheil 3 days ago 1 reply      
I'm wondering what the effect of this plan will be on Uber.
71
8note 3 days ago 0 replies      
the question is how to keep those cars clean though
72
rukittenme 3 days ago 0 replies      
> Enable your car to make money for you when you aren't using it

Gives "pimp my ride" a whole new meaning.

73
thruflo22 2 days ago 1 reply      
Car roof or house roof?
74
simonhughes22 2 days ago 0 replies      
Wow. Just wow.
75
crypticlizard 2 days ago 0 replies      
So tesla is in the business of making Model 3 factory factories.
76
lebca 3 days ago 0 replies      
and post Tesla acquisition of SpaceX,

Master Plan: Part Tres

Solar powered flying cars that will take you to Mars and beyond.

77
mgoldberg524 2 days ago 0 replies      
a
78
mitul_45 2 days ago 0 replies      
Tesla should buy Uber to manager all booking, locating taxis and other stuff. Then it would be badass combination!
79
dcw303 3 days ago 2 replies      
> Enable your car to make money for you when you aren't using it

I really like this, but the laws of supply and demand still apply. If you live in a sparsely populated area there's not going to be much for your car to do.

Great for those in urban centres, but then if you lived there why bother owning at all when there will be more cabs to hail?

80
jpeg_hero 3 days ago 1 reply      
Meh.

Sure, stuff sounds neat, but where are you going to get the capital from?

Another secondary share offering?

I guess the most concrete thing I saw was new factory for Model 3. Shouldn't that be your only priority?

Not designing an electric semi truck on paper to entice Joe Q Public into stepping up for another secondary share offer?

81
sonink 2 days ago 1 reply      
If I am on a self-driving car and it meets with an accident, there might be a case to be made against Tesla if I consider myself a very safe driver. Even though on an average self-driving might cause less casualties but that figure might not hold good compared to what my personal accident rate is.

And this should be good enough for law enforcement to nail Tesla.

I think, like everyone else, that Musk is probably the smartest entrepreneurs of our time. In this case though, maybe he is over his head a bit:

- Getting Tesla 3 to production volume will not be easy.

- Autopilot is NOT good enough to be used in production. This can be fatal to Tesla if FCC catches up. Tesla needs to quit Autopilot and focus only on getting Tesla 3 out. Tesla 3 will face competition sooner rather than later and market dominance is not guaranteed.

- SolarCity has absolutely no synergies with this business. It should be sold off.

- SpaceX is again a distraction given how hard it will be for Tesla 3 to roll off.

3
Stack Overflow Outage Postmortem stackstatus.net
854 points by gbrayut  3 days ago   313 comments top 71
1
dkopi 3 days ago 2 replies      
Perfect.Awesome bug. Awesome Post Mortem.This was just fun to read.

While this might have been caused by mistake - these types of bugs can be (and are) abused by hackers.

https://www.owasp.org/index.php/Regular_expression_Denial_of...https://en.wikipedia.org/wiki/ReDoS

The post also links to this video:https://vimeo.com/112065252

2
chubot 3 days ago 5 replies      
Ha! The same bug happened internally at my company. In that case it was a regex matching a URL taking so much CPU as to cause a DOS of a proxy server. I won't be surprised if it's happened to someone here too.

This is very timely, because minutes ago, I made a link to Russ Cox's articles in my Kernighan awk repo:

https://github.com/andychu/bwk

https://swtch.com/~rsc/regexp/regexp1.html

If you are not familiar with this issue, basically Perl popularized bad computer science... "regexes" are not regular languages.

They say that this particular case triggered quadratic behavior, not exponential, but the point is that there is a linear time algorithm to do this.

The file b.c in the awk repo implements the linear time algorithm:

https://github.com/andychu/bwk/blob/master/b.c

(and rsc's site has some nice sample code too, as well as caveats with regard to capturing and so forth)

3
smrtinsert 3 days ago 5 replies      
"This regular expression has been replaced with a substring function."

This should be the title of a book on software engineering.

4
smegel 3 days ago 5 replies      
> If the string to be matched against contains 20,000 space characters in a row, but not at the end, then the Regex engine will start at the first space, check that it belongs to the \s character class, move to the second space, make the same check, etc. After the 20,000th space, there is a different character, but the Regex engine expected a space or the end of the string. Realizing it cannot match like this it backtracks, and tries matching \s+$ starting from the second space, checking 19,999 characters. The match fails again, and it backtracks to start at the third space, etc.

That's not how backtracking works. A regex engine will only backtrack to try and make the rest of the regex match, i.e. it will take characters of the RHS of the string, not try and start "from the second character off the start of the string". I mean, if the engine tried matching from the second space, what would be matching the first space? Something has to.

Which meant, that even if the regex engine was incredibly stupid and could not figure out that a greedy block of \s was never going to contain a non-\s, it would only have to check 20,001 times, not 199000 (or whatever it was).

I can't reproduce this "bug" in either Perl or Python. The time taken to match a 30,000 block of space either followed by $ or XX$ was basically identical for \s+$.

There does appear to be normal backtracking going on, roughly doubling the search time for large strings terminating in non-\s. This is expected, as it has to check 20,000 during the first gobble, then 20,000 as it backtracks from the right 20,000 times.

 $ time perl -e '(" " x 100000000 . "X") =~ /\s+$/ && print "MATCH"' real0m0.604s user0m0.509s sys0m0.094s $ time perl -e '(" " x 100000000) =~ /\s+$/ && print "MATCH"' MATCH real0m0.286s user0m0.197s sys0m0.089s

5
StevePerkins 3 days ago 2 replies      
I'm surprised that a developer was able to fix StackOverflow without being able to look up the error message on StackOverflow.
6
redbeard0x0a 3 days ago 3 replies      
In the past, I have done Load Balancer status checks against a special /status endpoint. I queried all the connected services (i.e. DB, Redis, etc) with a super fast query (i.e. `SELECT version();`). Monitoring CPU/MEM usage for scaling was separate.

Comparing this to checking the home page, what is the best way to setup a health check for your load balancers?

7
alexbecker 3 days ago 3 replies      
I remember the day I learned that Python's "re" module uses backtracking for non-extended regexes. My tests covered lots of corner cases in the regex logic, but were too short for me to notice the performance penalty. Luckily I only caused a partial outage in production.

I actually got to talk to Raymond Hettinger (Python core team) about why re uses a potentially exponential-time algorithm for regexes when there is a famous linear-time algorithm, and (I suspect) most programmers would assume linear complexity. As it turns out, there was an attempt to re-write re to fix this, but the re-write never managed to present exactly the same (extremely large) API as the existing module. He advised me that "the standard library is where code goes to die."

8
StavrosK 3 days ago 4 replies      
I don't understand something: the regex expected a space character, followed by the end of the string. If the last character wasn't a space, this could never match. Why did the engine keep backtracking, even though it's easy to figure out that it could never match the regex?
9
mwpmaybe 3 days ago 1 reply      
This is why I always do:

 s/^\s+//; s/\s+$//;
Instead of:

 s/^\s+|\s+$//;
Weirdly, I've "known" this since I started writing Perl in the mid-'90. Not sure where I originally read it (or was told it). Funny how that works.

I try to write my regexes such that they anchor at the front of the strong or the back, or they describe the whole string; never an either-or anchoring type situation like this example.

Spaces at beginning of string (100,000 iterations):

 Rate onestep twostep onestep 62500/s -- -2% twostep 63694/s 2% -- real0m3.093s user0m3.066s sys0m0.018s
Spaces at end of string (100,000 iterations):

 Rate twostep onestep twostep 55249/s -- -9% onestep 60976/s 10% -- real0m3.453s user0m3.421s sys0m0.022s
Spaces in middle of string (only 500 iterations because I don't want to sit here for four hours):

 Rate onestep twostep onestep 7.11/s -- -100% twostep 16667/s 234333% -- real1m10.741s user1m10.207s sys0m0.228s

10
selckin 3 days ago 2 replies      
Is this the sort of thing that https://github.com/google/re2 was made to solve?
11
mplewis 3 days ago 3 replies      
I think this might have been the post they quoted.

http://stackoverflow.com/questions/38484433/in-corona-sdk-ho...

12
johncoltrane 3 days ago 2 replies      
A few months ago, a Stack Overflow representative asked me if their presence at a dev conference was justified. My positive answer more or less revolved around the importance SO took in the daily life of programmers everywhere.

If only she was there to witness the effect of a 34 minute downtime on an open space full of mobile/back/front developers.

13
junke 3 days ago 1 reply      
Nice bug. I tried to replicate this and indeed, the time to notice that no match is found is growing very fast with the length of the input. Using a substring check is a good fix, but I tried to change the regex to fix this and: if instead of an end anchor, you can add an optional non-whitespace character at the end of the pattern, then you only have to check whether the optional part is empty. Testing with very long strings which respectively match and don't match shows that the result is immediate in both cases.

 (defparameter *scanner* (ppcre:create-scanner '(:sequence (:register (:greedy-repetition 1 nil :whitespace-char-class)) (:register (:greedy-repetition 0 1 :non-whitespace-char-class))))) (let ((length 40000)) (defparameter *no-match* (let ((string (make-string length :initial-element #\space))) (setf (char string (1- (length string))) #\+) string)) (defparameter *match* (make-string length :initial-element #\space))) (defun end-white-match (string) (ppcre:do-scans (ms me rs re *scanner* string) (when (and ms (= (aref re 1) (aref rs 1))) (return (values ms me))))) (time (end-white-match *match*)) 0, 40000 ;; Evaluation took: ;; 0.000 seconds of real time ;; 0.000000 seconds of total run time (0.000000 user, 0.000000 system) ;; 100.00% CPU ;; 25,139,832 processor cycles ;; 0 bytes consed (time (end-white-match *no-match*)) NIL ;; Evaluation took: ;; 0.000 seconds of real time ;; 0.000000 seconds of total run time (0.000000 user, 0.000000 system) ;; 100.00% CPU ;; 11,105,364 processor cycles ;; 0 bytes consed

14
lambda 3 days ago 0 replies      
Hmm. I wonder why one of the followup mitigations is not to move to a non-backtracking regex engine by default.

Most of what you want to do with a regex can be done with an NFA or DFA based engine. That which can't be done with an NFA or DFA based engine is generally better handled with a parser than a regex.

There are plenty of good DFA based regex matchers out there; RE2, the Rust regex crate, GNU grep, etc. At a glance, it even looks like glibc uses a DFA, though it supports POSIX REs which support backreferences so it must use backtracking at least for REs that contain backreferences.

Predictable hash collisions were a big sources of DOS attacks in web scripting languages which use tables a lot, until they started rolling out randomized hashing algorithms to prevent easily predictable hash collisions. It seems like it would be best for languages and libraries to move to DFA based regexps, at least for anything that doesn't contain backreferences, to mitigate these kinds of issues from being easy to exploit.

15
kilroy123 3 days ago 4 replies      
> It took 10 minutes to identify the cause.

I'm impressed they were able to do this so quickly.

16
nanis 3 days ago 1 reply      
As perlfaq4[1] shows:

 > You can do that with a pair of substitutions: > s/^\s+//; > s/\s+$//;
It then notes, in an understated manner:

 > You can also write that as a single substitution, > although it turns out the combined statement is > slower than the separate ones. That might not > matter to you, though: > s/^\s+|\s+$//g;
[1]: http://perldoc.perl.org/perlfaq4.html#How-do-I-strip-blank-s...

17
brongondwana 2 days ago 0 replies      
Time to pop this old chestnut out:

https://blog.fastmail.com/2014/12/14/on-duty/

"At one stage, we decided to try to avoid having to be woken for some types of failure by using Heartbeat, a high availability solution for Linux, on our frontend servers. The thing is, our servers are actually really reliable, and we found that heartbeat failed more often than our systems - so the end result was reduced reliability! It's counter-intuitive, but automated high-availability often isn't."

One of these days we'll finish our new system and I'll blog about that, which is that the automated systems are allowed to take ONE corrective action without paging, at which point they flag that the system is in compromised state. Any further test failures trigger an immediate wake of the on-call.

18
tibiapejagala 3 days ago 1 reply      
I wondered about this for some time.

Simple regex (as in formal language theory) are matched in O(n) time by finite automaton.

Extended regex like PCRE are more powerful, but most of the time are implemented by backtracking engines, where really bad regex pattern might go exponential, but even simple pattern as in postmortem can go O(n^2).

Do implementations optimize simple regex patterns to O(n) matching? Even I wrote x86 JIT regex compiler for fun some time ago. Compilation time was really bad, but matching was O(n).

19
lazyant 3 days ago 2 replies      
"the entire site became unavailable since the load balancer took the servers out of rotation." I don't care about the regexp, this is bad SRE, you can't just take servers out of rotation without some compensation action.

Never mind that it looks like all web servers where taken out of rotation, even one server down could cause a cascading effect (more traffic directed to the healthy ones that end up dying, in a traffic-based failure). One action for example after n servers have gone down, (besides getting up other m servers) is to put (at least some) servers in a more basic mode (read only/static, some features disabled), not guaranteed but that could have prevented this and other type of down times.

20
jakozaur 3 days ago 0 replies      
Experienced something similar myself. Was even thinking about creating regular expression library which just allow "safe" and fast expression.

The trick would be to not allow only expression that can be translated easily to state automate.

Good regex: "Phone number [0-9]* "

Bad regex: ";Name=.;" as . can also match ";" and it can lead to bad backtracking. You should rewrite this regex to ";Name=[^;];"

RE2 is probably best implementation so far, but because it's tries so hard to preserve backward compatibility with all regular expression it is not that fast in average case:https://swtch.com/~rsc/regexp/regexp1.html

21
shanemhansen 3 days ago 0 replies      
Yesterday I couldn't use hipchat for a couple hours because it would lock up a cpu and fail to load. After doing some free debugging for them I realized they were locking up trying to extract urls out of some text with a regex. Simplified code: https://gist.github.com/shanemhansen/c4e5580f7d4c6265769b0df...

Pasting that content into hipchat will probably lock up your browser and webview based clients. Beware.

Lesson learned: don't parse user input with a regex.

22
rixed 3 days ago 0 replies      
Regex was not the main issue.The main issues were:

1. Rendering a page fails/does not terminate if some non essential subtask (rendering a single code block) fails/does not terminate.

2. They do not try to detect bad data (the way they certainly try to detect bad code)

3. Load balancing based on the rendering time of a single page

Code bugs triggered by bad data will happen again, with or without regular expressions.

23
antoineMoPa 3 days ago 1 reply      
Google cache saved me during these 34 minutes.
24
onetwotree 3 days ago 3 replies      
It seems like there should be a way to determine whether a regex can be compiled using the classic O(n) DFA algorithm or with whatever madness PCREs use to support backtracking and so on.

Anybody know if any regex engines attempt this?

Obviously you can still shoot yourself in the foot, but it's somewhat more difficult to do so in a situation like this where the regex in question "looks" cheap.

25
laurencei 3 days ago 4 replies      
Could this has been a deliberate/malicious act? Why else would someone post 20,000 consecutive characters of whitespace on a comment line?

Also, the "homepage" of StackOverflow does not show any 'comments' - it is just the top questions? Why was the page loading any comments in the first place?

26
animex 3 days ago 0 replies      
We had a similar issue arising from regex parsing of our SES routes on our SaaS Platform. We had made some changes to our generated SES file which caused it to balloon to 4x in size (tens of thousands of lines). Our only clue that something had gone wrong was suddenly extremely high IIS usage. With some help from Microsoft support, we managed to trace the stack during the high-cpu event to an ISAPI filter and ultimately our 3rd party SES plugin. We managed to fix the problem by being more efficient with our regex generation and reduce the number of rules the plugin was processing but it was eye-opening how much CPU was being consumed by regex processing.
27
Scea91 3 days ago 0 replies      
I like this because it shows how important it is to understand the inner workings of the tools in your toolbox. It could serve as a nice example in some 'Languages and Grammars' course at the University for additional motivation.
28
cyphar 2 days ago 0 replies      
I'm still confused why people would use a backtracking regex engine in cases when they don't need recursive regex extensions (or other questionable extensions like back references). A "correct" (from the CS perspective) regex engine wouldn't have had this or many other problems that people encounter when doing regular expression matching. If they had piped out to sed or awk this wouldn't have happened, since GNU grep, sed and awk use a proper regex engine.
29
grashalm 3 days ago 3 replies      
Easy to reproduce [1]. Just remove the a in the end and your timeout disappears. Anybody knows which regex engine they used?

[1] http://regexr.com/3drn3

30
revelation 3 days ago 4 replies      
They implemented trim with a regex? Neither Java nor .NET do that.

The postmortem here should probably be "why are you reimplementing trim".

31
ozten 3 days ago 0 replies      
My blog post[1] on how to test for catastrophic backtracking using RegEx buddy.

[1] https://blog.mozilla.org/webdev/2010/11/15/avoiding-catastro...

32
adrianratnapala 2 days ago 0 replies      
Backtracking regexes matchers are a Bad Idea.

It's true you need them to implement backreferences. But I've never used such a thing. If I were creating a runtime for some new language, I would simply ignore that part of the POSIX standard.

33
davidron 3 days ago 0 replies      
The whole postmortem focuses on a regular expression bug and how that bug was fixes and completely ignores the fact that if the home page becomes unavailable, the load balancer logic will shut down the entire site.
34
wfunction 3 days ago 0 replies      
I still haven't figured out why regex engines font use state machines where possible (i.e. in the absence of back references and such). Is that not an obvious optimization?
35
johnwheeler 3 days ago 1 reply      
ugh. i would've just sat there wondering WTF. then proceed to initiate daily backup recovery.
36
OJFord 3 days ago 0 replies      

 > It took 10 minutes to identify the cause,
Impressive, considering:

 > cause was a malformed post that caused one of our > regular expressions to consume high CPU ... called on > each home page view ... Since the home page is what our > load balancer uses for the health check, the entire site > became unavailable since the load balancer took the > servers out of rotation.

37
dear1777 1 day ago 0 replies      
Hmmn, if it was one request, how did it cause other web servers in the farm go down?
38
ozim 2 days ago 0 replies      
For me awesome part is clich that is quite popular on SO takes down SO. And resolution to replace RegExp with substring completes the picture. Just cannot stop laughing.

"Some people, when confronted with a problem, think 'I know, I'll use regular expressions.' Now they have two problems."

39
rocho 2 days ago 0 replies      
By the way, this is the post that broke StackOverflow:http://stackoverflow.com/questions/38484433/join-tiles-in-co...
40
random3 2 days ago 0 replies      
> Add controls to our load balancer to disable the healthcheck as we believe everything but the home page would have been accessible if it wasnt for the the health check

Wouldn't regular users, trying to access the homepage have yielded the same effect?

41
NetStrikeForce 3 days ago 0 replies      
Someone wiser than me said once that if you have a problem and want to fix it with a regex then you now have two problems :-)
42
unethical_ban 3 days ago 1 reply      
Not understanding why backtracking happened. Once it hit a non space, non end character, move on. Nothing before can match the regex.
43
porjo 2 days ago 0 replies      

 > 20,000+19,999+19,998++3+2+1 = 199,990,000
= 200,010,000, not that anyone's counting :)

44
zzzcpan 3 days ago 0 replies      
Seems like there is still no better way to deal with these kind of mistakes than preemptive Erlang-style lightweight processes.
45
bshimmin 3 days ago 0 replies      
In reading this post, I realised this was the first time I'd ever visited the Stack Overflow homepage.
46
babuskov 2 days ago 0 replies      
> This regular expression has been replaced with a substring function.

I always cringe when I see regex used for such simple string checks. In fact, Stackoverflow is full of accepted answers that "solve" problems that way.

47
JBiserkov 2 days ago 0 replies      
The Stack status page contains 3 script tags before the HTML tag.

This is what I saw on my Kindle 3 Keyboard:

This page contains the following errors:

error on line 2 at column 36: Extra content at the end of the document

Below is a rendering of the page up to the first error.

var __pbpa = true;

48
jimjimjim 3 days ago 0 replies      
paging jwz. something something two problems.
49
_RPM 3 days ago 2 replies      
They have limits on everything (comments per second, edits per second, upvotes per day, reputation earned per day, etc), it seems like they should have an upper bound character limit on what they accept too.
50
GnarlyWhale 2 days ago 0 replies      
Favourite comment from the Reddit thread on the matter:

"Well, that should stave off the imposer syndrome for another couple of days."

-u/minno

51
brokencube 2 days ago 0 replies      
Correct me if I'm wrong, but couldn't this could have been fixed by making the match possessive:

^[\s\u200c]++|[\s\u200c]++$

That should stop any runaway backtracking?

52
mtokunaga 3 days ago 0 replies      
" This regular expression has been replaced with a substring function." I came to rely on Regex so much that I almost feel we'd be the next.
53
Retr0spectrum 3 days ago 0 replies      
For more bugs caused by quadratic complexity:

http://accidentallyquadratic.tumblr.com/

54
jng 3 days ago 0 replies      
Any more proof needed that caching should become a system-provided service over the next 10-20 years, the same way memory management did in the past 10-20 years?
55
berkut 3 days ago 0 replies      
If it was in a comment, why was the home page loading it?

preemptive caching?

56
stop1234 2 days ago 0 replies      
Yes, one of the best postmortems, especially the technical part.

Am sure it was simple but curious to know what the replacement substr code is.

57
hamzalive 2 days ago 0 replies      
200010000 not 199990000 probably the author looped on a 0-based index. n*(n+1)/2 is even better ^^Nice post mortem though
58
zkhalique 3 days ago 0 replies      
This is great. I just want to add something that might not be well-known: StackOverflow is all hosted from ONE web app server! It handles all the writes.
59
percept 3 days ago 0 replies      
Productivity plummets worldwide (regex attack vector)
60
Osiris 2 days ago 0 replies      
Why isn't the trim applied when the post is created and not every time that it's displayed?
61
estrabd 3 days ago 1 reply      
TIL what language Stack Overflow is written in.
62
MalcolmDiggs 2 days ago 0 replies      
Regex: ruining your life since 1956.
63
rmdoss 3 days ago 0 replies      
Very interesting bug. People forget some times how expensive a regex can be compared to simple pattern matching.
64
Waterluvian 3 days ago 0 replies      
I want to believe that a cat fell asleep on the space bar. Then eventually woke up and posted.
65
davidwparker 3 days ago 0 replies      
This is great- regex errors always reminds me of this classic Jeff Atwood post (cofounder of StackOverflow): https://blog.codinghorror.com/regular-expressions-now-you-ha...
66
rosstex 3 days ago 0 replies      
Wow, I didn't notice today! I must not have been coding very much.
67
hstun 3 days ago 0 replies      
But... how did they search for a fix without resorting to Stack Overflow? :)
68
monochromatic 3 days ago 5 replies      
> So the Regex engine has to perform a character belongs to a certain character class check (plus some additional things) 20,000+19,999+19,998++3+2+1 = 199,990,000 times, and that takes a while.

199,990,000 isn't really all that many. I'm a little surprised it didn't just cause a momentary blip in performance.

edit: whoops, i guess that's per page load

69
fweespeech 3 days ago 2 replies      
The lesson seems to be "Always run trim() before running regex" and "validate content as much as possible before running regex".
70
yeukhon 3 days ago 1 reply      
This seems like a hard-to-expect edge case for real. I think catching edge case is needed (means more rigorous testing). This is the equivalence of algorithm complexity analysis. How bad can my algorithm be? But regular expression, to be honest, is usually something I hardly think about performance. I don't know about others, but most of the my input are small enough. How big of an input should I test? If I were to deal with a lot of characters, I would be doing substring replacement.
71
avar 3 days ago 3 replies      
My rephrasing of their follow-up actions:

* "Audit our regular expressions and post validation workflow for any similar issues"

* ==> "Not even people who've worked for years on the guts of regex engines can easily predict the runtime of a given regex, but somehow our engineers will be expected to do that".

* "Add controls to our load balancer to disable the healthcheck as we believe everything but the home page would have been accessible if it wasnt for the the health check"

* ==> "Our lb check was checking /index, that failed because /index was slow: Lesson learned, let's not lb check anything at all"

4
A practical security guide for web developers github.com
842 points by zianwar  2 days ago   65 comments top 24
1
buckbova 2 days ago 5 replies      
Had this SO link saved since probably soon after it was asked 7 years ago. Still relevant and still being updated.

http://stackoverflow.com/questions/549/the-definitive-guide-...

Includes:

- How to log in

- How to remain logged in

- Managing cookies (including recommended settings)

- SSL/HTTPS encryption

- How to store passwords

- Using secret questions

- Forgotten username/password functionality

- Use of nonces to prevent cross-site request forgeries

..........

And much much more.

2
niftich 2 days ago 1 reply      
Efforts like this are very good.

But one of the most serious problems with web development is how few frameworks ship with most of these sane answers out-of-the-box (edit: or don't ship concepts at the right level of abstraction)

When we all need to copy-paste some best-practice way of how to Argon2 a password and how to constant-time equality check a hash, we've already lost, in that we're reimplemeting these sane answers every time from the weeds.

I want to see more things like Django's automatic password hash upgrading [1].

Specifically, checklists like this effort's should be for people who develop frameworks, and not people who develop custom apps with them. With some things like CSRF protection, we're already there, but with so many other things, we're not.

[1] https://docs.djangoproject.com/en/1.9/topics/auth/passwords/...

3
0xmohit 2 days ago 0 replies      
Security Engineering: A Guide to Building Dependable Distributed Systems by Ross Anderson is available online for reading -- http://www.cl.cam.ac.uk/~rja14/book.html

A couple of other resources:

- 7 Security Measures to Protect Your Servers [0]

- SSH best practices [1]

In case one doesn't prefer to be overwhelmed with documentation, one could refer to: My First 5 Minutes On A Server; Or, Essential Security for Linux Servers [2].

[0] https://www.digitalocean.com/community/tutorials/7-security-...

[1] http://www.cl.cam.ac.uk/~rja14/book.html

[2] https://plusbryan.com/my-first-5-minutes-on-a-server-or-esse...

4
br3w5 1 day ago 1 reply      
Rather than just "JWT is awesome..." wouldn't it be more sensible and responsible to caveat this with some of the drawbacks?

I read this article recently (http://cryto.net/~joepie91/blog/2016/06/13/stop-using-jwt-fo...) that proposes not to use it for sessions but instead for the use cases listed at the end of the article. Follow-up article here http://cryto.net/~joepie91/blog/2016/06/19/stop-using-jwt-fo...

Also this https://auth0.com/blog/2015/03/31/critical-vulnerabilities-i...

5
laurencei 2 days ago 1 reply      
A great book is The Web Application Hacker's Handbook: Finding and Exploiting Security Flaws, 2nd Edition [0] - by learning how hackers search for and exploit various web issues - you'll be naturally aware of them to defend against. i.e. start thinking like a hacker and you'll be amazed at the issues you discover in your applications.

[0] http://eu.wiley.com/WileyCDA/WileyTitle/productCd-1118026470...

6
danneu 1 day ago 2 replies      
There's a whole section on input sanitization but nothing on escaping output.

If you're on the hook to sanitize all inputs, doesn't that mean you're not escaping output?

The biggest security mistake I've made so far in production was that one time I used an HTML templating library that didn't escape output by default.

7
fmavituna 1 day ago 1 reply      
Use static source code analysis and dynamic web app scanners.

They are easy to integrate into your SDLC, they are not going to replace manual testing or secure development practices but they'll help a lot. They'll pick up tons of stuff for free, they'll remind you best practices.

I have a startup (at least it still feels like a startup!) and we are developing a web application security scanner called Netsparker [0]. It found over 100 zero days in open source applications while testing it [1], including very popular vulnerabilities in applications such as Wordpress and Joomla. I guess that by itself proves how good scanning can be.

If you want to try it on your websites and see it for yourself drop an email / message to contact@netsparker.com with a mention of HN and I'll get you a fully functional trial that you can use on your own websites.

[0] Netsparker Cloud https://www.netsparker.com/online-web-application-security-s... - Netsparker Desktop https://www.netsparker.com/web-vulnerability-scanner/

[1] https://www.netsparker.com/web-applications-advisories/

8
andersonmvd 2 days ago 1 reply      
The problem with checklists, including this one, is that we tend to limit ourselves to what's in the list. Furthermore the list doesn't explain 'why' you should do things. They help, but nothing is a replacement for education. And when it comes to education, there's a decent write up I did and is still accessed in a daily basis [0]. I also recommend you to check OWASP [1] and read their "Testing Guide" to know many attacks and defenses.

[0] Security for building modern web apps https://dadario.com.br/security-for-building-modern-web-apps...[1] https://www.owasp.org

9
bogomipz 2 days ago 1 reply      
Where is the actual guide? Is this the TOC for a book? It looks good but I don't see the actual content, just a check list and a table of contents.
10
aruggirello 1 day ago 1 reply      
> Store password hashes using Bcrypt (no salt necessary - Bcrypt does it for you).

In PHP, I would rather recommend to use password_hash() with its own defaults since it's built-in and designed specifically for this purpose - and quite future-proof. But this is PHP specific.

> [] Destroy all active sessions on reset password (or offer to).

> ...

> [] Destroy the logged in user's session everywhere after successful reset of password.

I believe these are the same. The second one is clearer though.

Edit: clarified

11
droopybuns 1 day ago 0 replies      
This is one of the best examples of how Github nails collaborative document development I've seen.

It is striking how valuable much information is retained in negotiating the material here, vs email arguments with word documents and embedded content, where the app-seperation of submissions makes it too difficult to consume.

12
DrJokepu 2 days ago 2 replies      
* Don't let HTTP GET requests modify state, ever. It's very difficult to prevent CSRF via HTTP GET.

* Session keys are password-equivalents. Hash them with bcrypt or something before you store them.

* httponly is not incredibly useful. If the attacker can run JavaScript on your page, you're in trouble.

13
arekkas 1 day ago 0 replies      
"Check for no/default passwords for databases especially MongoDB & Redis. BTW MongoDB sucks, avoid it."

Come on, you're better than this. What the fuck.

14
ckastner 1 day ago 0 replies      
The following PDF focuses on just one specific aspect of security: cryptography, but deserves a mention nonetheless. Configuring various services such that insecure mechanisms are not used is not exactly a trivial task.

https://bettercrypto.org/static/applied-crypto-hardening.pdf

Edit: GitHub repo at https://github.com/BetterCrypto/Applied-Crypto-Hardening

15
pjmorris 1 day ago 0 replies      
The guide seems to have reasonable technical measures. I would like to see more discussion of risk, both in terms of what is being protected, and of who might be trying to attack. For example, you might wish to be more careful when developing a bitcoin wallet than when tracking baseball scores.

Shameless plug: I've been working on a somewhat less practical guide to software development security practices [1]. Even more shameless plug: I'm currently running a survey of security practice use in software development [2], and would welcome participants who work on open source projects.

[1] http://pjmorris.github.io/Security-Practices-Evaluation-Fram...

[2] https://ncsu.qualtrics.com//SE/?SID=SV_1HdQOa2lfX57vkF

16
vog 1 day ago 0 replies      
This reminds me of:

"The Basics of Web Application Security"(Cade Cairns, Daniel Somerfield)

http://martinfowler.com/articles/web-security-basics.html

It's an ongoing evolving publication at Fowler's website.

17
CiPHPerCoder 2 days ago 2 replies      
Please excuse me if this comes across as anything other than constructive criticism, but I don't believe checklists should be used to guide web developers to build secure software.

My reason for this belief is that, in my experience, it engenders tunnel vision and what I appropriately refer to as a "checklist mentality". There are developers who believe, "We're immune to the items on the OWASP Top 10, so we're secure," when there are entire classes of vulnerabilities that applications can be vulnerable to (say: using a weak and predictable PRNG for their password reset tokens) that isn't adequately described by the OWASP Top 10.

An alternative approach that I feel is more helpful is to organize insecurity into a taxonomy.

 * Code/data confusion * SQL Injection * Local/Remote File Inclusion * Cross-Site Scripting (XSS) * LDAP Injection * XPath Injection * Several memory corruption vulnerabilities * Logic Errors * Confused deputies * CSRF * Failure to enforce access controls * Operating Environment * Using software with known vulnerabilities * Using HTTP instead of HTTPS * Cryptography flaws * Yes, this deserves a category of its own * Chosen-plaintext attacks * Chosen-ciphertext attacks * Side-channel cryptanalysis * Hash collision vulnerabilities (e.g. length-extension) * Weak/predictable random values
You can further break down into more specific instances.

There are three types of XSS (stored, reflective, DOM-based). There are blind SQL injection techniques worth studying too. But the underlying problem that makes these vulnerabilities possible is simple: User-provided data is being treated as code. Any technology that prevents this confusion will greatly improve security.

For example: SQL injection is neutered by using prepared statements. You might one day forget to manually escape a single input (and it only takes one to be game over), but if user data is always passed separately from the query string (i.e. you never concatenate), there's no opportunity to make this mistake. There were also corner-case escaping bypass attacks (usually involving Unicode) that you might not be vulnerable to. With prepared statements, these clever multibyte character tricks accomplish nothing. The query string is already in the database server before your user's parameters are sent.

I believe teaching developers to think in terms of taxonomy (very general to very specific) will result in a greater understanding of software security and reduce the incidence of vulnerable code.

I've written about this before, in case anyone wants to link to something besides an HN comment: https://paragonie.com/blog/2015/08/gentle-introduction-appli...

---

EDIT: Opened an issue: https://github.com/FallibleInc/security-guide-for-developers...

18
ronreiter 2 days ago 1 reply      
This looks like a great candidate for stack overflow documentation.
19
yeukhon 2 days ago 2 replies      
The first thing jumped out is:

Store password hashes using Bcrypt (no salt necessary - Bcrypt does it for you)

A better approach would be recommending storing password with password-based key derivation functions (recommendation: scrypt or bcrypt).

I don't want to start the whole debate of scrypt vs bcrypt, GPU vs FPGA here (not qualified and we keep repeating the conversation every time the vs is on the table).

-

When parsing Signup/Login input, sanitize for javascript://, data://, CRLF characters.

Not familiar with why only "signup / login" input.

-

Serially iterable resource id should be avoided. Use /me/orders instead of /user/37153/orders. This acts as a sanity check in case you forgot to check for authorization token.

Had to re-think twice. A stronger argument in favor of /me/orders over /user/37153/orders is to avoid enumeration attack.

-

Any upload feature should sanitize the filename provided by the user.

I like this tip very much, but if the requirement is to keep the filename (think DropBox), you should sanitize filename before storing in the database.

-

Add CSRF header to prevent cross site request forgery.

I don't believe there is a standard header to do this. I had to look up "csrf header". Correct me if I am wrong. I think this is framework specific (if and only if framework supports it). A better recommendation would be to enable CSRF protection and consult with the framework you use. Most modern frameworks would have CSRF protection built-in (but implementation of CSRF protection varies!!!!!!)

-

Add HSTS header to prevent SSL stripping attack.

Simply put, after the user has visited a site with both HTTPS and HSTS header present, user agent like Firefox will honor the header and always attempt to load resource (HTML, JS, CSS) over HTTPS up to the max-age declared in the header. The caveat is you must have visited the HTTPS site first. To actually implement HSTS 100%, you should always redirect user (301) to HTTPS from HTTP.

-

Use random CSRF tokens and expose business logic APIs as HTTP POST requests.

Needs clarification on what "business logic" mean and why POST only? What about PUT and PATCH which also allow body to be used? GET I kind of get it.

-

If you are small and inexperienced, evaluate using AWS elasticbeanstalk or a PaaS to run your code.

Again, caveat is doing everything right. PaaS and IaaS shields you away from some common mistakes, but not all. You can have a remote code execution on an EC2 with instance profile with full access to the entire VPC and the execution is to remove all instances except the one it is on. Perfect.

-

Use a decent provisioning script to create VMs in the cloud.

I have to be a little picky... don't say decent. This is so ambiguous. Did you mean don't reinvent the wheel, or did you mean have a solid engineering process (code review, testing), treating infrastructure automation as software engineering as opposed to an ad-hoc scripting shop.

-

Check for no/default passwords for databases especially MongoDB & Redis. BTW MongoDB sucks, avoid it.

I get it. You own the article you write whatever you want. Professionally, if you want someone to take this serious, please don't say that. I have seem people running Oracle just as good as running PostgreSQL. I have heard companies running Apache as successfully as running Nginx. I have heard horror story about Cassandra and success story about Cassandra. MongoDB has a few old mistakes like default to listen on 0.0.0.0 (I heard it is fixed by now?). BTW, I have used and have managed MongoDB, I know some of the pains, but half of that came out of not knowing what the hell I was doing.

-

Modify server config to use TLS 1.2 for HTTPS and disable all other schemes. (The tradeoff is good)

Right tradeoff is to use data and figure out whether or not you need to support legacy systems like those stuck with XP. It may not be critical, but there are companies that do. Use data first before making a decision like this.

-

Four other thoughts.

1. sanitization of inputs - context matters. The same sanitation technique for HTML doesn't work for XML. That to me is one of the most complicated part in securing application. I am not surprised XSS is still #1 (or at least top 3).

2. Run code in sandbox mode. Not necessarily Docker or a container, but chroot and restrict application access to the available system resource. That's very important.

3. Always use reputable framework. As a young adult I love inventing shit, but whatever you invent for your work you are now responsible and the next person picking up your work after you leave is also responsible. So think twice. I am not picking on Node developers because I have seen Python developers doing the same thing - before you import a random package that does a few thing, look at the standard library. Sometimes maintaining a 100-line code yourself vs doing in two lines after an import from the code is written by a random open source lover and is mostly won't be maintained a few years from now is dangerous.

4. Always upgrade your framework, the toolsets you use, database server you use, etc.

I also think every framework should publish security best practice like https://docs.djangoproject.com/en/1.9/topics/security/ and even more details. Security is one of those things I'd wish I had more time to experiment and address. I am no longer active in that space of automation sadly, but from time to time I think about is the fault really on development practice and developers? Can we make everyone's life easier by having strong framework standard? Are we not making tools available? With so many formats being invented every year, we need to think about is our security flaws a result of our own creativity? Unfortunately, we can only hope for the best that we continue to improve security of our framework and we continue to add strong defaults. Also think about security testing. The low hanging fruit like detecting existence of certain security headers is trivial, but fuzzing and getting real good result of vulnerability within an app is extremely custom AND extremely hard to do (so many states, so little knowledge)... you've got very expensive consultants and then very inexpensive but also very general purpose security testing tools that may not do much and can expose common mistakes. One thought would be sampling and either repeating or mimicking user traffic and run simulations. Perhaps some machine learning stuff could help - not sure.

20
dionys 1 day ago 0 replies      
I saw this cool site a while ago:https://www.hacksplaining.com/lessons

It explains basic vulnerabilities in a very simple way and offers specific ways of avoiding them in different languages.

21
kevindong 2 days ago 0 replies      
I thought this line in the checklist was rather interesting:

> Check for no/default passwords for databases especially MongoDB & Redis. BTW MongoDB sucks, avoid it.

22
saasinator 2 days ago 0 replies      
I didn't see any mention on how to secure store the session id, only references to session data. It should be noted that information needs to be securely stored both on the client and server.
23
zkhalique 2 days ago 1 reply      
One thing I totally disagree with:"Set secure, httpOnly cookies."

That is just security theater. It's worse than useless because it makes you think you're more secure, when you haven't prevented attacks are all.

24
lukiebriner 1 day ago 1 reply      
My problem with this is another howler for security:

Creating something that already exists.

Although OWASP are not legally mandated, they are the most respected go-to people for this kind of stuff and have much more exposure that your "guide" ever will, it also has a much greater level of review and scrutiny so instead to trying to help by increasing the web noise level and possibly making your own mistakes/ommissions (some of which are mentioned below), why not instead get engaged into the existing community and increase the quality of that if needed?

5
When a crow dies, the other crows investigate the cause of death (2015) nationalgeographic.com
622 points by reimertz  4 days ago   228 comments top 49
1
SiVal 3 days ago 10 replies      
A VERY timely article for me. About a month ago here in Silicon Valley, I noticed that a crow had been violently torn apart and the pieces scattered all over my backyard lawn. I assume the killer was a raccoon--another improbably intelligent animal. How those fat, little ninjas do what they do is beyond me, but one had apparently caught a crow. A few hours after I noticed the carnage, I grabbed a paper grocery bag and some rubber gloves and went outside to collect the crow parts.

As soon as I touched the first piece (a large, black, detached wing), a dozen crows appeared out of nowhere flying in tight circles over my head (about the height of the roof of my 2-story house) and shrieking. They must have been standing watch for hours waiting to see what would happen. Within a minute or so, their numbers had doubled, swarming like bees and screeching. They went so berserk that I thought for sure they would swoop down and peck at me like Alfred Hitchcock's "The Birds", but they didn't. They stayed up there and screeched the whole time I was cleaning up (maybe ten minutes).

I took the bag over to the recycling bin on the side of the house. Property is expensive, so the houses are close together leaving only a narrow slit of sky above me. The crows followed me and flew back and forth right above the gap, still screeching at me. I hadn't killed the bird, but they were acting like an angry mob blaming the wrong guy.

I was already aware of the studies showing that crows recognize individual people and can bear grudges for years. I was afraid that's what I was going to end up with, but after that event, they never bothered me again, and I see crows around my house frequently. Maybe they DID know that I wasn't the killer, but they had some other agenda. From my perspective, they (and raccoons) are essentially alien intelligences living among us that I always underestimate and still don't understand.

2
sharkweek 3 days ago 7 replies      
So fascinating!

There is a young girl in Seattle who made friends with the local crows by feeding them, they soon were bringing her regular gifts in exchange for her snacks - http://www.bbc.com/news/magazine-31604026

I tried to do the same thing in my backyard for about three weeks until one day my wife caught me putting little pieces of bread on our porch railing. She asked what I was doing and I explained making friends with the crows (what else?!). She asked me to stop so I did.

My only hope is that they attack her now and not me, with some level of understanding that she was the one who ruined all the fun...

3
sirtastic 3 days ago 5 replies      
I believe ravens and crows are very similar, this is a personal (long, and not the great) story about a raven I messed around with in Yosemite National Park:

When I was up on north dome (not to be confused with half dome) there was a group of raven's hanging out on the rocks watching us eat our late lunch. I had an apple core that I tossed to the side and watched as a raven warily tried to approach it. I walked over and grabbed the apple core before the raven could grab it so I could entertain myself teasing the raven for a bit before we started down the mountain. I started by putting my arm back ready to throw the apple and as I did that I noticed the raven kneel a bit as if getting ready to launch itself. I thought this was interesting as it showed it was anticipating me throwing the object based on my arm motion. As I relaxed my arm the bird also relaxed.

I tried grabbing a rock and again watched it brace itself to launch from the rock then tried switching the apple core and the rock behind my back and tossing the rock hoping the bird would dart after it thinking it was the core. The bird didn't do as I expected and instead just watched me carefully never motioning for the rocks.

I tore a piece of the apple from the core behind my back and tossed it just as I had with the rocks and before the piece of apple even left my hand the raven leaped from the rock in its direction.

This blew my mind. Not sure how the raven knew it was a chunk of apple and not a rock.

I messed around with that particular raven for a good 10-15 minutes tossing various things in its direction, testing its reactions and trying to mess with its little raven mind. All I managed to do however was be impressed at its level of experience in dealing with Yosemite tourists such as myself.

4
ogig 3 days ago 0 replies      
I've been very involved with the carrion crows near my house. I feed them with peanuts. At first they wouldn't even come near the food. Now they salute, ask for food, do fly bys and they get extremely close to me. I have observed them long hours.

What most amazes me is how they communicate. They group and start talking complex things using low volume sounds. Maybe they discuss how to stash my peanuts, or what side they will use to approach me, or who will stay back and watch for dangers.

They also do long distance talking. At the mornings mainly, the group will remain silent while a designated one will song loud. You can hear distant groups responding. Sometimes this long distance chat start group movements to where I can't follow them.

Crow are intelligent, amazing animals. I encourage everyone to look at them more often, they do look at us all the time.

5
nl 4 days ago 1 reply      
This whole article is pretty interesting, but the last paragraph is worth quoting:

For instance, in 2008 Marzluff had researchers in caveman masks capture crows while others in a control maskDick Cheneylet the birds be. Afterward the birds ignored the harmless Cheneys but scolded and chased the cavemen, and did so for years.

Poor birds! Probably thought they could trust a Crow-Magnon.

Dick Cheney masks! Probably thought they could trust a Crow-Magnon. I have faith in the world again.

6
adamnemecek 4 days ago 6 replies      
If you find this interesting, you should really look into the intelligence of crows. They've been observed to remember human faces, wait on traffic lights, plan, and a whole bunch of things that are very impressive.

And don't even get me started on _jackdaws_.

7
nerone 3 days ago 1 reply      
For last 2 years, I am suffering from depression. I decided to isolate myself, which I knew was wrong. My parents were worried about my situation since I am living with them. I live in a village where this sort of behavior is condemned, and people are very hardworking(they are either farmers, fishermen, construction workers/helpers).What I noticed was, whenever I go outside of my house a bunch of crows will gather around and caw at me, until I disappear from their site. But they wont follow me in the streets. Some times more than 30 crows will gather near the trees, and kinda shout at me. (I always felt like they were yelling at me, to get a life :) )

So I decided to give a try to get more disciplined. From past to months I wake up at 5 am for gym, and have a routine life. Now none of the crows notice me :).

My explanation: These crows can sense how people live and behave. And they found out I was very odd in the community. :) :)

8
fbonetti 3 days ago 1 reply      
The image of the researcher wearing a mask and holding a dead crow is absolutely terrifying:

http://i.dailymail.co.uk/i/pix/2015/10/03/00/2D0746BE0000057...

9
nstart 3 days ago 2 replies      
So here's a story for the books. I live in Sri Lanka and we have massive numbers of crows always flying around in the urban areas causing quite the racket too. They are exceptionally intelligent and have been witnessed hitching bus rides regularly from one town to another. But the story I have happened at a school water polo match.

The location the match was being played at had a standard pool and a shallow junior pool right next to each other. During the match a wild attempt to score a goal missed the side and the ball smashed full force into a crow who was drinking water from the junior pool. This of course resulted in the poor crow's death. But what was amazing was that within a few minutes, every other crow in the vicinity had surrounded the dying crow and then decided that they would attack the spectators and the players. We had to clear the entire stadium and pool for a good half an hour till they allowed us to return (although they left their dead comrade behind who had to be cleared away by a hapless cleaning staff member).

One less impressive story but still relevant to the article. We had an antique air rifle at home that needed repairs. My uncle who did such repairs said he'd come along and look into it. That evening there were plenty of crows around as usual. Most of them were in the trees two houses away. I assume these were crows who had forever lived in urban areas and probably never been shot at let alone seen a gun. They've lived through plenty of fireworks and don't seem all that bothered by them whenever they do go off. But when I brought the gun out to show to my uncle, the ruckus caused was utterly deafening. And just like that, silence. Every single crow had scattered from the area. How they recognised an object like that and so instantly is a question and memory that has stuck around with me forever.

Amazing creatures who annoy me with their never ending din and tendencies to somehow make a mess of any garbage bag left out in the open, but they'll always have my absolute respect for their intelligence.

10
ProfChronos 3 days ago 1 reply      
Fascinating study. It really triggers questions around how animals' memory work - short term vs long-term, trauma vs joy, etc. I have a 2-year French shepherd (Beauceron) who, while being smart, has a very limited long-term memory. I am almost convinced that he cannot make the difference between waiting for me 5 minutes and 5 hours. At the same time, his trauma or joy memory works incredibly well: he perfectly remembers people he only saw 2/3 times when he was 2 months or objects that hurt him. I always feel like animals are trapped between an absolute lack of time and space consciousness and an incredibly sharp conscious of feeling
11
sverige 4 days ago 1 reply      
Mockingbirds also remember faces. They will attack if they think you are a predator. I had one who decided my daily walk was a threat - maybe I walked too close to its nest - and came straight at my face. Scary as hell. That bird then swooped to attack every time I went outdoors. To Kill a Mockingbird indeed.
12
DonHopkins 3 days ago 2 replies      
So is the full collective noun "A Murder Investigation of Crows"?
13
cossatot 3 days ago 0 replies      
My wife and I picked up a seemingly injured young crow from our yard before the neighborhood cats (or buses) found it, and we took it to a local wounded bird shelter. The crow's family(?) crowed incessantly whenever we walked outside for a few months afterward. They would even start when we were several blocks away, coming home from farther out, and would follow us to our door. (The young crow apparently had a head injury, and we never heard back from the place on its recovery.)
14
steveax 3 days ago 1 reply      
A couple of years ago, near dusk, I heard a hell of a bird ruckus and went out to the front porch to have a look. What I saw blew me away: a young coyote was making haste down the middle of the street, ears down and tail between its legs being pursued by a very noisy aggressive murder of crows. They literally chased him out of the neighborhood.
15
helloworld9 3 days ago 0 replies      
Quote from the Quran 5:31 [1] "Then Allah sent a crow, who scratched the ground, to show him how to hide the dead body of his brother. 'Woe to me!' said he. 'Was I not even able to be as this raven, and to hide the shame of my brother?' Then he became full of regrets."

[1] https://en.wikipedia.org/wiki/Cain_and_Abel_in_Islam

16
John23832 3 days ago 2 replies      
This may sound weird, but in the country (rural Virginia) we already kind of knew this.

If you had a farm and crows get in your crops, the solution was to shoot a crow and leave it. The rest of the crows would stay and circle, but they would leave the crops alone.

* Not advocating violence against animals. Just sharing that tidbit.

17
sdkjfwiluf 3 days ago 0 replies      
On my way to work one morning I came across a raven funeral. I didn't know that's what it was at the time. All the ravens (Australian) were gathered broadly in a disk amongst the trees, there were about 20 of them, all quiet and quite still. I had a choice of routes, either through them or around them to the right, I chose the latter as I didn't want to disturb whatever they were doing.
18
MichaelMoser123 3 days ago 0 replies      
Crows also seem to be doing analogies http://www.scientificamerican.com/article/crows-understand-a...

Douglas Hofstadter says that thinking is all about making analogies, so that is all pretty remarkable.

https://www.amazon.com/Surfaces-Essences-Analogy-Fuel-Thinki...

19
emilong 4 days ago 0 replies      
To be fair, I also have unfavorable reactions when faced with mask-wearing people holding dead crows. ;)
20
rollthehard6 3 days ago 0 replies      
The notorious Doritos shop lifting sea gull of Aberdeen https://www.youtube.com/watch?v=Kqy9hxhUxK0And another in South Shield, who favours Greggs - https://www.youtube.com/watch?v=S4QXyUjQCgE
21
prestonpesek 3 days ago 0 replies      
Scientists? Or WITCHES!?!?!?

Seriously though, the human behavior study on this is as fascinating as the animal one, and would be really hard to explain in a Salem courthouse.

And the picture of the masked scientist holding a dead crow? Add that to the library of images that my subconscious will use to manufacture nightmares.

Edgar Allen Poe would be proud if he could see this masterpiece of creepiness.

22
rudedogg 4 days ago 4 replies      
My dog has been on a killing spree this summer, sometimes catching 2 birds in a week :(.

It's amazing how the birds react when she catches one. They all go nuts, and more fly in and start squawking.

I worry they'll call in a hawk to carry her away lol.

23
bpp 3 days ago 2 replies      
Holy crap the photo of the volunteer in the mask is terrifying.
24
agumonkey 3 days ago 2 replies      
Probably a ridiculous claim, but I think ants have a ant death detection system too. After being invaded, putting dead ants on the path they used to reach the food drawers made them backtrack immediately and stop using that trail altogether.
25
betolink 4 days ago 1 reply      
"...Afterward the birds ignored the harmless Cheneys" No such a thing as a harmless Dick Cheney.
26
enraged_camel 3 days ago 0 replies      
By the way, when there are crows around, do not stare or point at other birds' nests if you spot any among the tree branches. Crows are very good at reading human gestures. If they see a bunch of humans staring and pointing at something, they will understand that there is something of interest in that direction, and will inevitably find and destroy the nests.
27
andyidsinga 3 days ago 0 replies      
I've always liked this one about the raven and bald eagle: https://www.youtube.com/watch?v=Z0w9q125TSI

BTW - I once read somewhere that its its illegal to keep native birds (like crows, robins etc).

28
dmix 3 days ago 1 reply      
> Each volunteer was either holding a dead crow, standing near a dead red-tailed hawka crow predatoror standing near a dead red-tailed hawk holding the dead crow.

I had to reread this about 5 times to make sense of it. Am I just tired from a long work day or is that a poorly phrase sentence?

29
slavik81 4 days ago 1 reply      
I seem to have found a video of a magpie funeral. They're also corvids, like crows and ravens.

https://youtu.be/60Zg9sGnQf8

30
conjectures 3 days ago 0 replies      
Very interesting. I'd love to know how many calls were made to the local police station reporting people in creepy masks standing around with dead crows.
31
btbuildem 3 days ago 0 replies      
> For instance, in 2008 Marzluff had researchers in caveman masks capture crows while others in a control maskDick Cheneylet the birds be. Afterward the birds ignored the harmless Cheneys but scolded and chased the cavemen, and did so for years.

I feel like they should have swapped the masks around for this one.

32
Speakeasys 4 days ago 1 reply      
That mask is super creepy.
33
Dowwie 3 days ago 0 replies      
Ravens played a role in norse mythology, serving as "shamanic helping spirits" who scout for the god Odin. https://en.wikipedia.org/wiki/Huginn_and_Muninn
34
douche 3 days ago 0 replies      
Growing up with a large garden out in the country, we always had a lot of trouble with crows picking newly-sprouted corn. They'd go right along the row and pluck out the seedlings to eat the kernel at the root. At least, they would, until my father would manage to get one with the shotgun and then hang it up from the scarecrow. After that, there wouldn't be a crow in sight for months.

They are smart creatures.

35
viggity 3 days ago 0 replies      
More anecdata - there were a bunch of crows who were harassing my elderly outdoor cat. So I grabbed my BB gun and shot one from my bedroom window. The two other companions flew off, but when I went to go dispose of the body, 6 more birds flew over to see what I was doing. Cawing at me the whole time.

I didn't want to kill the damn thing. But, our whole street was crow free for at least 2 years.

36
MistahKoala 3 days ago 0 replies      
Gulls behave similarly, being 'attracted' to predators that have caught a gull and circling in order to learn about their behaviour.

https://web.stanford.edu/group/stanfordbirds/text/essays/Gul...

37
runamok 3 days ago 0 replies      
I saw a crow get hit by a car right in front of me with a thump because it moved too slow from a mid street meal. Instantly probably 6 or so crows started shrieking and came over to the incident. It was quite sad because it seemed they could tell that something terrible had happened.
38
gerbilly 3 days ago 0 replies      
Reminds of this video where a crow and a feral kitten are friends. The crow even feeds the kitten!

https://www.youtube.com/watch?v=-fAGzY9rnaA

39
codezero 3 days ago 0 replies      
I've rescued several injured crows and every time I've been followed closely by at least two other crows as I brought the crow to my car. Amazingly intelligent and social creatures.
40
ionwake 3 days ago 0 replies      
Are there any academics here who could shed light on who is considered more intelligent - a Crow or a Raven?

I am aware Ravens are known to mimic sounds more than Crows, unless I am mistaken.

41
nxzero 3 days ago 0 replies      
Reminds me of this video:

Epic Cat Fight w/ Two Birds

http://m.youtube.com/watch?v=WQd9kuXpUYU

42
tripzilch 2 days ago 0 replies      
So, does this mean we should change the collective noun to "a murder-investigation of crows"?
43
tomdan 3 days ago 0 replies      
I see you liked the article I linked on reddit yesterday :D
44
mozey 3 days ago 0 replies      
Gives new meaning to the word "scarecrow"
45
peshkira 3 days ago 0 replies      
...for the watch
46
trevorg75 3 days ago 0 replies      
Murders investigating murders.
47
obj-g 3 days ago 0 replies      
That pun at the end.
48
lossolo 3 days ago 2 replies      
It should have 2015 in name.
49
mathieuu 3 days ago 0 replies      
In some regions, they also burn the body to make sure they don't come back from the dead.
6
Create React Apps with No Configuration facebook.github.io
711 points by vjeux  1 day ago   222 comments top 52
1
thereact 1 day ago 2 replies      
This is great since it provides an OFFICIAL opinionated set of tools for building React apps which is typically the largest barrier of entry for new developers looking to experiment with this technology.

However, it is missing a lot of core features that typically come standard with Webpack/React boilerplates. Directly from their Github:

Some features are currently not supported:

 Server rendering. Testing. Some experimental syntax extensions (e.g. decorators). CSS Modules. LESS or Sass. Hot reloading of components.
So a great first set of features for a simple React starter project, but for those of you looking to expand the development toolkit from this currently limited configuration, check out the following link to search React boiler projects on github based on a number of criteria like the ability to search by features included such as CSS Modules, Hot Module Replacement, etc.

http://andrewhfarmer.com/starter-project/

For those looking to learn more about the ecosystem, the following resource lists might be useful.

More React resources: https://github.com/enaqx/awesome-react

React/Redux resource links: https://github.com/markerikson/react-redux-links

2
orf 1 day ago 8 replies      
> Having just attended EmberCamp a week ago, I was excited about Ember CLI. Ember users have a great getting started experience thanks to a curated set of tools united under a single command-line interface.

This is one of the best things about Ember. `ember new`, `ember serve`, ember generate component my-component`, `ember build`, `ember deploy`, `ember install`. It's opinionated but it lets you get productive right off the bat. I tried React but after a couple of days I just couldn't get it working, waaay to many options. So I switched to Ember and haven't looked back.

3
seangrogg 1 day ago 2 replies      
I think one of the best things I've done to date is actually distance myself from the React community. While I love some of the tooling that has come out of it (Redux, React-Router) I think the community (as an amorphous entity) over-emphasizes the need/desire for transpiling, linting, testing, etc.

Since then, I've "reverted" to building things in ES5, working in multiple files without bundling, etc. and I have to say the enjoyment I get out of using React has cranked up considerably.

I am happy to see they are converging on some standards - that will definitely make building new apps much easier from a common starting point. I just hope they can walk the fine line between "opinionated" and "bloated".

4
jfdk 1 day ago 1 reply      
This is actually pretty huge. #1 complaint/barrier/hate with getting started with React is all the tooling to do it "the right way"

Kudos to React team for bringing a superior pattern and making it actually practical to use.

5
andrewstuart 1 day ago 0 replies      
This is the most exciting thing to come out of the ReactJS project since it started. The very best investment a technology platform can make is onboarding new developers. ReactJS is recognising that and it's great news that they are making the hardest bit easier.

I whined a while back on exactly this topic.

"Babel 6 - useless by default - a lesson in how NOT to design software. "

http://fourlightyears.blogspot.com/2016/03/babel-6-useless-b...

The last line of the above griping blog post says: "The right amount of configuration is none."

So it is awesome to see someone who DOES know how to design software.

Dan Abramov's blog post says: "Zero Configuration. It is worth repeating: there are no configuration files or complicated folder structures. "

Babel gets it precisely wrong, this new ReactJS tools aims to correct the Babel complexity error.

6
firasd 1 day ago 0 replies      
This is great. React has this weird dual nature in that on one hand, you can drop it in as a <script> tag and it 'just works'. On the other hand, if you want to build using it, you're going to end up needing things like webpack, babel, etc, not to mention other common libraries, to the extent that it takes over your front-end stack (also because it takes over any rendered document nodes, and if you want to build a SPA you eventually use it to render everything inside <body> if not the whole document). So it's good to resolve this conflict by providing quicker ways to get started with common tools.
7
msoad 1 day ago 3 replies      
This is lacking tons of features that other boilerplates already have but I think this was a great move because we needed a source of truth for doing app structure in React.

In a different note, I think if you write it yourself from scratch you'll have more control and knowledge down the road when it comes to nasty bugs but I won't blame you for choosing this over spending weeks setting up a React app.

8
amavisca 1 day ago 1 reply      
Under the hood this is Webpack + Babel + ESLint with sane initial configuration.Love it.
9
vlunkr 1 day ago 1 reply      
My team has wasted so much time configuring webpack. This is a big win for React IMO.
10
vicapow 1 day ago 0 replies      
I'm sucker for self descriptive "boring" names like "create-react-app"

Also, sweet project!!

11
stoikerty 1 day ago 0 replies      
I'm fairly close to finishing the conversion of my `dev-toolkit` into an npm-module. It is almost no-config, has scss, server-side-rendering, hot-reload and more. I'm a one man band but will get there. It's all on https://github.com/stoikerty/universal-dev-toolkit

The npm-version sits in a feature branch, just look for the corresponding PR if you're keen.

12
tlrobinson 1 day ago 1 reply      
I think the best part of this is the "eject" feature. It's great to be able to spin something up quickly but migrate to a custom solution if you outgrow it.

However, it would be nice to be able to tweak some of the configurations (Babel, ESLint, Webpack) without completely "ejecting".

13
venuzr 1 day ago 1 reply      
As someone new to React, I wonder

a) How is this different from getting a custom starter kit/generator from Yeoman. Searching in yeoman, I see several for "React" with the top one having over 9.5k stars http://yeoman.io/generators/

b) Is Facebook planning to maintain and keep this generator current? Why don't they just contribute/recommend an existing generator

14
hoodoof 1 day ago 0 replies      
This is a great idea and sorely needed. Too many frameworks rigidly avoid integration with the ecosystem because they do not want to be seen to bless any given third party technology.

In the case of reactjs however it is extremely important because the ecosystem is absolutely necessary and absolutely damn complicated.

This is precisely what needs to be done to help people get started. Well done.

15
thegayngler 1 day ago 0 replies      
So I was looking through the modules in react-scripts module and I noticed postCSS, HMR is activated (I tested this myself). I also installed react-router with ease and it appears to work just fine. So far so good from where I'm sitting. I was able to start coding right away while I was half assed looking through the modules to see what is actually in there and just play while I talked with my roommates. It took me all of 10 minutes.
16
kcorbitt 1 day ago 3 replies      
Sane defaults and pieces made to go together are critical to lowering the adoption barrier and building a community, so huge props for that. But no ability to configure anything at all? I think that for most people, at some point there will be some small change to the default configuration their environment will require[1], and that means they'll need to jettison the entire project. It's nice that this is easy to do, but it would be better if it weren't necessary.

[1]: For example, I run my app from within a Vagrant Virtualbox machine that doesn't forward filesystem notifications correctly, so I have to configure Webpack's hot reloader to poll for changes instead of listening for fs events.

17
mohsinr 1 day ago 1 reply      
Loving it! I always wanted to get started with ReactJs but looks like time has come! I tried the module and I am loving the "Welcome to React" page on my localhost! Thank you!!!

PS. Already in /src/App.js , and wow live reloading without gulp or browersync , it is so simple to get started! Thank you!

18
griffinmichl 1 day ago 0 replies      
After spending hours yesterday teaching a colleague about webpack, babel, configuration, etc, this is exactly what the React community needs. Finally some fucking sanity in the ecosystem.
19
fdim 1 day ago 0 replies      
Finally something that may convince me to switch from https://github.com/thardy/generator-ngbp - all I want is to focus on writing components not figuring out how to link gazillion dependencies
20
dustinfarris 11 hours ago 0 replies      
Incredible turnaround! I remember seeing Dan Abramov's tweet [1] a while back saying React could learn from Ember's CLI. Two weeks later, here it is! Impressive!

[1]: https://twitter.com/dan_abramov/status/752863664290553856

21
joemaller1 1 day ago 1 reply      
This is great and I will be moving my React projects in this direction. At very least this project represents a de-facto standard and guidance about how to work with React.

However I do wish the React team would pick between ES6 classes and `React.createClass`. I think I remember the main React tutorial was rewritten in ES6 at one point, but then switched back. I've read arguments both ways, but I suspect they ES6 is still too much of a barrier to entry.

People who aren't up to speed with ES6 will still be shaving a lot of yaks before actually jumping into React.

22
pests 14 hours ago 0 replies      
Do not forget React is not equivalent to an SPA.

Almost all SPAs give the entire body over to React but its also possible to choose a smaller DOM node and add React progressively to any existing website view that would benefit from the React paradigm. In this setup (at least) server-side rendering is no longer needed and thus simplifies setting up the build process.

So its not all or nothing, you can pick and choose where to use React based on your needs and requirements.

23
bruth 17 hours ago 0 replies      
This is a great start. I too got frustrated with the overwhelming complexity of the "boilerplate" or "starter kits" that have all the bells and whistles. Having something very simple that you can exit if necessary is nice. The approach I took was just to document each tool or plugin that I may want to add to my project since it is generally very quick to do so: https://github.com/bruth/modern-javascript-starter
24
Bahamut 1 day ago 1 reply      
This is great! Toolchain pain really sucks, and makes being able to get started on a project harder for many, when all you want to do is get a setup running and start creating app code. Having an opinionated CLI start up a scaffold is great - one can also peel apart this whenever one has to prepare for modifying the build chain for specific purposes (for example using Rollup to optimize bundled code, adding new build steps, etc.).

The only thing I disagree with here is not allowing it to be pluggable - IMO it should be flexible and allow users to tweak the setup as desired. Of course, it should focus on getting the core experience right, but in the long term I absolutely think it would be better to have a pluggable CLI.

25
matthoiland 1 day ago 1 reply      
> Some features, such as testing, are currently missing. This is an intentional limitation, and we recognize it might not work for everybody.

With Ember CLI you get a great testing setup with Qunit. While I prefer Mocha over Qunit, I'm at least glad that testing is a first class citizen in the CLI.

26
ola 1 day ago 1 reply      
I created something similar a month ago

https://github.com/olahol/reactpack

Doesn't seem like this project differs that much, although this looks to have the backing of core React developers.

27
brooklyndude 22 hours ago 1 reply      
Is it me, but is not Angular just taking over? Kind of Google vs Facebook thing. Just think Google won this one.
28
robertfw 1 day ago 0 replies      
I've solved my webpack config woes by using HJS-Webpack[0] which describes itself as "Helpers/presets for setting up webpack with hotloading react and ES6(2015) using Babel."

It provides you with a base configuration object, which has been setup with any loaders that it has detected in your node_modules. You can then extend and customize as needed.

[0] https://github.com/HenrikJoreteg/hjs-webpack

29
silasb 1 day ago 0 replies      
Very very awesome. This is very much needed. I work with a lot of older Java people and showing them the ins/outs of webpack/eslint/React is killing productivity. Thanks FB.
30
thegayngler 1 day ago 0 replies      
This was one of the downsides to us using React at work. I did a presentation on Webpack and React and my manager who is the VPE said having to figure out and choose tooling was a concern for him. I responded we should know what the tooling is doing and introduce pieces into our stack rather than go all the way in. This allows us more freedom on how and when to upgrade or change different tools in our front end stack.
31
dack 1 day ago 1 reply      
This is really great! However, I think this speaks to the need for a better API in general for this sort of stuff.

At the moment it's "all or nothing" in that you can decide to let everything be configured, or nothing be configured ("ejecting"). This makes perfect sense, but I think a more ideal solution would be having layers of configurability that let you more gracefully set your preferences without completely abandoning this tool's utility.

I'm not saying that's easy, but it's a direction I'd be excited to see.

32
codenamekt 1 day ago 0 replies      
This is great. One of the biggest hurdles is getting started which is why there are so many react boilerplates. It would be awesome to see projects like this grow so that it would auto configure based on libraries you would like to use. Want to use Redux? Just run the `create-react-app -m redux hello-world ` and you would get everything with the addition of redux and it's configuration.
33
mfrye0 1 day ago 0 replies      
This is awesome. Learning the whole modern build ecosystem was such a headache. It's great it see best practices rolled up for new users to experience.
34
ralusek 21 hours ago 0 replies      
For CLI generation utilities, I haven't found anything that comes close to this guy:

http://fountainjs.io/

36
platonichvn 1 day ago 1 reply      
Definitely a great way to lower the barrier to entry. The eject feature is sweet since it removes the risk of lock in. Looking forward to integrated unit testing libraries in a future release. While you're at it let's add redux. :)
37
JoeCortopassi 1 day ago 6 replies      
This is great for someone who wants to get started to learn React, but is missing a ton that is needed for a real world production app.

 * No isomorphic rendering * No hot module replacement * No generators * No dockerization * No Sass support * No test environment setup * No code splitting
It would be cool to have a production ready tool from Facebook, but I'll stick with gluestick for now https://github.com/TrueCar/gluestick/blob/develop/README.md#...

38
lucaspottersky 1 day ago 2 replies      
Expectation:- "Hey, look, this can SOLVE ALL THE PROBLEMS"

Reality:- "Hey, look, this actually BRINGS IN A WHOLE LOT OF OTHER PROBLEMS too!"

:sad:

39
marknadal 1 day ago 0 replies      
The day we now see "compiled successfully" in the Command Line as the necessary "easy" starting point for frontend web devs.
40
arianvanp 1 day ago 0 replies      
This is really neat. especially the fact that I can 'eject' at any time when I need more power. Love it!
41
deepsun 1 day ago 1 reply      
Is there something like this for React Native? I'm interested in recommended directory structure.
42
uptownhr 1 day ago 1 reply      
wanted to share http://github.com/uptownhr/kube. I also wanted to tackle this problem but also handling SSR as well.
43
kjhughes 1 day ago 1 reply      
Does this help with React Native too?
44
crudbug 1 day ago 2 replies      
Having a consistent API with ember-cli will make this more useful.

$ react-cli <>

45
smrtinsert 1 day ago 0 replies      
This is not a react problem, this is a nodejs problem.
46
wrong_variable 1 day ago 2 replies      
Just wanted to know, am I the only person who is unhappy with react ?
47
rhinoceraptor 1 day ago 3 replies      
What the heck is that terminal font?
48
hex13 1 day ago 1 reply      
it seems like a solution to the Vjeux's challenge: http://blog.vjeux.com/2015/javascript/challenge-best-javascr...

(If we don't count sharable requirement).

49
smcgraw 1 day ago 0 replies      
Amazing.
50
Exuma 1 day ago 0 replies      
Looks cool
51
mcs_ 1 day ago 0 replies      
Thanks !!!
52
mderazon 1 day ago 1 reply      
In the spirit of zero configuration, it would be nice if it included Standard JShttps://github.com/feross/standard
7
I got arrested in Kazakhstan and represented myself in court medium.com
767 points by drpp  2 days ago   230 comments top 26
1
grizzles 2 days ago 12 replies      
This dude just grassed on all the people who bent the rules to be nice to him. With pictures. For a moment of blog fame. Very uncool.
2
steven777400 2 days ago 3 replies      
Great read. I've never been in a situation where a bribe might have helped and so it's good to get some idea of "how bribes work" for possible future reference.

I have noticed that it seems valuable to project being "poorer" rather than "richer" when traveling. Naively it might seem like throwing money around would grease all sorts of wheels but the opposite has been my experience (although probably I just don't have enough money to throw around to really grease the wheels).

3
exabrial 1 day ago 6 replies      
I don't know even what to make of this... The preamble suggests that Kazakhstan justice was superior to American justice, but after reading the whole story I'm like... dude, grow a sense of self-responsibility. It's not like your visa expiring was a surprise... you knew exactly when it was going to happen, and YOU CHOSE to violate it without giving yourself any wiggle room. Then you willingly participated in a corrupt system, rewarding those who profit from it.

It would have been so much easier and involve less questionable ethics to just leave more than 24 hours in advance.

4
nathan_f77 2 days ago 2 replies      
Ooh, I have a relevant story that I've never shared before.

I was living in Kazakhstan for a while and had to take the overnight train to Kyrgyzstan for a visa run. They're very old trains. I woke up around 3am and had to use the restroom. The train had just stopped at a station in the middle of nowhere, and it was the middle of winter. I went to the restroom. I flushed. But when I looked down through the toilet, I saw snow and train tracks. These trains didn't have anything to collect waste, they just flushed straight onto the ground. I immediately heard some loud whistles and shouting, and footsteps.

I rushed back to the bed and hoped that no-one saw me, but it was too late, and then we spent the next 30 minutes talking to soldiers. I wasn't sure if they were asking for extra money, or if it was something we needed to pay anyway because we were crossing the border.

It was a tiny train station in the middle of nowhere, and we had no SIM cards, so I started thinking about what we would do if they kicked us off the train. I was actually kind of excited about the idea of building an igloo and sleeping there overnight, and then going to get some help in the morning. That would have been a better story, but in the end they just let us go.

So don't flush any ex-soviet train toilets when you're stopped at a station in Central Asia.

5
vbezhenar 2 days ago 2 replies      
I'm surprised, that a western man is so ready to participate in corruption. I live in Kazakhstan and I would be very afraid to bribe an official, it could become worse very fast and if you are caught with that, you'll have to bribe much more people or end up in a jail with a very serious offence. I definitely don't recommend to bribe an official in Kazakhstan, usually it's better and safer to follow a law.
6
dakics 2 days ago 2 replies      
Great story!

5 years ago we went on a Mongol Rally. Driving European vehicle through the Stans makes you easy prey for local policeman. My co-driver had a strategy of befriending them and sharing small gifts (pens, lighters etc.). Most expensive were Tajik GBAO guards, they got headlamps. During my shifts I had 2 encounters but played dumb, even though I'm native in similar language and could communicate. No bribes given. :)

Had a great time in Semey, KZ and later across the border in Barnaul. Must go back some day, driving, of course:). If you love big skies of US west, you'll be in heaven in Kazakhstan.

7
jlg23 2 days ago 4 replies      
Great read, but I don't buy it. I've not been to Kazakhstan but traveled Africa and South America extensively. I've been to war zones. A lot of things don't add up:

* A cop takes a bribe, is surprised the tourist is in some computer and then returns the bribe? I've never ever encountered a cop who takes bribes but does not know how the system works. They are not this stupid. This, by the way, is the best way to avoid bribing: Point out you accept punishment and let them work out the consequences for them - paperwork, getting you to jail etc. All this for a visa that expired a few hours ago? I'm sure they'll find a less work-intensive way to let you go.

* Cops being happy to have some "criminal" around for getting drunk and they even pay? No, they rather take your money and get drunk with their friends.

* The girl's story did not make much sense (abortion, breaking up, being raped, leaving school, being arrested, being dug on by a male guard while making out with a female guard and all of this within 24h? wait, what, I am missing some connections here).

I'm not saying that the base of the story ain't true, but there is, IMHO, a lot of storytelling in there, too.

8
mind_heist 2 days ago 4 replies      
Is that picture going to get Irlan in trouble ?
9
swimnow 2 days ago 4 replies      
Something similar happened to me. I was a citizen of Uzbekistan at the time. I was about to fly out of Almaty. When the lady at the checkpoint saw my passport, she asked where my exit visa was. I never knew all the Uzbek citizens needed one to leave the post Soviet territory. So, I was denied my seat on the plane. Had to renew my ticket for $50. Come next day, another lady at the checkpoint says the same thing - without the exit visa I am not going anywhere. The plane was already boarding and I was about to miss it again. She saw me getting agitated and says "why don't you talk to this man here?". He said everything could be arranged for mere $300. Even though I was an actual student and poor as a church mouse, I had to pay it. I was let out.
10
gheeohm 1 day ago 0 replies      
I'm amazed by the amount of negative comments.

In my opinion, this is a great, well written traveling story, where no objective, willful harm was caused by the author. What I got from it was that a book should not be judged by its cover, it's important to try and relate to people when traveling, and violence against women is a horrible problem.

If the names weren't changed, you could say he was a bit naive (although the odds of this ever reaching Irlan seem somewhat small to me), but ultimately, he portrayed all of the people he met in a positive light that makes me want to visit Kazakhstan.

I think being in such a situation, out of one's comfort zone, gives great perspective on what really matters (although it's not for everyone).

11
mildbow 2 days ago 0 replies      
Heartwarming story.

What I've experienced is, all over the world, people will try to help: if you seem like you need help and don't have all the answers/money.

For travelers -- if you know the local language, speak it. I had a scary experience in the same part of the word and managed to "ingenious" my way out of it. Speakign the local language just helps people identify with you more.

treat them like a person and they'll treat you like you are a person.

12
HNaTTY 2 days ago 2 replies      
Reading this and then going back and reading his SF arrest story (linked in the article) is a real contrast.
13
atmosx 1 day ago 0 replies      
All ex-communist countries were poverty is the status quo have a similar way of dealing with things. We call it bribe, but most of us are on the bright side of the planet, financially speaking at least.

If Irlan didn't wanted to be bribed there would no discussion. All the chit-chat was in order to induce the victim to bribe him.

I heard much more salty stories from my father who was a Businessmen in the Balkan area in the 90s and 00s. After 2002 the situation in the Balkan area improved a lot, but it's not uncommon for police officers to get bribed, it's their way to make ends meet and the easier way for a foreigner to get things done.

14
joelhaasnoot 2 days ago 4 replies      
Cute story, but wonder if the outcome would be the same if the author hadn't originally been Russian and didn't speak Russian.
15
jgust 2 days ago 2 replies      
Can anyone tell me what this means?

> ...in another ten years, you will see, well be living like Arabs.

16
polytap 1 day ago 0 replies      
This story reflects most poorly on the author himself.
17
fsckin 2 days ago 1 reply      
This reminds me of Bert Kreischers' story about his travels in Russia as part of a language immersion trip.

https://www.youtube.com/watch?v=8PAtFsJY5q0

18
kharms 2 days ago 1 reply      
"I could have called the US embassy, but from past experience I knew their help was often useless, and occasionally even harmful."

This was an interesting throw away. I wonder what his experience was.

19
ommunist 1 day ago 1 reply      
for god sake, the author has to at very least change names of the guys who sincerely helped him to avoid serious troubles.
20
staticfish 2 days ago 0 replies      
what a wonderful story. really puts arguing about typesafe frameworks in perspective..

A+++ would read again.

21
Twirrim 1 day ago 0 replies      
"He who represents himself has a fool for a client" - Abraham Lincoln.
22
agentgt 1 day ago 0 replies      
The author is very gifted writer. I was hoping for a quick skim but I could not stop reading.
23
lunchTime42 1 day ago 1 reply      
We are programmers and architects - do not bend the rules, we made them, your bending suggests flaws and imperfection and is offending.

Trains shall run on tracks. Spontanious agents are a loss of controll. The horror.

24
johansch 2 days ago 3 replies      
a) Sensationalist headline - something that seems amazing to someone who is not e.g. russian - but the person is actually russian

b) Outing the photo of a cute 18 yo girl "who just got raped"

Yes - an interesting story - but this is extreme clickbait likely for profit. And problematic in other dimensions. This guy is an opportunist who does not seem to care about other people.

25
fiatjaf 2 days ago 0 replies      
Is this real?
26
cloudjacker 2 days ago 0 replies      
and the award for biggest snitch of the year goes to OP......

can you like take this down?

8
Apple says Pokmon Go is the most downloaded app in its first week ever techcrunch.com
408 points by doppp  1 day ago   204 comments top 16
1
jandrese 1 day ago 15 replies      
Shows you just how much pent up demand there was for Nintendo to release games on mobile.

Getting a huge first week download count is a lot easier when you have literally decades of brand recognition. Being a free download certainly didn't hurt either.

It remains to be seen what the customer retention numbers look like. I saw some absolutely insane projections earlier this week about how Apple and Nintendo were going to make billions off of Pokemon Go. I don't see how they're going to sustain the current game as it gets fairly grindy and there isn't much to do once you've caught them all. Maybe some compelling new features will be added to keep players from getting bored? Direct peer to peer battles and possibly trading for example.

2
MattyRad 1 day ago 3 replies      
When someone told me that Pokemon Go was exploding, I looked into it, and got really excited about its concept. People getting outside, interacting though a long-loved game, using real landmarks to denote checkpoints, playing a localized "king of the hill" type minigame. The architecture behind it and it really feels like it's using bleeding edge VR push us into a more social and fun world.

That said, I also feel like it's equally the biggest missed opportunity to date. Usually, I just see players walking, heads down, not talking. It was downright eerie when I was downtown one Tuesday night at midnight, and it was dead quiet despite ~60 Pokemon players meandering about. They should have introduced PvP earlier (hopefully it's around the corner!), and better yet, make it so you get more exp for battling people you haven't battled before. Spur people into social interaction!

3
chipperyman573 1 day ago 6 replies      
I'm confused by the title. Is PoGo the first app to reach x downloads in the first week of release, or is it the most downloaded app of all time, just one week after release? Slow internet won't let me view the article.
4
curiousgal 1 day ago 9 replies      
This game has become a victim of its own success. Niantic has be strangely silent about bugs and server outages. I foresee a massive drop in interest soon.
5
hogwash 1 day ago 0 replies      
Funny retrospective on the last 25 years of AR:

http://unanimous.ai/augmented-reality-then-and-now/

6
meerita 13 hours ago 0 replies      
A game can't have everything the first day of launch. They released this to test it against the market. Now that it is a success, changes will come to increase retention and purchases. Now the next biggest events will be promoted also with Pokemon things to do in the place, like "get this rare pokemon on the Vegas Electronic Event. The game is a real success and the mechanics described by Richard Bartle proved that people loves to collect, it is the Diogenes syndrome but in mobile version.
7
kin 1 day ago 2 replies      
The numbers will absolutely drop. I mean, there's definitely a ton of content that can be added like earning gym badges, Gen 2-6 Pokemon (which people don't really care about), trading, PvP, etc. But, at the end of the day I doubt Niantic has the time/resources for that. The execution has been rather poor.

Still though there's a demand for Nintendo software on mobile. They just need to really to execute. They're really lucky we're tolerating these huge bugs (nearby Pokemon and frozen Pokeball after catch still outstanding).

8
Osiris 1 day ago 0 replies      
75% of the time I launch the game, I'm confronted with an error that I couldn't be logged in.

Maybe the game is so popular because it feels like a rare resource. It's so hard to get into the game that when you do you have to play it as long as possible until the servers go down again.

9
xlayn 3 hours ago 0 replies      
As time pass there are more iDevices.... so the number can be irrelevant.So maybe a percentage as download/devices?
10
smaili 1 day ago 4 replies      
Would love to know the app who previously held the record.
11
kevindong 1 day ago 0 replies      
The dropoff in interest has already started. I pretty much stopped playing last week. I got to level 14 and the amount of grinding required was just ridiculous (the amount of XP you earn per action does not increase as you level up meanwhile the XP required to level up goes up exponentially). The bugginess of the game really did not help.

The dropoff in interest has already, objectively speaking, started[0]. It's currently (as of July 22) 66% of its peak (per Google Trends). In my personal experience, interest on my college campus has already subsided. It's not completely dead, mind you, but the hype is over.

[0]: https://www.google.com/trends/explore#q=pokemon%20go&date=to...

12
blhack 1 day ago 2 replies      
I think that the biggest feature that Pokemon Go will add, that will hopefully come soon, is the ability to broadcast your position.

This is something that I wish ingress had done. The game is a multiplayer game, there is no doubt about that. I'd love to be able to open map map, see that some of my friends are over playing at $foo location, and then go meet them there.

13
Bonsailinse 7 hours ago 0 replies      
These numbers are probably the reason why niantic was totally caught by surprise and have these massive server issues.I really don't like to see the app offline every time I have to go for a longer walk, but hey, I take it easy and wait for either niantiv upgrading their ressources or the userbase to shrink, which shouldn't take too long imo.
14
TheMagicHorsey 1 day ago 4 replies      
Have I missed something, or is this game just about walking around collecting pokemons with eggs? Is there anything else to it that I missed? The interface isn't illuminating.
15
mp3geek 1 day ago 0 replies      
How does the numbers stack up between the downloads on Google play vs Apple?
16
melling 1 day ago 1 reply      
So, do we have an entirely new class of games/apps that are about to appear? Like Pokmon go but for ...
9
Introducing Stack Overflow Documentation Beta stackoverflow.com
476 points by sklivvz1971  2 days ago   125 comments top 34
1
pietroalbini 2 days ago 9 replies      
What I'm most worried about is the duplication between the canonical documentation of a project and the StackOverflow one.

As the author of an open-source project, I try my best to write a great documentation, and I would be a bit annoyed if people started to add examples to StackOverflow just to gain reputation there instead of contributing to the "official" one.

Also, SO is ranked way higher than the smaller-projects' documentation on search engines, pointing developers there. This can be problematic, for example, if a big release comes out and the SO documentation is behind.

The documentation for a lot of projects is really bad, I know, but I prefer a solution which doesn't disrupt the work of the mainteiners which writes good and extensive documentation.

2
akavel 2 days ago 2 replies      
In the footnote of the post they wrote they considered naming it "SO Examples", but didn't. For me at least, using "SO Docs" to name the site is actually much more confusing. As a result, for the whole post I thought it is a dynamic manual, with per-function docs and examples. Only after browsing to the actual site I slowly realized it's not a manual: it's rather a "book" of examples. A list of HOWTOs, speaking in a language of yesteryear. As far as I see, it's impossible to build a MSDN in it. I couldn't add full documentation for one of Go packages here borders cross vague, overlapping concepts, not clearly cut packages. To make it clear: I don't want to deny that it can be useful suppose as examples for learning; but when looking for docs, I like being able to browse them systematically.

So, two possibilities: either something revolutionary I didn't fully grasp yet or Examples, not Documentation.

For now, I much prefer the model of Go docs, MSDN, or PHP manual with user comments if talking about docs.

3
kyriakos 2 days ago 4 replies      
Great idea I especially appreciate the fact that its done by example. A couple of things that bothered me are:

1. The UI needs some refinement. I was looking to find a topic to post about and from my 10 minute browse I realised that if I was meant to find information in this documentation it's really hard to find what you are looking for. After you drill down to a tag it feels "unstructured". Readthedocs layout feels more user-friendly.

2. To future proof the documentation examples should come with a version number they apply to. For example there is PayPal and DropBox API examples which in a few months might no longer be valid.

4
troymc 2 days ago 8 replies      
This is awful, another trick site that fools people into doing work that they could be getting paid to do, all for the joy of getting some "karma." Well kids, karma ain't gonna pay the rent. If you want to get experience volunteering to write documentation for software, then find the existing official documentation and add to that (or start it, preferably in a repository close to the actual code).

Any profits that get made off this "documentation" (i.e. incoherent bag of examples) will not flow to the community or company behind the software in question. It's a parasite, leeching off the success of other projects which it neither created nor cares about.

How did they get corporate partners for their launch? That's easy. It looks to those companies like free labor.

5
greggman 1 day ago 1 reply      
I'm actually very conflicted about Stack Overflow in general and of course the new documentation section as well.

For whatever reason I answer many (most?) of the WebGL questions. At some point I felt I needed to make longer form answers with better working examples so I made http://webglfundamentals.org

But, now there's this huge conflict. I spent a couple hundred hours on making webfundamentals.org but when someone asks a question on stack overflow I'm not supposed to just link to whatever I wrote. I instead I'm supposed to effectively transfer all my content to SO. Something about that just feels wrong. SO is making money from the content I created which feels a little weird (yes I know I get other people's content back). Also, while I get that SO's gamification is part of what has made it so successful it's also feels like it turns many things into a competition. I try to tell myself don't worry about those points I'd be lying if I said they didn't affect me at all in various ways.

Taking all that and adding to documentation, as an example, when I contribute to MDN I feel like I'm doing something purely positive. But if/when I contribute to SO Docs I already know I'm not going to feel the same. One reason is because SO Docs will be making money from my work. The only thing I get in exchange is some "score" on my name I can maybe use to get a job. Maybe that's a fair trade since I don't get a score on MDN?

I'm not sure how to make my point. I love that I find answers on SO but something just doesn't feel right and I don't know how to express it.

6
rudedogg 2 days ago 2 replies      
Stack Overflow's point system is really annoying. I've been programming for around 10 years, but because I don't participate much I can't even comment on an answer.

A month or so ago I was stuck on a problem and thought I'd go through a tag for something I'm familiar with, and submit some answers to try and get enough points to be able to fully use the site.

One answer had a new programmer following a tutorial and using an old method signature. I commented that the tutorial he was following is out of date, and listed the correct method to use.

The person downvoted my answer, and then pasted (basically) the same code as the answer to his own question and accepted it. I know I just had bad luck in this case, but it's pretty frustrating.

Not allowing a user with low points to do some functions makes sense, but let them submit content and allow other users to determine if it's useful. I could have (and wanted to) comment on dozens of answers, which would of helped out a lot of people and saved them time/frustration.

7
JamesBaxter 2 days ago 1 reply      
If this gains traction I don't think we'll feel the benefits till a little later down the line.

I've contributed a little to the Swift tag but the real advantages come from technologies that don't already have good documentation.

I'm considering starting a tag for the enreco HTML -> PDF generator I use quite a lot as the official documentation isn't great.

8
IshKebab 2 days ago 1 reply      
This is awesome. I can't count the number of times I've found an Android function that has almost no documentation. I usually post a question and answer on SO if I work out how it works.

I only wish it were organised a bit more like normal documentation - i.e. into classes and methods. Might make it easier to find things.

9
michaeldwan 2 days ago 1 reply      
I like the concept (reminds me of gobyexample.com) but this really isn't documentation. Instead of more code snippets I want high quality annotations (like snippets!) atop official documentation. If more people were driven to canonical docs but with a guiding hand I think we'd see more people actually understanding the code they write.
10
jackcarter 2 days ago 1 reply      
I hope they release a private enterprise version, like they have for StackOverflow[0]. I'd love to host this on-prem to document my company's codebase.

Anyone know of a similar solution that's available now?

[0] http://meta.stackexchange.com/questions/16054/is-the-stack-e...

11
metakermit 2 days ago 2 replies      
Cool new project I think it might become very helpful one day. Documentation for a lot of open source projects is pretty bad and having to figure out a new contributing workflow every time just to hop in and help a bit is quite problematic. Hopefully, the unified interface SO users are used to will give docs writing a big boost.

That said, I still hope they output some sort of GitHub repo of all the accepted changes to make it a bit less walled-gardeny. A CC license is nice and all, but having the content in a repo as well would put my mind at ease.

12
makecheck 2 days ago 0 replies      
I feel like this might be best used to document unexpected behavior or stupid things that maintainers may be too proud to acknowledge in their official manuals.

It should not be used to add redundancy. And it should not be used to accumulate cruft that doesnt belong in any manual or list of examples. People have already gone for low-hanging fruit; for instance, do we really need examples of how to initialize a list?

13
newman314 2 days ago 1 reply      
I understand this is (for now) for devs. I would love to see an equivalent (including private offering) for infra.

Getting people to document things on the infra side is deplorable in general and I would love to try anything that helps improve that.

14
reitanqild 2 days ago 3 replies      
Really curious to see if this succeeds, fails like discourse or gets overrun by what I consider to be a major destructive faction of nitpicks and deletionists on SO.
15
krat0sprakhar 2 days ago 1 reply      
If anyone is wondering how this looks and is impatient to read the blog post, here's an example Documentation page for Java Streams - http://stackoverflow.com/documentation/java/88/streams#t=201...
16
jstoiko 2 days ago 0 replies      
This is cool.

It would be nice to have an API for this so that my IDE can pull these examples.

17
bouh 1 day ago 0 replies      
I really hope that all open source documentation quality will increased by adopting such kind of tools. The fact that SO is used an informal documentation was a strong signal that there were much improvement to be done. I am just sad because I wanted to work on project like that :'(. Great job SO !
18
RyanHamilton 2 days ago 0 replies      
Awesome! I've actually been working on a java IDE that allows uploading examples together with the results to a website:http://jpad.io/example/1s/generating-random-int-array-within...

The idea is to build up libraries of examples in different areas, allow easy code sharing and to remove some of the cruft needed. It's good to see stackoverflow hit this "code examples" area better.

19
vorg 1 day ago 0 replies      
Didn't see the word "test" in the article (except once talking about beta testing S.O.'s new product). And the word "test" isn't in these comments either, so...

Will updated examples be run through a program testing them in some way? If not a full run within a playground-style environment, then through a type checker?

And will the examples be tagged with version numbers? Some programming languages are notorious for changing syntax and/or semantics between minor versions.

20
d2ncal 1 day ago 0 replies      
While this sounds like a good idea and will probably do well, it also centralizes more critical information and access to one for-profit company and product.

I really wish that we as a community would spend more effort towards better, decentralized systems.

21
simonswords82 2 days ago 1 reply      
So this makes Stack Overflow a library of example code curated by people who already frequent their website. Am I reading that right?
22
rohanpai 2 days ago 0 replies      
This is super cool!

I am interning at Sourcegraph and we have heard devs want better usage examples too, so we automatically show all uses of any function

check out: -https://sourcegraph.com/github.com/golang/go/-/info/GoPackag...- https://sourcegraph.com/github.com/golang/go/-/info/GoPackag...

23
tongcx 1 day ago 1 reply      
It would be nice if there is SO for business. Lots of companies have terrible internal documentation, Q/A tools.
24
RyanHamilton 2 days ago 0 replies      
Awesome! I've actually been working on a java IDE that allows uploading examples together with the results to a website: http://jpad.io/example/1s/generating-random-int-array-within....

The idea is to build up libraries of examples in different areas, allow easy code sharing and to remove some of the cruft needed. It's good to see stackoverflow hit this "code examples" area better.

25
grkvlt 2 days ago 1 reply      
[META] This is popular, but is there a way to notify the mods of duplicates so that conversation doesn't get fragmented? I flagged https://news.ycombinator.com/item?id=12135897 and https://news.ycombinator.com/item?id=12136086 but I don't know if that's the right thing to have done?
26
jcastro 2 days ago 0 replies      
Anyone know if there are plans to roll this out to other stackexchanges? Would love to have this on askubuntu.com.
27
krick 2 days ago 1 reply      
I really don't want people to start using something like this instead of improving the official documentation. I hate the fact it probably cannot be helped.

To "do the same thing for docs that we did for Q&A"? Yeah, except there just was no sane platform for Q&A before. And for docs there is, you know the docs!

28
allendoerfer 2 days ago 1 reply      
I think a big problem is user motivation. On Q&A sites you get the reward to directly help someone and have a conversation. This is missing for documentation. I wonder if their gamification is enough to motivate people or whether they have to find a deeper human desire that they build into this and satisfy.
29
jbrooksuk 2 days ago 2 replies      
Odd.

Couldn't find blog.stackoverflow.comThe Q&A site blog.stackoverflow.com doesn't seem to existyet.

You can vote for it to be created through our democratic, community-driven process at area51.stackexchange.com, or see a complete directory of all our Q&A sites at stackexchange.com.

30
iamrohitbanga 2 days ago 0 replies      
Can we use existing SO answers to curate documentation?
31
lucaspottersky 2 days ago 0 replies      
there's gotta be an integrated way to copy&paste examples (i.e. via autocompletion/snippet) from SO to our favorite code editor/IDE :D
32
ofcapl_ 1 day ago 0 replies      
looking forward for some standarized integration with github's wikis
33
anotherevan 2 days ago 0 replies      
Will be interesting to see what the moderation guidelines will be like, and how zealously they are "interpreted".
34
speps 2 days ago 0 replies      
Will there be an example of a bad regular expression that could take down a website?
10
Wire open-sourced github.com
423 points by arunc  21 hours ago   100 comments top 24
1
gfosco 21 hours ago 2 replies      
A link to a GitHub organization isn't great.. I'd say this is better: https://medium.com/@wireapp/you-can-now-build-your-own-wire-... but even that doesn't clearly explain what Wire is. Visit https://wire.com to find out it's an encrypted video and group chat app.
2
grizzles 20 hours ago 1 reply      
A bold move by Wire. Open source is still a very disruptive play, and the world needs something like this. If they manage this well and triple down on developer engagement, it could work out quite nicely for them. EDIT: Thread title is slightly misleading. It looks like they did a Telegram. There is no server here.
3
deltaprotocol 19 hours ago 6 replies      
I must say that my first impression is beyond positive.

One to one and group chats, group video and audio calls, GIF search built-in, doodles, the best implementation of photos in the message stream that I've seen, poking and playable Spotify and Soundcloud music by just sharing links? All with end-to-end encryption?

I have that "too good to be true" feeling but, still impressed. Just waiting for possible audits and more feedback from the security community.

Edit: It's also Switzerland based, already supports Win10, MacOS, Web, Android and iOS, and to complete has the cleanest design I've seen in a messaging app.

4
laksjd 15 hours ago 0 replies      
They offer a password reset function. How does that work? Do they hold my private key in escrow? I'd certainly hope not! Or does the password reset work by creating a new keypair? If so, does this at least generate WhatsApp style security warnings for people chatting with me?

With some digging I've found a way to verify key fingerprints so that's nice, but it's manual, not QR assisted :(

5
melle 20 hours ago 1 reply      
I believe all their good intentions and I do hope they succeed. But for me it's too early to tell whether their business model will hold. If they build up a sufficiently large user base, but fail to monetize it and sell the company to e.g. Microsoft or Facebook, then I doubt how much of their original privacy / openness remains.

Another thing that I wonder about: Does being Swiss-based give them a privacy advantage?

6
saghul 18 hours ago 0 replies      
Lots of good stuff in there, thanks Wire! I just wish they had gone with something other than GPLv3 for libraries, like LGPL. Looks like they changed them on December, from MPL 2.0 to GPLv3.

At any rate, there are lots of us who can use the code with that license :-)

7
jacek 14 hours ago 0 replies      
I am a user. I switched myself and my family from Skype a few months ago and it has been great so far. Quality of video and audio is great, Android app works very well (better than web based desktop versions). And it also works in a browser, which is great for me (Linux user).
8
prayerslayer 7 hours ago 0 replies      
Not sure if these are for realsies, but there are some API keys in the webapp repository:

https://github.com/wireapp/wire-webapp/blob/master/app/scrip...

https://github.com/wireapp/wire-webapp/blob/master/app/scrip...

9
nanch 21 hours ago 1 reply      
See https://wire.com for more information since the linked repos provide no context. "Crystal clear voice, video and group chats. No advertising. Your data, always encrypted."
10
redthrow 9 hours ago 1 reply      
Why does this Android app require a phone number to sign up?

At least Hangouts lets me use the app without a phone number.

11
happyslobro 14 hours ago 2 replies      
I found a file that is available as either MIT or GPL. Or is it only available under a union of the terms of both licenses? An intersection? Who knows, IANAL. https://github.com/wireapp/wire-webapp/blob/0cf9bf4/aws/main...

Why do people copy the license all over the place like that?

12
jalami 20 hours ago 1 reply      
Side note, but it's kind of strange that images on their site require cookies enabled to view. I didn't dig into a reason, I just white-list the sites I want to use cookies and found it odd that there were big white spaces before doing so.
13
_bojan 18 hours ago 1 reply      
Didn't see that coming. I think Wire is struggling to get new users and this move could put them on the map.
14
mahyarm 21 hours ago 3 replies      
Now all this needs is a few good third party audits, verifiable builds and it's the holy grail of encrypted communications!
15
20andup 21 hours ago 1 reply      
I wonder what the business model is?
16
pedalpete 20 hours ago 3 replies      
I don't get how they can make statements like this "Only Wire offers fully encrypted calls, video and group chats available on all your devices". Webrtc is encrypted by default.
17
stemuk 21 hours ago 3 replies      
I wonder how they encrypted their chat on the web client. Scince the Signal protocol is kind of the gold standard right now, probably their solution might in the end be the better one.
18
sanjeetsuhag 16 hours ago 2 replies      
Can anyone explain to me why they use an UpsideDownTableViewController ?
19
aleken 20 hours ago 1 reply      
Otto is my new best friend. I cannot see any information about a bot API on their site though...
20
maxpert 12 hours ago 0 replies      
Good to see people using Rust in production :)
21
mtgx 9 hours ago 1 reply      
I've been asking for three things from Signal for the past almost two years:

1) desktop app

2) video call support

3) self-deleting messages

Signal finally (sort of) delivered a desktop app, but it still doesn't have the other two. Wire has the first two, but it's still lacking the last one. I hope one of them will have all three of these features soon.

22
mei0Iesh 20 hours ago 0 replies      
Thank you! Wire is the best, with multiple device support, clean mobile app, and a desktop client. It'd be nice if it were a standard open protocol so everyone could implement it, and find a way to allow federation. I'd pay to help support.
23
arthurk 15 hours ago 2 replies      
Is there a way to download the OSX app without the Mac App Store?
24
vasili111 14 hours ago 1 reply      
Where is Windows client source code?
11
Google tags Wikileaks as a dangerous site google.com
338 points by xname2  1 day ago   140 comments top 23
1
nl 1 day ago 10 replies      
The biggest downside to the NSA revelations is how quickly people accept conspiracy theories.

Wikileaks just released a big email dump. People looked at it with Google Chrome, and it detected malware in the archive. That blacklisted the site it was downloaded from.

There is no big "Google is protecting the Democrats and hates Wikileaks". Wikileaks was serving malware, and Google detected it.

2
user837728 1 day ago 2 replies      
This is technically accurate since I found out myself this week when browsing the AKP email leak. Some of the attachments in the emails were clearly malware of some sort. See for example: https://wikileaks.org/akp-emails/emailid/27482
3
AWildDHHAppears 14 hours ago 1 reply      
I don't think there's anything to see here. Google now tags it as "safe." The mechanism worked; the website administrators removed the malware, and the warning was removed.

See! Everything works in a rational way. There's no conspiracy.

4
Sylos 1 day ago 1 reply      
I figure this link needs to stand here somewhere, even if it's just for someone trying to understand the political implications that this could have: https://wikileaks.org/google-is-not-what-it-seems/
5
dpweb 1 day ago 0 replies      
More interesting is the debate here in the comments where people are unsure if it's legal for them to read something on the Internet. I doubt Google is censoring Wikileaks. Too obvious. But startling is the chilling effect nowadays.
6
tszming 1 day ago 0 replies      
[youtube.com] https://www.google.com/transparencyreport/safebrowsing/diagn...

# Some pages on this website install malware on visitors' computers...

# Some pages on this website redirect visitors to dangerous websites that install malware on visitors..

7
astronautjones 1 day ago 1 reply      
it could be political, but it's probably because they're hosting all of the attachments from all of the e-mails that were leaked - including spam
8
brudgers 1 day ago 2 replies      
Shows me "not dangerous" at UTC 00:43 22.07.2016.
9
rbolla 1 day ago 1 reply      
its not a dangerous site anymore...

as of 7:15 PM PST.

https://www.google.com/transparencyreport/safebrowsing/diagn...

10
cesarbs 1 day ago 0 replies      
If you refresh the page multiple times, it switches between "Not dangerous" and "Dangerous downloads".
11
dljsjr 1 day ago 7 replies      
I'm not sure that this is completely tin-foil hat worthy.

I was working at a defense contractor in 2010 when the big leak of all the cables occurred, and was forced to learn a lot of things I wouldn't have otherwise, including something that maybe a lot of people don't fully grasp:

When stuff like this leaks, if any of the information is considered sensitive/classified/restricted in any manner, the act of it being leaked does not dissolve its restricted status. In other words, if you are a regular US citizen and you go to Wikileaks and look at something that is classified without having the proper security clearance, then you're now on the wrong side of the law.

I don't think there's any political shadiness going on here, I think Google is just trying to be on the correct side of the system. Whether or not that system is on the right side of some moral or ethical line is a different conversation entirely.

12
fixermark 1 day ago 0 replies      
"Current Status: Not dangerous."

Did this change in the intervening (clock-check) 4 hours, or is there some definition of dangerous I'm missing?

13
smoyer 14 hours ago 0 replies      
As of approximately 1000 EDT (US) on 07/23/2016, the link above gives the status of wikileaks.org as "Not Dangerous".
14
retox 1 day ago 0 replies      
Andrew Simpson was possibly the first to report. Comes very soon after DNC email leak.https://twitter.com/Andrewmd5/status/756529847762087936
16
seoguru 1 day ago 0 replies      
I am not getting the warning on my browserchrome 52.0.2743.82 beta
17
mjwilliams 18 hours ago 0 replies      
It says "not dangerous"
18
prashant10 9 hours ago 0 replies      
actually it doesnt anymore...
19
throw2016 13 hours ago 0 replies      
This really doesn't matter. The kind of people who are concerned about the information revealed by wikileaks, Snowden, Manning and the burgeoning surveillance infrastructure have little reason to trust what Google says or does.

What seems off is the default kneejerk response especially in places like HN where one would assume a far more informed audience - working in the industry which is spearheading this - to brush things under the carpet or make discredited, desperate and increasingly irrational references to conpiracy theorists.

There have always been conspiracy theorists and always will be, but the current narrative on surveillance has moved so well beyond that point that to have this discussion tarred by these tired and banal references to conspiracy theorists is completely disingenious and makes those making these arguments look out of touch.

If you know what has been revealed so far it should not be difficult to engage with some degree of seriousness at the issues at hand without immediately resorting to strawmen.

20
MooBah 15 hours ago 0 replies      
Welp, GJ - Google Changed it back.
21
MooBah 15 hours ago 0 replies      
yelp - looks google changed it back. GJ Thread!
22
cLeEOGPw 19 hours ago 2 replies      
HN is a liberal safe space. Of course any article that reflects badly on liberal ideas will be removed. that political-story "gravity" is the same thing as reddit's "modified algorithm" to hide /r/The_Donald. There is an enormous push of a certain agenda from all major corporations in US, neither HN, nor reddit or google is an exception.
23
colordrops 19 hours ago 2 replies      
This crosses into personal attack and that is not allowed here. Please don't do it again.

We detached this subthread from https://news.ycombinator.com/item?id=12148835 and marked it off-topic.

12
Alleged founder of worlds largest BitTorrent distribution site arrested arstechnica.com
378 points by fcambus  3 days ago   341 comments top 49
1
akavel 3 days ago 4 replies      
ArsTechnica seems to have the most detailed and best linked (PDF + DoJ press release) article as of now:

http://arstechnica.com/tech-policy/2016/07/kickasstorrents-a...

Somewhat more/complementary details seem to be available on: https://torrentfreak.com/feds-seize-kickasstorrents-domains-... e.g. regarding methods:

[...] The complaint further reveals that the feds posed as an advertiser, which revealed a bank account associated with the site.

It also shows that Apple handed over personal details of Vaulin after the investigator cross-referenced an IP-address used for an iTunes transaction with an IP-address that was used to login to KATs Facebook account. [...]

Some aspects which seem interesting to me, from what is reported:

that apparently KAT owner tried to shield off DMCA takedown requests (which I'd see as trying to affirm being legal);

that according to the articles he seems to not have used Tor (or fumbled in it).

(Assuming no parallel construction and that he's actually the guy, etc. etc.)

EDIT: I couldn't really find any Polish sources suppose because it's middle of the night here... (the single article http://www.dobreprogramy.pl/Zalozyciel-Kickass-Torrents-zatr... seems to be written based on the above English-language ones)

2
CaptSpify 3 days ago 8 replies      
As an American, I wish we would stop doing this. It isn't effective, and it's a waste of time/resources that could be better spent elsewhere.

I'd even argue that its counter-effective to progress. Instead of punishing people for making more efficient systems, we should reward them, and try to integrate.

3
derefr 3 days ago 9 replies      
It seems like a lot of the most well-known pseudo-legal BitTorrent "groups" (PopcornTime, YIFY, ISOHunt, now Kat) turn out to be one-man shops, and as such, just completely dissolve as soon as their owner crosses paths with law enforcement. In some cases, these services are integral enough to the "scene" to be brought back by others. But other times, everything just stops for a while.

This seems like a bus-factor problem. Why does it keep happening? Why aren't these sites being run by multi-national teams that can survive a loss like this?

Even The Pirate Bay is "just" Swedish, so a sufficiently-motivated Swedish Government could shut TPB down. Meanwhile, there's no single country that could shut down e.g. Wikipedia.

4
eggy 2 days ago 9 replies      
Aside from the legal technicalities here, I mostly ponder the future of IP. I think Napster positively affected the music distribution world in the long run. I am not very black-and-white on this issue, however, since there are many contradictions by both sides.

I read the majority of comments here on HN about dated business models, big corporation dislike, the old executives don't understand the new market, etc..., but then a young indie artist in LA finds out Zara the clothing retailer has obviously copied her designs, and the lynch mobs are out to boycott Zara, send letters and other things to Zara and their attorneys. [1]

I have not inquired directly, but I am guessing a number of the indie artist's supporters have downloaded a torrent or two. How do they morally distinguish the two, or how does anybody who is against copyright or property rights of IP?

[1] https://www.buzzfeed.com/victoriasanusi/an-independent-artis...

5
kayman 3 days ago 4 replies      
The cat and mouse game continues.

Remember The Pirate Bay?

Why don't studios have their own similar sites where theyallow free torrents of some shows and offer paid torrents.

As a busy person, I'd much rather pay for something which guarantees:

- high quality- no subtitles- no buffering issues- no viruses- click and play

6
josho 3 days ago 4 replies      
I find it interesting that a Polish man was charged by US laws, rather than under Polish law.

I think he opened himself to US law by hosting the servers at one point in the US. Regardless, it is rather fascinating that his first visit to the US could potentially be from extradition.

7
spodek 3 days ago 0 replies      
> Assistant Attorney General Caldwell said that KickassTorrents helped to distribute over $1 billion in pirated files.

So, two or three files, by Hollywood accounting.

8
jerrac 3 days ago 3 replies      
Does anyone know if there is data somewhere on how much money has been spent by governments (specifically the USA) on attacking copyright related stuff?

My main complaint about this is that I'd rather my tax dollars be spent stopping crime that causes physical harm.

9
blackflame7000 3 days ago 3 replies      
If you think about it, torrent sites are like a modern day robinhood. They take profits from the rich and bring enjoyment to the poor.
10
ceejayoz 3 days ago 1 reply      
> It also shows that Apple handed over personal details of Vaulin after the investigator cross-referenced an IP-address used for an iTunes transaction with an IP-address that was used to login to KATs Facebook account.

I find it darkly ironic that a legal purchase of music helped them catch the guy.

11
dmix 3 days ago 1 reply      
Another black market business opportunity brought to you via fed money. DOJ, FBI, polish police, etc, etc all spent tax money on this takedown. All working so the next guy can make a website and make $16 million / year. And it only takes one guy to run the site apparently.

Who knows maybe the next guy will use Tor, Bitcoin, read Grugq's blog and be 5x as expensive to hunt down. Thanks US tax payers!

12
downandout 3 days ago 2 replies      
This will be interesting to watch. Torrent sites only host torrent files; I'm sure he'll argue that the DMCA requests were invalid because the people filing them didn't own copyright to the torrent files, which were the only thing that the site distributed. Where do we draw the line? Do we prosecute people for posting a magnet link? If a movie studio puts an MP4 of a not-yet-released film on its servers, is it illegal to link to it?

It will be an interesting case to watch if he takes it all the way to trial. I don't think it's nearly as open-and-shut as the DOJ would like everyone to believe every case it files is though.

13
megous 2 days ago 0 replies      
So if extradited he may be tried. Now has jury of your peers (fellow citizens) any meaning in case of trying a non-US citizen? Jurors must be US citizens, but they will not be his peers, really.
14
steve19 3 days ago 0 replies      
" unlawfully distributing well over $1 billion of copyrighted materials.

bullshit of course. He simply hosted hashes of torrents that other people uploaded.

As far as I know he even acknowledged dcma takedowns.

15
LeoPanthera 3 days ago 3 replies      
Well we all switched from TPB to Kickass... which one do we switch to next?
16
sergiotapia 2 days ago 2 replies      
So that's why KAT's been down all day.

Two things: If you're going to build a torrent indexer, don't profit from it. Keep it alive yourself, with NO ads, just plain HTML and JS and images.

Second: This is why I vastly prefer usenet.

17
neurocroc 3 days ago 0 replies      
This is really unfortunate and sad news.
18
ben_jones 3 days ago 1 reply      
If I was making millions of dollars from illicit activities I would practice incredibly rigorous opsec. I realize modern lifestyles don't align with any kind of anonymity but come on. Here's a short list:

- No Apple

- No Facebook

- No Google

- Don't live in a 5 eyes or affiliated country

- Know the extradition and legal precedent in all countries visited

19
simbalion 3 days ago 0 replies      
The folks who operate sites like KAT and TPB are heroes.
20
codecamper 3 days ago 1 reply      
When will the feds learn that this is whack a mole.. and every time the mole improves. (I am 1428x times smarter...)
21
cm3 3 days ago 1 reply      
Is there a legal lesson to take from this when it comes to using cloud services hosted in the US where you can be affected by US laws just because you hosted a site with links to PDFs from Elsevier in an S3 bucket which itself happens to be pointing to a US datacenter? I haven't read it all, but there must be more to it than having used a US hoster that made the guy an easy litigation target.
22
DyslexicAtheist 2 days ago 1 reply      
A gentle reminder that the US can hunt you down and punish you under US law even you have never stepped foot in the country:

>> "According to a Department of Justice press release sent to Ars, Vaulin was arrested on Wednesday in Poland. The DOJ will shortly seek his extradition to the United States."

23
Kenji 3 days ago 1 reply      
Wait, you can get extradited to the US for hosting (magnet) links? Madness.
24
cocotino 3 days ago 3 replies      
Wow, this sucks, I love KAT, what now? TPB is rubbish.
25
novaleaf 3 days ago 2 replies      
Very interesting. It seems he got busted by making the mistake of once-upon-a-time hosting in the US and Canada, thus providing them grounds for persecution.

So reading in that context, they would have been free and clear if it were not for that mistake!

27
Grollicus 2 days ago 0 replies      
~15 Years ago I went to to Poland on a student exchange. They held a presentation about their school and stuff and in the middle of the presentation theres a popup from emule saying it has finished downloading something. Mind you this was a school computer on their school network.

I suspect this has changed, but back then they were pretty laid back about that sort of thing.

28
msie 3 days ago 6 replies      
So Bittorrent is decentralized but you need a centralized index of the available torrents? Is this correct? Can you find anything without a site like KAT? TIA.
29
LeonM 2 days ago 0 replies      
> Vaulin is charged with running todays most visited illegal file-sharing website, responsible for unlawfully distributing well over $1 billion of copyrighted materials, Assistant Attorney General Caldwell said in the statement.

I can't keep myself from giggling and thinking about this:https://www.youtube.com/watch?v=GZadCj8O1-0

30
ungzd 2 days ago 0 replies      
Today such databases of cultural works metadata (movie descriptions, music album track lists) are illegal because you can use some of metadata (checksum of files) as identifier to associate it with actual content in p2p network.

What if we modify bittorrent dht or similar thing so it'll use some other identifier: wikidata id, oclc, instead of "checksum of checksums of files"? Next day Wikipedia and library catalogs become illegal?

31
marty69 2 days ago 0 replies      
Someone has taken a dump of db ? I can't access to api to download dump. I try to connect on mirrors like http://kickasstorrents.video/ or https://kat.host/
32
ForFreedom 2 days ago 0 replies      
In the light of KAT and PirateBay being down, which is the next most worthwhile torrent website.
33
belorn 2 days ago 1 reply      
> KAT does not host individual infringing files but rather provides links to .torrent and .magnet files

What is a .magnet file? My understanding is that .magent link is the key hash in a distributed hash store (DHT).

34
gggggg11111 2 days ago 1 reply      
After reading the whole PDF document of the complaint one thing jumped out

Here we have Apple, Google, Facebook, Coinbase, FDC Servers and few others handing over info on email accounts, wallets, hosting records and so on.

35
bekirbek 2 days ago 0 replies      
kat was actually the best torrent website at the moment, and I'd say by far. This is a very inefficient way for the US government to spend resources and money, they won't be getting anywhere.
36
prirun 1 day ago 0 replies      
I think it's ridiculous that Homeland Security is involved in a copyright infringement case.
37
androtheos 3 days ago 0 replies      
Time and resources well spent, glad we have our priorities in order. Maybe Hollywood actually runs the country and not the banks and corporations, ahh, ahh, I mean elected officials of the government.
38
avree 3 days ago 2 replies      
Can't read this site because of their adblocker detection.
39
liquidise 3 days ago 0 replies      
Perhaps this is pedantic, but i wonder if magnet links provide any legal securities when compared to actually hosting the torrent files themselves.
40
sergiotapia 2 days ago 1 reply      
Now that's KickAssTorrents is dead, what alternatives are there?
41
bickfordb 3 days ago 0 replies      
I'm surprised he didn't use a virtual currency to sell ads.
42
ommunist 2 days ago 0 replies      
Oh shit! And why Google is still running than?
43
jbverschoor 2 days ago 0 replies      
So he pays for software and goes to jail :-)
44
roozbeh18 3 days ago 1 reply      
how are newsgroups safe all these years? how comes feds aren't going after newsgroups?
45
fallo 2 days ago 0 replies      
R.I.P.
46
Rfgakmall 2 days ago 0 replies      
01001110011000000101
47
Rfgakmall 2 days ago 0 replies      
010000000111000101011
48
stop1234 2 days ago 0 replies      
I just had a crazy brain fart, what if bitTorrent and bitcoin had a child?
49
youngButEager 3 days ago 2 replies      
"Do you have a legal right to distribute this content?"

"No."

"Do you respect the people who spent their personal time and their money to make the content you distribute?"

"Yes of course."

"No you don't or you would get their permission first."

"I don't think whether I respect the creators or not has anything to do with making their content available, which is what I want to do."

"Will you please get their permission to distribute what they spent their time and money -- part of their life -- to create?"

"Nope, sorry. I'm distributing it for free. They can take a hike."

Real sad, these modern morals. Real sad. If you don't respect others, they ain't gonna respect you.

13
A Beginner's Guide to Understanding Convolutional Neural Networks adeshpande3.github.io
359 points by kilimchoi  2 days ago   29 comments top 9
1
Dzugaru 2 days ago 3 replies      
Have yet to see an illustration that grasps multichannel convolution filters (MCCF) concept clearly. Why those channel stack size keep growing? How are they actually connected?

The thing that each conv filter consists of kernels in multiple channels (that's why first layer filter visualisations are colored btw - color image is a "3-dimensional" image) - and we convolve each kernel with corresponding input channel, then sum (that's the key) the responses. Then having multiple MCCF (usually more at each layer) yields a new multi-channel image (say, 16 channels) and we apply new set of (say, 32) 16-channeled MCCFs to it (which we cannot visualise by themselves anymore, we need a 16-dimensional image for each filter) yielding 32-channel image. That sort of thing is almost never explained properly.

2
chrisruk 2 days ago 0 replies      
http://arxiv.org/abs/1602.04105# -- This paper is awesome for a use of CNNs, for automatic modulation recognition of RF signals.

I'm attempting to use their approach with GNU Radio currently -

https://radioml.com/blog/2016/07/18/towards-a-gnu-radio-cnn-...

3
danielmorozoff 2 days ago 0 replies      
Great writeup from Stanford CS231 course:http://cs231n.github.io/convolutional-networks/
4
sjnair96 2 days ago 0 replies      
Damn the author is a freshman!
5
thallukrish 1 day ago 0 replies      
A human child learns much more easily by seeing only a handful of images of a cat and then almost being able to say any type of cat image as it grows (without ever seeing 1 million or billion images). So, there seem to be something that shows that more than the amount of data, the "reality" of seeing a real cat probably includes all possible aspects of a Cat ? There seem to be something missing with this whole deep learning stuff and the way it is trying to simulate the human cognition.
6
vonnik 2 days ago 0 replies      
Here's an intro to ConvNets in Java: http://deeplearning4j.org/convolutionalnets.html

Karpathy's stuff is also great: https://cs231n.github.io/

7
crncosta 2 days ago 0 replies      
Very well illustrated.
8
chronic81 2 days ago 2 replies      
it is very introductory, just as it supposed to be for beginners.

I doubt he tries to be a though leader, rather this post looks like a notes that he made while learning about CNN and published them since they might be useful as a quick-start to someone else.

9
cynicaldevil 2 days ago 2 replies      
I am new to CNNs/machine learning, but here's my $0.02:Regardless of which technique you use, it seems that the amount of data required to learn is too high. This article talks about neural networks accessing billions of photographs, a number which is nowhere near the number of photos/objects/whatever a human sees in a lifetime. Which leads me to the conclusion that we aren't extracting much information from the data. These techniques aren't able to calculate how the same object might look under different lighting conditions, different viewing angles, positions, sizes, and so on. Instead, companies just use millions of images to 'encode' the variations into their networks.

Imo there should be a push towards adapting CNNs to calculate/predict how the object might look under different conditions, which might lead to other improvements. This could also be extended to areas other than image recognition.

14
Edward Snowden's New Research Aims to Keep Smartphones from Betraying Owners theintercept.com
292 points by secfirstmd  2 days ago   146 comments top 17
1
smartbit 1 day ago 2 replies      
The poor mans way of stopping your iDevice from transmitting, is by putting it in DFU mode [0]. This regretfully will prevent you from using it for anything else too, unlike airplane mode. And some will probably argue that a nation state could mimic DFU on an active phone, but it is a viable option that anyone afraid of being under surveillance could chose. The timing of DFU mode can be quite difficult, this video [1] has been help to millions.

Alternatively some use an iPod with only Signal installed. As stubborn Moxie requires access to the address book [2], the iPod address book is exclusively used for Signal addressees.

[0] https://www.theiphonewiki.com/wiki/DFU_Mode#Entering_DFU_Mod...

[1] https://youtu.be/bITIiGswjF

[2] https://whispersystems.org/blog/contact-discovery/

2
jakobdabo 2 days ago 6 replies      
When I place my smartphone on the desk near the computer speakers any time it is going to ring the speakers start making a funny noise a second or two before the ringing starts. So I presume it must be possible to DIY a cheap sensor for GSM signal detection based on a little speaker.
3
pigeons 1 day ago 0 replies      
The Neo900 is designed to detect unauthorized radio transmission from the modem and power the modem down in a fraction of a second, and notify you. It seems to be the only device that will have that capability.

https://neo900.org/

http://neo900.org/stuff/cccamp15/ccc2015talk/neo900-wpwrak_C...

4
dewster 2 days ago 5 replies      
Probably just showing my ignorance, but there is a processor running in the phone, and it is connected to the various chips on the board, and you can run your own apps that could query the chips directly? If the OS disallows this, I'd be hacking the OS, rather than the hardware.

How did we get to this point, where our personal computing devices are completely out of our basic control? We live in bizarro world.

5
semi-extrinsic 2 days ago 7 replies      
No disrespect to Snowden and Bunnie, but it seems to me that a much simpler solution giving you a much higher OPSEC is to buy a smartphone with a removable battery. No battery, no radios are on.

And if you are truly paranoid, it's simple to disassemble the phone and look for/remove any backup batteries. I know, I had to pull the backup battery from my wife's Moto G after it fell in the sink.

6
ISL 2 days ago 2 replies      
Why go through test points rather than directly detecting RF emission?

In addition to the required hardware modification, a sufficiently nefarious attacker might be able to spoof test points. RF power detection, on the other hand, can't lie. If it's going to communicate, the phone must transmit.

An RF-detection tool would be as easy as a phone case (and could double as a backup battery for the phone). It'd be far simpler and easier to adopt than directly hacking on the hardware.

Edit: My concerns are partially addressed in the actual paper: https://www.pubpub.org/pub/direct-radio-introspection

7
jmiserez 1 day ago 0 replies      
If you can't trust your phone, how would you ensure that it doesn't just record everything (audio, etc.) when in airplane mode and uploads it somewhere later, once you disable airplane mode.

Seems to me that removing the battery would be safer.

8
sangnoir 12 hours ago 0 replies      
How hard would it be to make the following after-market modifications?

1. add a physical "off" switch that cuts battery power to everything

2. (Hard Mode) Cut power to all radio chips/subsystems (GSM, WIFI, bluetooth) while leaving the rest smartphone operational for taking pictures or recording audio?

9
walrus01 2 days ago 1 reply      
The problem that got Colvin killed is at the RF/layer 1 layer in the OSI stack... Iridium and Inmarsat phones operate in the L and S bands (1.2 to 2.0 GHz) which is not difficult to do radio frequency direction finding on, if the Tx source remains active. Particularly easy if you have access to Russian military grade DF equipment. The protocol layers and crypto are moot if you are radiating and have a determined DF adversary.
10
phones 2 days ago 0 replies      
Of interest perhaps, here is a full source code of an Android phone software and its baseband firmware:

https://github.com/mtker/MT6735_Longcheer

Actually there is some .o files in the baseband but easy to pull apart in IDA. Each one relates to a single .c and there are export symbols.

11
DigitalJack 2 days ago 1 reply      
This does seem feasible for the specific use case of a protected phone for "clandestine" meetings.

My initial thought was they'd have to redesign it for every phone, but that's not necessarily the case. If eaves dropping is such a concern for you, I would think you would be okay with not having the latest gen phone. Or having an old one just for these sorts cases.

I suppose the concern then shifts to whether this device is easily subverted, or whether it's easy to determine if it has been subverted.

12
cowardlydragon 1 day ago 0 replies      
Almost like you need a faraday cage for the phone, with an internal antenna, a "router" through the faraday cage that you have hardware/software control, and then an antenna to rebroadcast outside the cage.

Basically, a radio firewall. So you can enforce absolute radio silence if needed. And log the signals.

13
rosser 2 days ago 1 reply      
How does this address masking "bad" transmissions behind "good" ones? Instead, the spooks will just make sure not to upload your chat logs until you start Tindering the next time, or something.
14
zanny 2 days ago 0 replies      
You know, if we had source access and hardware blueprints to these devices and actually owned them, this wouldn't be a problem.

But trying to solve an obvious problem (proprietary basebands, phones, and hardware) with bandage solutions kicks the problem down the road. We need to liberate the hardware eventually for liberty's sake.

15
venomsnake 2 days ago 1 reply      
Isn't that device a Faraday cage?

And if you are in war zone - using a phone with removable battery is absolutely mandatory IMO.

16
frockwearer 2 days ago 1 reply      
This same sort of approach has been used by terrorists in the past.
17
calebm 2 days ago 2 replies      
I wonder if the use of the word "betraying" in the title is a subtle jab at Snowden.
15
How I built an app with 500,000 users in 5 days on a $100 server medium.com
417 points by kiyanwang  2 days ago   166 comments top 41
1
joshstrange 2 days ago 5 replies      
This article left a really bad taste in my mouth. I don't believe GoSnaps == GoChat in terms of complexity and the constant back patting and self congratulating is really distracting. There were a couple of decent takeaways but largely the whole post revolved around how "How smart am I?" and "What great foresight I have".

I really don't approve at all of the GoChat shaming going on. The author may be 100% correct that GoChat made mistakes in writing code that doesn't scale well but that doesn't give him a blank check to beat and berate GoChat. It reads as a very discouraging post to newer/less experienced programmers in my opinion, essentially "Don't even bother making something unless you know it can scale to millions of users" which I think is a terrible message to be sending.

2
zongitsrinzler 2 days ago 5 replies      
The real takeaway from this is that the author uses Hackathon Starter (https://github.com/sahat/hackathon-starter).

I have used it for multiple projects and it gives a huge head start compared to starting from zero. Signing up, logging in, resetting the password, uploading, etc all seem like easy work but when you pile them all up you can easily spend a week just getting to the point where you are within minutes of cloning the starter repository.

However the failure of GoChat is not relevant to Pokemon Go. While GoChat might have done something very wrong comparing 1mil users to an app with tens of millions of concurrent users is invalid. Pokemon Go would be a NoGo running on a single Node.js machine without any sort of balancing.

3
mootothemax 2 days ago 1 reply      
Ah, the good old "I could build StackOverflow in a weekend" line of thinking - I'm sure we've all been there.

There's a world of difference in building a photo sharing app with XXX,XXX users vs. building a chat app with XXX,XXX users.

When you do anything that involves chat or that level of concurrency, surprises will bite you in the behind, multiple times, even if you desperately try to use as much existing software as possible.

(as anyone who's taken a look at ejabberd, thought it'll play nicely, and then load tested their code will tell you)

Frankly, PHP vs. Rails vs. Node[1] vs .Net vs Java will be the least of your troubles.

[1] I do fear that the author is going to find a nasty surprise or two for themself regarding Node's performance issues

4
jbardnz 2 days ago 4 replies      
First I would say Pokemon Go has done incredibly well to handle such massive growth so quickly, no doubt they were able to leverage a lot from Ingress but I could imagine many other companies having days of downtime while trying to scale up so quickly.

I also tend to disagree a bit with the article. For every situation like this were early scalability is important their are a 1000 MVP apps that are prematurely optimized or over engineered. At the end of the day the chance of anyone building an app that will get over 100,000+ in a week (and keep those users coming back) is very very very slim.

5
gedrap 2 days ago 2 replies      
>>> Where would I have put my images? In the database: MongoDB. It would require no configuration and almost no code.

Why... would anyone actually do that in anything more than a classroom example for an application like the one described? Amazon S3 and similar services have very decent libraries for pretty much every popular programming language, why would you re-implement that?

>>> MVP and scalability can coexist

I'd replace that with less catchy but probably more correct 'experienced devs can make more scalable mvps with little extra cost, if any'. MVP doesn't mean lets just go silly and make the quickest and dirtiest decision imaginable.

It's a matter of experience to recognize potential problems and the respective potential solutions, and program accordingly. SQL schema is a pretty good example. Often it makes a big difference in scaling and often you can design the initial schema to be much more scalable with some experience and a few moments of planning.

6
0xmohit 2 days ago 3 replies      
> If I would have built GoSnaps with a slower programming language or with a big framework, I would have required more servers. If I would have used something like PHP with Symfony, or Python with Django, or Ruby on Rails, I would have been spending my days on fixing slow parts of the app now, or adding servers. Trust me, Ive done it many times before.

> As said, GoSnaps uses NodeJS as the backend language/platform, which is generally fast and efficient. I use Mongoose as an ORM to make the MongoDB work straightforward as a programmer.

7
agentultra 2 days ago 2 replies      
There's a little too much self-congratulatory prose in here. And poor advice (Use NodeJS because its fast).

But there is one take-away at least... design your application around your data and how your users will interact with it and performance will generally fall out of that. And it doesn't take much to start that way rather than leaving it as an after-thought.

People might break out the (oft-misquoted) "premature-optimization" horse for a little beating, but performance does matter. At least the bounds matter for most applications. You might not need to eek out every cache line but you can set targets up-front to say, "We cannot tolerate more than Xms req-to-res time" and bake that into your design.

8
tie_ 2 days ago 0 replies      
Survivorship bias anyone?

How many times did a project fail, because it the non-aspects (e.g. scalability) were undernegineered? How many times did it fail because it couldn't ship on time/budget due to excessive engineering? We do not normally read such stories, because they are totally unexciting, taken separately. And one failed case of GoChat does not a worthy stat make.

Ultimately, good job to the guy for nailing a sweet spot between his skills and the market of the application created by those skills. Just do not assume that's everybody's sweet spot.

9
RubyPinch 2 days ago 2 replies      
> But this would have been totally disastrous under any type of serious load. Even if I would have simplified the above query to only include three conditions/sorting operations, it would have been disastrous. Why? Because this is not how a database is supposed to be used. A database should query only on one index at a time, which is impossible with these geospatial queries.

> On the database side, I separate the snaps into a few different collections: all snaps, most liked snaps, newest snaps, newest valid snaps and so forth.

Pardon my ignorance, but don't most databases have some method of handling these issues?

(defining multiple indexes for use, having support for geospatial data, having support for like, subsections of the existing dataset, etc?)

I thought that the main goal was to offload the developer's code's logic onto the performant database, as opposed to offloading the database's logic and caching onto the developer's code? is the former not practical?

10
CarolineW 2 days ago 0 replies      
The two previous submissions have a few comments scattered between them - here are direct links to those comments:

https://news.ycombinator.com/item?id=12122681

https://news.ycombinator.com/item?id=12132605

https://news.ycombinator.com/item?id=12132610

Despite getting a few votes, neither of those submissions got any real attention first time round - no doubt pure chance that this one has got enough attention to hit the front page.

11
allendoerfer 2 days ago 0 replies      
I read the story a while ago and was waiting for the criticism in the comments. Now one comment [0] already pointed out many of the issues of the article.

What's been mentioned in other comments but not explained in great detail is the database design, so I want to expand that:

The right way (TM) to do databases is to design a solid schema to keep data integrity and then apply indices and caches depending on your application needs. To be honest his application seems super simple to cache top-down, so a few lines inside the nginx config (which seems to scare him for some reason) would probably do. But if you use a real database (also TM) you can go bottom up, too:

1. solid schema with constraints

2. indices depending on your application

3. stored procedures, database views

4. some non-relational cache like MongoDB to cache denormalized data

5. maybe something in memory

6. (application)

7. nginx caching

He started with 4. What he did is not a solid database design to brag about, instead he hardcoded a cache inside his application. If he wants to scale his application vertically or horizontally he will have big problems, because he misses a point at the beginning which contains the truth on which everything else is build upon. If he starts scaling up and then wants to change his schema he is basically in hell.

What he did is nothing bad. It is exactly "the MVP way". MVP is not about slow or buggy software but a really small feature set and applying YAGNI. MVP is nothing bad, he seems to have great sucess with it! What I am criticising is not how he build his software but what he wrote about it, comparing it to a much harder case and thinking it has something to do with good design.

[0]: https://news.ycombinator.com/item?id=12135748

12
roddux 2 days ago 1 reply      
>If I would have used something [..] Python with Django [..] I would have been spending my days on fixing slow parts of the app

>GoSnaps uses NodeJS as the backend language/platform

Is NodeJS really that much faster than Python in practise-- even with a fast framework (Falcon, pycnic, hug.rest) and Pypy? I know a lot of work has been put into making V8 fast but I didn't realise it was notably faster than Python.

13
iamleppert 2 days ago 1 reply      
You could have built most of the data side entirely static. First convert the user coordinates to simple mercator XY. Just divide or round that down to some precision and put the resources in a namespaced S3 bucket/path. Then just do a directory listing on resources in that bucket. You could even name them the full precision xy coord so you could still sort by distance, within the bucket.

Let S3 be your database.

You dont need the full precision of a geospatial query or database if youre building a simple app that organizes content by location. Depending on your density you segment few 100 meters or few 1000 meters.

14
arviewer 2 days ago 0 replies      
This reminds me of a quote from Biz Stone, Twitter founder: It takes ten years to become an overnight success.

https://twitter.com/liftapp/status/472080510546903040

15
maxencecornet 2 days ago 1 reply      
>GoSnaps grew to 60k users its first day, 160k users on its second day and 500k unique users after 5 days (which is now)

How did you market the app ?

16
Veratyr 2 days ago 0 replies      
Surprised he didn't talk about dedicated/colocated servers. For $100/month he could have had a E3-1231v3, 32GB of RAM, 2x480GB SSDs and unmetered gigabit bandwidth from OVH.

Instead he paid $100 for 4 hyperthreads, 15GB of RAM, a few GB of storage and fast but horrendously expensive bandwidth (assuming he used the n1-standard-4, which matches his description).

If he'd set it up to scale the number of servers with load or something it'd make sense but this doesn't make any at all.

17
zuck9 2 days ago 2 replies      
Reading this leaves me doubtful whether to use MongoDB or not again:

https://www.google.com/search?q=dont+use+mongodb

https://www.reddit.com/r/programming/comments/3dvzsl/why_you...

Do people at big startups use MongoDB in production?

18
kimshibal 2 days ago 2 replies      
Our company migrated to elixir 2 months ago. We have 2M users per server at $20/month.
19
adeptus 2 days ago 4 replies      
Pff that's nothing. I could build a fake app, in less than 1 hour, for $100 and get about 5 million downloads in 1 day.

Step 1. Find some opensource app code

Step 2. Call it Pokemon Go 2!

Step 3. Upload it to Appstore & link it to dropbox

Step 4. Spend $100 on African "talent" to give fake 5 star reviews & positive comments in app store.

Step 5. Hit F5 repeatedly at Appstore to watch the download counter increase to 5 million in 24 hours.

Step 6. Profit ?!?!

Step 7. Post story in /r/nosleep because too much guilt fooling 5 Million people.

20
ekiara 2 days ago 0 replies      
In both cases the developers have committed to a pretty big monthly payment for an app that serves hundreds of thousands of users.

4000USD is a huge amount and even 100USD monthly is a lot to spend out of pocket without a plan to recover that money. Do they have any plan of making money out of these sites or are they purely CV/portfolio pieces?

21
stonewhite 2 days ago 1 reply      
I just don't get how he goes on and on about uploading images to cloud storage instead of mongodb, which he makes it sound like a very genuine decision.

Is it just me or what he telling is rudimentary?

22
bojo 2 days ago 0 replies      
I'm less interested about the technology and more interested in whether he has a plan for monetizing all those users.
23
tckr 2 days ago 1 reply      
and earned $0.
24
vacri 2 days ago 0 replies      
Aw... it's a $100/mo server, not a $100 server.
25
jackweirdy 2 days ago 0 replies      
The idea of putting into different collections up front is pretty smart. To generalise it into a broader lesson, I guess you could say it makes sense to make a one-time effort up front to save complexity down the line.
26
cocktailpeanuts 2 days ago 0 replies      
It is true that GoChat doesn't need to be that expensive to maintain and his analysis is pretty much correct (I've maintained something that had similar amount of traffic, similar dynamics, and didn't cost me arms and legs at all, far from it. It's amazing how cheap you can start a company nowadays)

But no need for bashing someone else. These things are a fad so this GoSnaps thing will probably go the same way as GoChat anyway.

27
kriro 2 days ago 0 replies      
Solid read, good basic thinking with regards to scalability via basically prefiltering data except for the one query you need to run at runtime.

It's a bit strange that the author mentions Scala as lean/fast with lots of libraries (along with JS and Go) but Java is too bulky. I'd say modern Java 8 can be used in a pretty lean manner. There's also nice and small web frameworks (Spark etc.).

28
nickpsecurity 2 days ago 0 replies      
There's a lot of flak over poor comparison to photos and self-promotion. However, his overall point is still true: just putting a little effort in upfront with assumption you will succeed can prevent these problems. My baseline for evaluating this is "Did they do at least as well as someone who spent 30 seconds on Google?" Short version: doing better wouldn't have required a ton of thinking.

Here's what 30 seconds Googling "highly-scalable chat architecture" gave me:

http://highscalability.com/blog/2014/10/13/how-league-of-leg...

https://engineering.riotgames.com/news/chat-service-architec...

http://doc.akka.io/docs/akka/1.3.1/scala/tutorial-chat-serve...

Note: Like to have seen numbers for field-test of the above in the article. Yet, it would've gotten someone thinking.

http://lsirpeople.epfl.ch/nbonvin/dl.php?f=chat-report.pdf

http://doc.utwente.nl/68325/1/imp-pasm.pdf

Previous times doing this for web services led me to highscalability.com with many architectures to imitate with fairly mature software components available. At this point, the common ones should practically have templates for "enter metrics expected here" then click to deploy.

29
webtechgal 2 days ago 0 replies      
Here is my take on this:

1. MVP vs. scalability: While building scalable product/s right from the MVP stage is generally a good idea, it may not be particularly beneficial or applicable to most scenarios. I mean

a) how many typical startups happen to scale to 500k or 1M users within days from launch?

b) most founders would be needing an MVP mainly for market validation, as a proof-of-concept and for the purpose of attracting seed/startup funding

c) many founders - especially non-coders - may not have the luxury/resources to have scalability built in to the MVP

2. The original story goes to reconfirm my belief, based on multiple past experiences going back many years, that database continues to remain a (huge) bottleneck for web apps with high traffic volumes and max possible database optimization (right from config tune-up to table structure design/normalization to query optimization) can pay huge dividends in most cases.

30
nathan_f77 2 days ago 1 reply      
If anyone has more tips about how to get 500,000 users in 5 days, I'm sure we would all like to hear them.
31
antoineMoPa 2 days ago 0 replies      
For me, this article is very reassuring.

My server used to be at <0.8% cpu usage. Now that I installed mongodb with almost nobody using my app (< 2 person per hour), my cpu is always at ~1.6% (It doubled because of mongodb!). At first, I feared that my cpu use would be enormous as soon as I would get new users. Now I guess my cpu% increase is due to some overhead that will not grow too much with db size/use (if the author was able to make an app of this scale with mongodb). I'll also try the lean() mongoose thing.

32
CameronBanga 2 days ago 0 replies      
I'm all for critiques of software and how to improve work, but do we need to rag on the guy who made GoChat? Looking at the project, it was clear it was a single guy or a couple people, working to put out a project for experience.

It's poor form to self-aggrandize and say "move fast, make MVPs, etc", and then write a post pointing out over and over how people messed up, when they were trying to move fast and make an MVP.

33
projectramo 2 days ago 1 reply      
Everyone is talking about the technical feat, but the real insight for me is that you should hitch your wagon to a rocket ship.

Pokemon is a rocket ship right now, and any new app has this enormous exposure advantage.

It is also important that it scale well or else you'll squander your advantage.

For what it is worth, I was very impressed by the technical stuff. (It is making me laugh to read about how disappointed others are. I feel like I missed something.)

34
joesmo 2 days ago 0 replies      
His conclusion about Doctrine and other ORMs eating CPU and being the huge bottleneck in the app lines up with my experience using the same. The MVC framework itself, Symfony/Rails in his case, can indeed also be a huge bottle neck, though much less than the ORM yet higher than the DB calls themselves. That too has been my experience often.
35
andy_ppp 2 days ago 2 replies      
I'm going to use Cassandra for part of my application - the bit that might conceivably be unperformant and very difficult to cache - even though it'll take a few extra days now to get working over using Postgres I'd rather just do this at the start than have migrate a write heavy and main part of the apps functionality while live.
36
mijoharas 2 days ago 2 replies      
> I personally love Erlang and would never use it for an MVP, so all your arguments are invalid.

Could anyone elaborate on the point the author was trying to make here? is it that erlang doesn't have many pre-existing libraries (for building an MVP) or is not fast enough (or something else)?

37
EGreg 2 days ago 0 replies      
The more interesting question is how did these apps get their users in the first place?
38
ryanbertrand 2 days ago 0 replies      
Great job! One thing I noticed is your app requests my location always (even when the app is not on the foreground). It seems like you would only need my location while I am in the app.

You might get a higher acceptance rate.

39
aato 2 days ago 2 replies      
I'd be curious to know what kind of image recognition software the author used to detect relevant images and if it came with a significant performance hit.
40
gwbas1c 2 days ago 0 replies      
Basically, the author knows that best practices are truisms and require common sense to apply.
41
muneersn 2 days ago 0 replies      
Is there any way to simulate high load (Millions of users) for testing?
16
Git for Windows accidentally creates NTFS alternate data streams latkin.org
347 points by latkin  3 days ago   174 comments top 13
1
smhenderson 3 days ago 14 replies      
The root cause of all this is a relatively obscure NTFS feature called alternate data streams.

Obscure indeed, I've never seen them used for anything other than hiding malicious content. Curious, I read about them on Wikipedia[1] and it turns out they were originally created to support resource forks in Services for Macintosh. Browsers also use them to flag files downloaded from the internet.

[1] https://en.wikipedia.org/wiki/NTFS#Alternate_data_streams_.2...

2
kazinator 3 days ago 3 replies      
The colon has been special since the dawn of DOS. For instance, you cannot use "con:" as a file name. (In fact, in a fit of extreme stupidity, DOS also claimed some devices with no Colon suffix, like "con" and "prn", effectively making these into globally reserved names in any directory.)

Stock Cygwin does something special with the colon character, so the Cygwin git shouldn't have this problem. A path like "C:foo.txt" is not understood by stock Cygwin as a relative reference in the current directory of drive C; the colon is mapped to some other character and then this is just a regular one-component pathname.

In the Cygnal project (Cygwin Native Appplication Library), paths passed to library are considered native. So that certain useful virtual filesystem areas remain available, I remapped Cygwin's "/dev" and "/proc" to "dev:/" and "proc:/", taking advantage of the special status of the colon to take this liberty. You can list these directories (opendir, readdir, ...) and of course open the entries inside them; but chdir is not allowed into these locations. (Unlike under stock Cygwin, where you can chdir to /dev). chdir is not allowed because then that would render the library's current working directory out of sync with the Win32 process current working directory, which would not be "native" behavior.

3
Someone 3 days ago 1 reply      
It's not alone. In MS SQL Server, you can name a database "foo:bar". If you give a database such a name when you restore it from disk, you'll find that the database takes zero bytes on disk (at least, that's what Explorer claims) Your disk space is gone, though.
4
duncans 3 days ago 1 reply      
Related to this bug: used to be a vulnerability in IIS back in the late 90s where you could append ::$DATA to a file name (e.g Foo.asp::$DATA) and download a server-side script's source code.
5
Grue3 3 days ago 1 reply      
I had a related problem with Dropbox. Some files uploaded from my Linux machine were not synced to my Windows machine. Later I narrowed down this problem to images being saved from Twitter, which have URLs ending with ":orig". On Linux, Firefox happily saves such images as "blahblah:orig.jpg", whereas on Windows it uses space instead of a colon. And of course Dropbox on Windows would completely ignore filenames that contain colons and tell that the directories are synced, when they obviously aren't.
6
artifaxx 3 days ago 10 replies      
That is quite the obscure and interesting issue to run into! Who puts colons in their filenames though? I haven't ever seen that used...
7
mcculley 3 days ago 4 replies      
This is interesting. I was just recently working on an app where I wanted to ensure the UI wouldn't accept problematic characters in filenames. Obviously, Unix has problems with '/'. I'll add ':' to the list. That's unfortunate. What else should portable apps avoid?
8
AWildDHHAppears 3 days ago 2 replies      
MacOs (i.e., Os9 and before) had special meaning for colons, too. I wonder what would happen for git on those platforms.

Edit: Apparently colon is _still_ a special character on Mac! http://stackoverflow.com/questions/13298434/colon-appears-as...

9
jorangreef 2 days ago 0 replies      
The flip-side of this:

I was running a fuzz test on a backup tool, which verified that file data and metadata (including timestamps) as reflected by Windows were exactly as produced by the fuzz test.

I noticed that for some ".eml" files this was not the case. The mtime of these files was being modified by something else after the initial create by the application. At last, it came down to a Windows process which was automatically indexing ".eml" files and creating an ADS for each of them, thereby touching the mtime.

This was intentional on the part of Windows, but I never saw it coming.

10
xg15 3 days ago 5 replies      
The problem should be addressed, but the proposed workaround seems strange. So git should refuse to write the file to disk? How am I supposed to use a git repo that contains such problematic files on Windows then?
11
sickbeard 3 days ago 3 replies      
putting colons in your filenames are almost as weird as alternate data streams.
12
fowl2 3 days ago 2 replies      
"McAfee Web Gateway" thinks this is porn, great.
13
ragsagar 2 days ago 0 replies      
Wonder why this site is blocked in UAE! :|
17
Reddit is still in turmoil techcrunch.com
332 points by minimaxir  2 days ago   482 comments top 34
1
twblalock 2 days ago 29 replies      
I think much of Reddit's problems with its userbase boil down to an early failure to manage expectations.

It's pretty clear that the Reddit corporation doesn't want Reddit to be an anything-goes, absolute free speech zone with no moderation or anti-harassment policies -- but that's what the site actually was for many years. Now, when the company cracks down, users think their freedoms are being curtailed. The mistake was ever allowing that kind of freedom in the first place, because people developed an expectation that it would persist.

Compounding that problem, the fact that the site was unregulated for so long caused it to attract the kind of people who need to be regulated the most. In other words, it's no surprise that the most tolerant communities attract people who are difficult to tolerate.

I suspect Twitter is having similar issues dealing with harassment, after letting it happen for so long. If there is a lesson in this, it is that online communities which plan to implement anti-harassment policies ought to do so from the beginning, and develop the expectation among the users that such policies exist, and will continue to exist. Don't just tack them on after several years, and don't enforce them inconsistently and arbitrarily as Reddit has done.

It will be difficult for Cond Nast to get its money's worth out of Reddit now. I doubt it will ever shake its negative reputation.

2
Iv 2 days ago 3 replies      
Here is a good time to repost a paragraph about why one should not trust Reddit, by Reddit's then CEO, Yishan Wong:

> I am continually astounded that people sort of trust corporations like they trust people. We can talk all day about how the current team is trustworthy and we're not in the business of screwing you, but I also have to say that you can never predict what happens. reddit could be subject to some kind of hostile takeover, or we go bankrupt (Please buy reddit gold) and our assets are sold to some creditor. The owners of corporations can change - look what happened to MySQL, who sold to Sun Microsystems, who they trusted to support its open source ethos - and then Sun failed and now it's all owned by Oracle. Or LiveJournal, which was very user-loyal but then sold itself to SixApart (still kinda loyal) which failed and then was bought by some Russian company. I am working hard to make sure that reddit is successful on its own and can protect its values and do right by its users but please, you should protect yourselves by being prudent. The terms of our User Agreement are written to be broad enough to give us flexibility because we don't know what mediums reddit may evolve on to, and they are sufficiently standard in the legal world in that way so that we can leverage legal precedents to protect our rights, but much of what happens in practice depends on the intentions of the parties involved.

> The User Agreement is intended to protect us by outlining what rights we claim. But it cannot protect you - you must protect yourself, by acting wisely.

3
qwertyuiop924 1 day ago 2 replies      
Reddit is a crossroads: It's an intersection between the cultures of 4chan (which is in itself an intersection of japanese and american sensibilities), and the culture of usenet, and internet forums, and a dozen other cultures besides.

None of these cultures handle censorship well. They all originated in an environment where, to some extent, you could say whatever the hell you like.

Many of Reddit's early users came from these cultures, and they were responsible for the early culture of the site.

And now, Reddit is desparately trying to adapt itself, and attract people from Twitter, Facebook, and Tumblr, whose cultures are radically different, and perhaps even to some degree less toxic than the pre-September usenet, whilst also being more toxic. I don't know how.

The point is, a culture that previously only dealt with unacceptability in relative terms - this is unacceptable in this context - is now dealing with absolute unacceptability - this is not acceptable, ever. This isn't a change that people will likely adapt to well. This is prompting a migration to sites like Voat, and others.

The problem is, Reddit is introducing censorship which is incredibly inconsistant to a site where the concept of censorship is anathema - Bans, yes, people get punished for breaking the rules. But having your posts quietly vanish without warning?

No wonder the userbase is pissed.

Unless I got it completely wrong, which is possible.

4
xnull2guest 1 day ago 4 replies      
Reddit's content curation has come at a time when social media writ large (Facebook, Twitter) has become linked into State Department and DoD programs. Counter-intelligence objectives are fought on the 'private property' of social media servers that host the content of individuals. Fighting the 'War of Ideas' in the 'cognitive domain of warfare', the effort to starve unwanted ideas for a place to roost and feed others to maturity is certainly useful, but it comes at a cost.

There is some value in ungoverned spaces, where advertisements, political astroturfing, politicized content curation ("no 'RT' allowed, but we'll allow VoA and Sky") play a secondary role to the contributions of individuals.

The internet was supposed to be an ungoverned space - a 'piazza' or 'forum' - but when it wasn't and when the 'Web' wasn't social media was supposed to fill this gap. Behind the cry of those protesting the take down of 'revenge porn' and 'fat hate' postings, I hear the more sober voice that adknowledges that there's one less place that's a safe and free place for expression - as unpopular as some of it may be.

5
grandalf 2 days ago 5 replies      
Why does Reddit have to become a media empire?

The formula for doing that is pretty well-known by this point:

We'll see a ban on throwaway accounts and a push for real names, then a ban on third party URL shorteners, then interruption ads, and finally some sort of paywall.

Reddit is a useful piece of internet infrastructure, and I'd be pleased if it would stay that way. It doesn't need to become its own media empire with its own Rupert Murdoch, etc.

Some things that could be improved:

- opt-in home pages that are tailored at specific audiences. The standard one is pretty low quality.

- more detection/policing of voting rings and vote fraud in general.

6
jokoon 1 day ago 1 reply      
I like reddit. I don't really care so much about the frontpage. I like other subreddits where discussion is central (bestof, subredditdrama, changemyview, self, ask<insert-subject>), or content subreddit (games, wallpapers, military, photos). The default subreddits feel like google news.

What's important is the users and how there is room for them to exchange both ways, unlike standard medias.

There also are many people watching for bias, would it come from moderation, brigade, corporate, etc. You will often reads posts about actual professional in a field explaining you something, and it often is enlightening (granted that I would not trust reddit for a decision that implies my own existence).

Generally, reddit works because the users can see and feel that people are exchanging, talking, sharing, reacting. It's "alive". Even facebook cannot really pretend being that lively place, that "bazaar".

What must be really tough is how you manage that many teams of moderators. That must be a nightmare, but to me it seems that it's vital. Fortunately it seems that they will always find people for that, because their subreddit revolves around something they like, and they will often do a good job (it seems) because they want to promote that hobby, not that it will directly benefit them financially (example, moderator of askhistorians).

7
retox 1 day ago 1 reply      
The human race is comprised of horrible people, any website that accepts user contributions will attract contributions from horrible people. As a service owner you have a decision; either you say that everyone's opinion is valid, horrible and all, or you say no; these are the rules around what you can post and anything outside those boundaries is subject to removal.

What you absolutely should _not_ do is build a brand around being in the first category and then transition to the second. Especially if all your content is user contributed.

Of course, no one thinks of themselves or their in-group as horrible. You could substitute horrible for flawed if it makes you feel better.

8
minimaxir 2 days ago 6 replies      
Reddit, from a business perspective, baffles me. During the Yishan Wong/Ellen Pao era, we had Reddit-Made and Reddit TV, both of which bombed especially. Under Alexis Ohanian, we had Upvoted and Formative which as the article notes were killed silently.

Reddit released a native app and an image host years too late. (I just checked the data and it is not killing Imgur: Reddit image usage was 18% in the top image subreddits at beginning of June, today it is 25%).

The biggest fundamental change Reddit has made in the time since is...making self-posts count for karma. And tracking outbound links.

It really shouldn't be that impossible to have a successful business with hundreds of millions of users. Especially with the wealth of data available to Reddit.

9
anjc 2 days ago 4 replies      
It's bizarre that a company seems to be struggling to administer diversity of staff, without being certain of its own medium-term success.

There's no point in having your quota of "people of colour", as the article puts it, if the business model is unsustainable and leads to people being fired anyway.

Why not focus on creating a successful company first, and then worry about things that carry an administrative and management overhead.

10
jfoutz 2 days ago 6 replies      
I didn't realize drinking on the job was a thing. I've had the odd company party with beer in the late afternoon, but that's perhaps twice a year.

I'm not a teetotaler by any stretch of the imagination, but drinking at work seems counterproductive.

11
bane 2 days ago 14 replies      
Why the hell is Reddit trying to make their own content when the entire point of the site is for the users to create the content?

Don't make a Reddit podcast, make Reddit a podcast hosting network.

Don't make a Reddit video show, make Reddit a video hosting site.

Don't make a Reddit magazine, make Reddit a source for anybody to publish their own magazines.

Why does Reddit have writers and editors and creative directors? It's like a rock band having a position for a flower arranger.

Is this the point where Reddit has official jumped the shark? Where to next for my cat picture memes?

edit

Want to increase quality and revenue? Give a cut of advertising revenue to the mods of successful, high quality subs.

Incentivize for the behavior you want.

Provide the platform and get the users to provide, mod and benefit from the content.

12
jeiting 2 days ago 6 replies      
Since Alexis and Huffman returned I've seen more happen with the brand than in the several years preceding. I don't know who made their mobile app but it is damn good. I resisted at first but am now using it as my primary means for consuming content on Reddit.
13
GarrisonPrime 1 day ago 0 replies      
"Ooh, lots of people like this thing. It's popular."

"Great! Let's take advantage of that popularity to make a ton of cash!"

"Hm. We'll have to dramatically change pretty much everything about how it operates."

"What could go wrong?"

14
mark_l_watson 1 day ago 0 replies      
I am surprised that the core Reddit functionality is not run mostly on autopilot.

I only subscribe to a few subreddits (lisp, Ruby, Haskell, AGI, and a few others) and the user supplied content is plenty good enough for me to visit the site once a day.

15
spaceheeder 1 day ago 1 reply      
If this is related to the problems at Reddit it is only so tangentially, but my feelings on that site have been very mixed since I deleted my account there. I think that the subreddit structure and making it "a community of communities" showcases both the best and the worst of audience bubbles. At their best, like-minded people share interesting things with each other, build communities, and even form friendships. At their worst, they become echo chambers that are almost as liable to turn on themselves as they are on outsiders.

I've heard people make the case that audience bubbles are bad for society at large, because they narrow down what kinds of conversations people have. But ever since leaving Reddit, I've noticed my own outlook on life improving. I think that audience bubbles cause an individual harm, similar in kind to that reported by people who de-convert from extremist religious or political ideologies.

I wonder how much better off people would be if social networks implemented some kind of "group hug" algorithm that made posts less likely to spread if they were too in-groupy, and made people more likely to receive posts from wider and wider venn diagarams of adjascent audience bubbles the more insular their own posts seemed to be. You wouldn't even have to force people to confront antagonistic views, just make them more likely to see more moderate ones.

16
jsprogrammer 2 days ago 1 reply      
I would like to see some reporting on the 10% of reddit's recent $50 million raise that will be distributed to the users.

Recent changes (eg. stealth adding link tracking) and comments (eg. Huffman's, we know everything about you) have been user hostile and making the distribution would garner some good will.

17
thekevan 2 days ago 2 replies      
My use of reddit is way own. I'm tired of going to the front page and seeing so many submissions about things I don't care about like multiple video games, dumb inside jokes like r/circlejerk or all those repetitive links about Trump or Sanders.

I know I can buy gold and customize the front page, but I am a little hesitant to pay money to make the front page not suck. I have ads turned on in my adblocker so they do get ad revenue from me, I'm not totally freeloading Also, there is a limitation to the number of subreddits you can exclude. That is the nail in the coffin right there for me buying gold.

Finally, I'm not impressed with some of the censorship and social policies they have and don't really want to support a business who seems to have either questionable or widely varying policies on things.

The result, I check it maybe once a day, down from several times a day.

18
a_small_island 2 days ago 0 replies      
>"Reddits Upvoted podcast, which Ohanian launched in January 2015, also appears to be abandoned. Aside from a single episode published in June, the podcast hasnt been updated since October 2015. The Formative video series produced in partnership with Google, which aired new episodes roughly once a month since its launch, has been dark for four months."

Hopefully any employees hired specifically for these ideas were able to find other groups at the company or future employment.

19
orionblastar 1 day ago 1 reply      
Well seeing failures in Reddit, are almost the same failures I saw in other dotcom startups, not having a business plan that works.

When Reddit was formed, it supported free speech of everyone. They didn't have a plan to earn a profit, they just wanted a better discussion board than Slashdot, Kuro5hin, Digg, Stumbleupon and others. Digg eventually had to make changes to their site that went into the paid accounts and paying for promotion/advertising of links/info.

I first studied computer science and data processing aka information systems it was later called. Then I went back to college to earn a business management degree to learn how to make working business plans.

At South Park they had a skit about Underpants Gnomes, that parodied the startups out there:

Step 1 Steal Underpants!

Step 2 ?

Step 3 Profit!

This is basically a joke, but some companies have an incomplete plan like that.

Ellen Pao was a patsy for the board of directors to blame when the changes they wanted to implement would prove to be unpopular but attract better advertisers that had liberal points of view to support Reddit.

I cite Kuro5hin, because it once was a very good site, and it didn't have a good working plan or a very good editors or management and eventually spirals down into a forum controlled by the trolls that chased everyone else away. Then it was mismanaged and then it went down and went to a new server and was never recovered from backup. Kuro5hin never had a good working business plan, it was like an Underpants Gnome business plan. The users created the content, it got voted up or down, section or front page, and if a story didn't make it you could always post it as a diary in the 'Ghetto' section as the users called it.

Reddit is suffering what Kuro5hin did, the trolls start to take over certain subreddits, and drive people away. They post racist, offensive, and mean things and all band together to vote it up to the front page. Subreddits like /r/Ferguson that was about the Ferguson riots and Mike Brown got taken over by trolls posting racist stuff and so Reddit quarantined that subreddit and gave warnings to people subscribed to it.

Ellen Pao was a scapegoat who carrier out an agenda by the board of directors. She was given the job of CEO knowing that she would fail, and make the changes the board of directors wanted that would make her unpopular to users, but popular with advertisers.

At that point Reddit was no longer about free speech, but censorship, Reddit didn't trust the users to create content so they hired editors to create their own content and blog. Sort of like what Digg did. If they follow Digg they will take paid promotions of links and try to shut down accounts they don't agree with and delete and censor them.

I have to say looking at it from a business angle they can't monetize content if they keep banning and censoring users and try to take control of what appears on the front page. Either they are for free speech or not, either they want controlled speech that meets Liberal guidelines and a Social Justice Agenda, to attract more people like that to provide a safe place on the Internet, or they let the users decide and vote on it democratically even if they don't agree with the politics, or speech, and then the trolls get control of the front page like they did with Kuro5hin.

Actually I like bane's suggestions that Reddit make podcast hosting networks, video networks, get into e-publishing and other stuff that they can sell advertising on or use to pay for a membership to remove the advertising.

There has to be some sort of sane way to earn an income, by advertising, or paying for memberships to avoid seeing the advertising, and making a paywall to verify accounts for $1 or $5 to keep the spammers and trolls out that want to use free accounts and bots to control what is on the front page.

They need a Baysian filter to detect the spam and junk, the same way email programs do it. I've seen a lot of spam and junk posts in /learnprogramming and other subreddits and I always flag it, but it takes a long time for someone to look into it.

After the Ellen Pao scandal many alternatives to Reddit got founded. They have to treat their employees as human beings with equal rights, which is what they are supposed to believe in via liberal values, but instead they fire employees and don't work with them to settle differences. Reddit seems to be hostile towards diverse hires, even using some like Ellen Pao as scapegoats and patsies.

They need to take responsibility for their mistakes, change their business plan so it works, and find a way to hire more diversely and treat employees and contractors better so they don't leave or get fired.

20
cocotino 1 day ago 0 replies      
Big chunk of the article gone explaining how the diversity policies have failed, but I don't see any explanation on how they would have helped the site or the community.

What if the company is failing because instead of focusing on hiring competent people (of which they have a severe lack, at least in the engineering side) they focused on having a diverse team?

21
fit2rule 1 day ago 1 reply      
Let reddit die, lets all go back to USENET.
22
dghughes 1 day ago 0 replies      
My uncle used to say to me nothing ruins a business faster than changing it. He gave an example of a diner that did well for decades then decided to expand and very soon after went out of business; bankrupt.

To me reddit has done well because it has stayed the same for years digg is the diner that expanded.

23
SixSigma 1 day ago 0 replies      
No mention of the battle with The_Donald

https://www.reddit.com/r/The_Donald/comments/4oenqn/uspez_ad...

Reddit shit its pants when the Trump Train came to town and disturbed the echoes in the chamber.

I have a "Freedom From The Press" Reddit t-shirt. I am embarrased to wear it.

24
programminggeek 1 day ago 1 reply      
It seems like the tribe that built the site and the tribe that is trying to run it now are not the same thing. The culture of reddit is not civilized, equal, or any of the other HR type directives they are going to try and make.

If the people in charge now tried to start reddit back then, with all the focus on fairness, equality, correctness, and inclusiveness reddit would never exist.

It would be something else. I'm not sure it would be bigger, smaller, better, or worse, but I know it wouldn't be reddit.

You can't have a bunch of "bros" build a popular site and then pretend that they didn't. You can't bring in a bunch of nerds into a community and then kick them out and take it away from them once it's popular and successful.

Actually you can try and do those things, but it won't work because the tribe will reject you and go somewhere else. It happened to slashdot, it happened to digg, it will happen to reddit and Hacker News too.

If you don't understand the tribe, you can't hope to lead them. You don't lead the tribe by pretending it's not what it is.

Reddit's owners and operators seem to be ashamed by their tribe. That is going to be their downfall.

25
dbg31415 1 day ago 0 replies      
There's a lot of sloppy censorship on Reddit.

Mods are mods aren't paid, so they use a lot of very broad auto-moderator bots -- many of which are very poorly written. You can get a permanent ban from a Subreddit for having a user name the bot finds offensive, for example. You can have a post removed because you didn't end your question in the syntax the bot was expecting... and even if you fix it, and appeal to the mods, and they reverse the decision, your post gets restored as older and without any votes so nobody will ever see it. It's more that the moderation is an example of bad automation -- I think this is what reasonable users get up in arms about.

26
ta12347 2 days ago 2 replies      
Reddit isn't Internet infrastructure. It is a startup. The plan from day one was to make a kinder, gentler imageboard, get acquired, and make the founders rich.
27
u238ed 2 days ago 3 replies      
Wow. What a hit piece. I bet their anonymous source happens to be the ex VP of Product that was fired by Pao, and is now building a 'safer' Reddit. Conflict much? In their sources quote, it's evident the source knows a lot of the inner workings of the leadership, is a man, and doesn't work there anymore. Well, that leaves only the ex VP, seeing as though Pao was a woman.

So they took all of Dan's words as truth? The guy with the competing Reddit? Come on Techcrunch. That's pretty bad.

Now, for the facts:

1. None of these people left voluntarily. They were let go. So they need to stop with the 'people are leaving in droves' nonsense. Reddit has has almost multiplied in employee count in the last year, and has moved to a new office, much larger office building.

2. All of these people, except for Nicole were part of an experimental product that was cancelled. If this was Google or Facebook, you wouldn't even know. It happens every day. But because you got a juicy tip from an ex-employee building a shitty competitor, you run with it. Because, hell, it's Reddit, and you just might hit the frontpage!

3. Shit like this, "The plans to overhaul Reddits reputation as a hotbed for harassment and to remake the company as a multi-media publisher have yet to prove successful at it seems that the departures of senior employees are impacting Reddits product and performance." -- WTF does that mean? They tried Upvoted, it didn't work, and they're folding it back into Reddit.com and letting go of people that they don't need for the next iteration. That's business. It has nothing to do with color of skin or your genitals.

4. Numbers always dip in the summers. Especially for Reddit. The kids are off for the summer. It would make sense that they peaked right before summer. According to Alexa, Reddit is the 9th largest site in the US, and up 9 spots this month on the Global list to #27. So it seems they are definitely growing.

28
u238ed 2 days ago 2 replies      
You should also disclose that you're the source for the TC article. It's pretty obvious.
29
awesomerobot 2 days ago 0 replies      
tirefire keeps on burning
30
revelation 1 day ago 0 replies      
But re-blogging content from AMAs seems at odds with Upvoteds mission to produce original journalism, and many of the writers hired in October 2015 were let go just three months later

What a joke, a bunch of kids playing startup one more time in their favourite playground. Utter disgrace.

31
beedogs 1 day ago 0 replies      
Still a garbage dump full of racists, misogynists, and low-class trolls, too. Reddit should be nuked from orbit.
32
gdulli 2 days ago 1 reply      
Reddit is a content management system for spam and stupidity.
33
ben_jones 1 day ago 1 reply      
Reddit is, was, and will for the foreseeable future be, a giant pile of porn. Denial of that fact shows a fundamental lack of knowledge of not only Reddit but the internet as a whole..

I can understand that this idea would be extremely unpleasant to a lot of people. And a slow migration to curated content may seem more wholesome. But it doesn't work that way.

34
intoverflow2 1 day ago 3 replies      
The main problem with Reddit is the users dishonesty to themselves.

Ask a reddit user why they like reddit they'll probably mumble something about AMAs.

Then visit /r/all and a completely different picture will be painted for you as the sites most popular content is a mixture of pornography, racist jokes passed off as being subversive and political ranting.

18
Why I wont give talks about being a woman in tech soledadpenades.com
372 points by robin_reala  3 days ago   352 comments top 44
1
someone7x 3 days ago 3 replies      
I went to ngconf this year and two of the talks that stood out to me were given by women.

One was the angular materials talk / demo and it was amazing. The presenter was exuding tech prowess, I was blown away by how easy she made it look to make a dog adoption website. One of the best talks.

The other was by the CEO of girl scouts giving a patronizing 5 minute talk about how we need all help women in tech succeed and change ourselves so the world can change for the better. One of the worst talks.

I had those in mind when I read the article and for that reason I think I can see where she's coming from. When a woman just gives a tech talk, it's just a tech talk incidentally given by a woman. Isn't that the goal? More talks like the first one I described?

2
btilly 3 days ago 8 replies      
I personally know several successful professional women who have a policy of refusing to belong to any women-only groups. Their reason is that in their experience such groups are populated by people seeking reassurance. The result is that they offer the "support" of lowered expectations. Which won't help you succeed.

One also pointed out to me that if a group of men were to form a men's only business club, that would be seen as sexist. It is no less sexist to form a women's only club, but nobody sees fit to criticize it.

This is not a bias against women in general. They just refuse to deal with people whose identify first as women, and only secondarily as professionals.

3
thonos 3 days ago 3 replies      
I went to a recent tech conference and a few things came back when I read this article. In particular that the conf had a lot more female talkers than male ones (It was curated and invitation based).

Most talks were good but a handful were nuggets where clearly the deciding factor why that person got the talk was because she was a woman and not her expertise in the area.

Your typical dose of women who code talks were im there too but one that stood our from the rest was a woman who thought she kept having to tell people things like "use your slackbot to tell people to stop using 'guys' and 'team' instead." or "women need remote work so they can cry silently when their male colleagues steal their ideas".

I am not denying that there are gender issues in tech (though in my career path I have yet to encounter them), but I paid good money out of my own pocket for that conference.

I am not going there to see you speak. I am going there to learn and get value for my money.

4
renegadesensei 3 days ago 0 replies      
I feel similarly about being a black guy in the tech industry. It is always the deliberate efforts to "reach out" to minorities that make me feel the most uncomfortable and unwelcome. I have given talks on AWS, Cassandra, Python, and other subjects. You could never get me to talk about "being a minority in tech." Similarly I live in Tokyo and have no interest writing or talking about "being black in Japan."

Thoughts from my blog: https://righteousruminations.blogspot.com/2014/11/another-si...

Recent thoughts on tokenism:https://righteousruminations.blogspot.com/2016/07/on-changin...

5
droopybuns 3 days ago 0 replies      
I respect this person's rationale. I'm thankful to hear her perspective.

I know lots of incredible women in tech doing great things. I assume they and everyone else wakes up each day and has to figure out what challenges they'll be overcoming, and how they will end up spending their one life on this earth.

So this public rejection of gender-specific talks nourishes me, because I am dropping all packets when someone starts to talk about gender or privilege issues.

I feel that enthusiasm for these topics is a tell that the speaker is a narcissist who believes other people exist to either validate their own opinions or serve as an adversary. It is uncompassionate.

I suspect a group of academics organically instrumented a taxonomy that directly mirrors established trolling tactics. They have spent the last 10 years providing gender studies philosophies that are being implemented by graduates- who will now get to discover firsthand whether these ideas are constructive.

Kids now think that disagreement is evidence of cultural misogyny and racism. Well, I disagree, but I'm not going to bother trying to engage with this type of person. Where is your diversity now? How is your behavior going to cultivate the outcome you desire?

It's left me feeling exhausted and repulsed by the topic. I'm wary of some women in tech now because of their enthusiasm for these ideas. It is very frustrating. I'd like to support them, but I also want to lead a happy life. I want positive, encouraging people around me. The privilege crowd just doesn't seem healthy.

This person seems pretty thoughtful. She has nothing to gain from posting something like this. I feel a little less cynical after reading her post.

6
6stringmerc 3 days ago 0 replies      
Very nicely worded sentiments and I think it's a good counter-point, a rational and thought out one, to the knee-jerk habit of having a trend-chasing, "WE CARE!" framing around certain issues. This piece is strong in pointing out that "ISSUE X IN TECH" is not particularly a "tech talk" - it's more in the sociology/humanity side of discussion, right? I like how this tries to make that distinction.

On a personal note, I think I appreciate the article a bit more because I could substitute "handicapped person with condition X" for the same kind of framing that she's discussing. I don't want to be known as X, I want to focus on the subject matter. If I happen to be an inspiration for others in the X group, super, great, blaze a trail and thank me later if you really feel compelled, but that's not the purpose of me pursuing success. It's not "in spite of X" it's just that X is another inconvenience in the way of goals, much like having to pay taxes or empty my cat's litter box, scope and effort aside.

7
pselbert 3 days ago 2 replies      
Sandy Metz, of "Practical Object Oriented Design in Ruby" [1] fame made a comment about this on The Bikeshed[2] recently. She stated that she refuses to make reference to gender when she is giving her talks, though her gender is ultimately what got her the opportunity to write a book and talk in the first place.

Ultimately she is regarded as an amazing teacher and a dynamic speaker, not because she is a woman in tech.

[1] http://www.poodr.com/[2] http://bikeshed.fm/70

8
cocktailpeanuts 3 days ago 0 replies      
I'm glad there actually is a competent woman who can say this out loud. Nowadays it's impossible to say anything against "We need more women in tech" without being called a sexist.

I do realize it is harder for women but the world is not a fair place. Poor people who were born to poor parents are born into an unfair world. A white guy or asian guy who really wants to play basketball in NBA finds himself in an unfair situation. But that's what powers these people. A lot of successful people came from bad background because they grew up being sick of this unfairness and they tried hard to get there.

To use the NBA example, you never see Jeremy Lin or Yaoming giving talks about how "We need more asian basketball players in NBA". They are well aware of how that's how it is, but still managed to succeed by pushing themselves hard.

Again, I do realize it's unfair, but if I were someone in an unfair situation I would spend 100% of my time working hard to overcome it, instead of using my precious time thinking and talking about how my group needs to be more well represented.

9
StavrosK 3 days ago 0 replies      
Soledad came to my city (Thessaloniki) for a conference a few months ago, and gave a very interesting talk on the new audio/graphics APIs in browsers. It was a great talk, and, I agree with her, much more interesting than "I'm a woman, here's my experience".

I also dislike the mentality this mindset implies that I, as a man, should be surprised that a woman can code, and should therefore ask her about how she managed that feat, as if it's not pretty much exactly the same as how I started.

10
mc32 3 days ago 0 replies      
"It not only is very insulting and distracting, but also pigeonholes you into talking about being a woman in tech, instead of woman who knows her tech. It feels like, once again, were delegating on women and other vulnerable collectives the caring for others matters, in addition to their normal job. That is not OK."

Sometimes, identity, gets in the way of things.

Do what you feel comfortable with. Do it for yourself. Don't do it for or because of others. Feel that you want to do it for your sake and for its own sake. Doing something because of agendas, can be good for the group, but, it's less clear it's always good for the individual.

In a nice world, you'd be valued for many things, not just your economic productivity and contribution. And our identifiers would be afterthoughts. But for friends and foes alike, some at least, it's clear identifiers are important and some would want to find leverage and make use of the opportunity. Yet, it's not owed, and it's up to you if you feel comfortable with lending yourself for a cause, as it were.

That said, just do what you like to do, don't explain it as a result of principles, etc. What I mean, our decisions don't have to be internally politically explained, or consistent. Just like liking or not liking broccoli does not have to be internally politicized to like it or not like it (or bacon).

11
ap0 3 days ago 0 replies      
Showing that women have great technical aptitude by giving a legitimately interesting tech talk is much better for the cause of promoting diversity than just talking about being a woman or minority in tech, IMO.

I worked at a large online retailer that catered primarily to women, and internally there was a large push to hire more women. We hired two women on my team. One was fantastic, one was horrible. The fantastic one passed the interview loop without reservation, and would have been hired regardless of her gender. The other did not do as well and multiple people had reservations, but she was hired anyway. She was an immediate burden and terminated after three months.

The first one didn't need any sort of handicap for being a woman -- she was qualified and competent. The other one just didn't belong in this role. But management aimed for diversity over competence, and ended up hurting morale.

Treating people like equals is the best way to achieve equality. Don't insult them, and don't let those who legitimately don't have the skills necessary through because of their identity. Seems pretty common sense to me.

12
jordigh 3 days ago 0 replies      
> No! The answer to an all male line-up is not a talk on women on tech by a women. The answer is diverse people in the line-up, talking about tech.

At the last US Pycon, where 40% of the speakers were female, there were a broad range of topics. Several female speakers did speak about "soft" issues like being a woman in tech, but many others also spoke about "hard" purely tech issues. There were also males on both sides of the soft/hard line. You can see the topics here:

https://us.pycon.org/2016/schedule/talks/

I don't think it's a problem to give soft talks, and I think Pycon is doing a great job of increasing diversity. It's not perfect and there's work to be done, but I really don't see soft talks by women as an evil to be avoided. If people want to give soft talks, let them do it.

13
pritianka 3 days ago 0 replies      
I 100% agree with this article. Whenever I am invited to talk, I always speak about my work and expertise, as opposed to women in tech stuff, because being there and being good at what I do, is much more effective than statements about diversity (IMHO). The only time I've done women in tech type conversations, it's been in small, intimate settings for an all-female audience. In that scenario, it makes sense to discuss the challenges and learn from each other.
14
spoiler 3 days ago 0 replies      
I agree 100% with the author.

Another thing that I feel this "let's talk about women in tech" attitude is causing is causing a negative effect, rather than a good one. An example of this is my (female) friend who rolls her eyes at any mention of "women in tech" and makes jokes about all these online "troll" feminists[1].

We should as a communitylike the author herself saidfocus more on inspiring women to join the industry; not talk them into it. I think the author's suggestion to have confident women talk about their awesome tech is a great start!

[1]: I am not trying to discredit feminism. We can't deny there are some people who take it too far when they have online discussions; this happens regardless of the topic being discussed.

15
return0 3 days ago 0 replies      
Something similar happens in science too. Anecdotally i watched it happen with my supervisor, as the latest "women in science" wave started about a decade ago. She got a number of administrative positions, which I increasingly felt she got "because she was a woman". That led to her being visibly left behind in her scientific field. It's funny, because she s far from what you would describe as feminist.
16
qwertyuiop924 3 days ago 0 replies      
Yes. I agree. So much.

These are my metrics for a good tech talk:

1) It is informative

2) It is entertaining

3) It is actually about tech

Well IAWAT (I Am A Woman In Tech) talks can be 1 and 2, they cannot be 3. That would be okay, if they were informative or entertaining, but they so often aren't. Many just quote the same statistics we've heard before, and call for change. There is a reason that I will watch Piotr Szotkowski's "Standard Library, Uncommon Uses," Or Linus Torvalds' talk on Git, or Hilary Mason's opening talk at FutureStack, or absolutely every talk Bryan Cantrill does (even if it IS just to play Bryan Cantrill Bingo), or countless other talks whose names I forget. Because they are informative, they are entertaining, and they are about tech. And male or female, if you give a good talk, I'll listen. If you just want to get on the stage and talk about your gender, then I will be out of your talk faster than an ICMP packet travelling down an empty fiber cable.

17
tomc1985 3 days ago 0 replies      
The secret to diversity is not giving a shit about stupid criteria like race or gender, and shutting the hell up about it. Quiet acceptance of anyone who fits and does good work.

People talk about diversity like it's some magical talisman. It isn't. Diversity is: not being turned off by someone who doesn't possess whatever stupid criteria you think make for a good human being. That's all.

18
jordigh 3 days ago 0 replies      
This is called the Unicorn Law:

http://geekfeminism.wikia.com/wiki/Unicorn_Law

19
lalos 3 days ago 0 replies      
Enjoyed 'but also pigeonholes you into talking about being a woman in tech, instead of woman who knows her tech.'

Subtle difference, I believe that's the purpose of this up and coming podcast show that I've been following [1]. Women having a space to talk about tech instead of talking about how's it like to be a women in tech.

[1] https://thewomenintechshow.com

20
Frondo 3 days ago 10 replies      
Personal story time...I was at a recent conference, listening to men, women given talks. One of the women, in her talk, uses an interaction with her daughter as the framing narrative for the talk (imagine Socrates and Glaucon, only this woman and her daughter). Almost immediately, three guys to my right start making fun of the speaker, sotto voce, making all manner of little jokes to themselves, that are becoming increasingly gender-specific.

When one of them says loudly enough for me and others to hear, "Your daughter sounds like a real bitch! hur hur", I turned to them and said, "You guys need to knock this off now."

Embarrassed silence. And it stayed that way. Of course, I'm a tall well-built, well-dressed white guy, i.e. all the things that automatically command respect.

The thing is, I don't think those guys were bad people, I don't think they sat there in their minds thinking, "let's tear the woman down". I really don't think they thought about it at all. I also don't think they'd have sat there, chattering away, if a man had used the same framing story (father and son, father and daughter).

I think there's a lot of unconscious, unconsidered, unthoughtful bias that they (and we) all carry around by default, that makes it easy to pick on weaker people if you're going to pick on anyone. And there are a lot fewer women around these tech conferences, and they're used to placating aggressive people, putting up with shit, etc.

I guess, I wish women didn't have to talk about being a woman in tech. I hope they keep doing it, though, until people start thinking about this kind of asinine behavior.

21
415Kathleem 3 days ago 1 reply      
While I can certainly understand where the woman who wrote this article is coming from, I really enjoy hearing and seeing women in tech. I work in tech, albeit peripherally (EA/Admin role at the moment, just getting in the door), and 99% of the time I feel like I've wandered into a men's club. I am treated really well by my male colleagues (I'm lucky enough to work with a group of kind, talented people, though), and generally am treated well by the men I meet at meetups and the SF tech scene, but they don't see me as a threat. I'm not on GitHub responding to code reviews and changing things they've worked on. I'm not competing for their jobs. I have a feeling that the second that happened, a large percentage of the men who are now cordial to me would be less than that. I guess my point is that though I see why nobody wants to be a token female on a panel, and nobody should be coerced into giving talks they're uncomfortable or unqualified for, as a woman just stepping into the scene, it would be really great to see more women speaking out visibly in the field.
22
triplesec 3 days ago 0 replies      
The article makes a great point that just being a woman doesn't make you a good speaker about [the sociological problems] of women in tech. Rather than have her out of her skillset, hire her to talk about databases, or other parts of her expertise, and hire academics, HR and others (of both genders, according to skill relevance) qualified to talk about social-tech challenges.
23
cloudjacker 3 days ago 2 replies      
I agree

Now for the counterpoint, there are a lot of people in the marginalized group that actually do look up to people that advertise their marginalization.

Everyone that doesn't advertise that they are a "woman in tech" or a "black executive" flies completely under the radar.

There are literally groups I've been invited to where people make lists of these inspirational characters, because they want to support their businesses more than others. And the people that never said "I'm a black female software engineer that got VC funding" never show up. People assume they don't exist, when the reality is that isn't what they wanted to be known for!

Any compromises?

24
colmvp 3 days ago 0 replies      
I think it depends on the state of knowledge. The tech industry is well aware of the gender/ethnicity distribution. And quite frankly, at least companies, community groups, and profound leaders have progressively been doing something about it for the last number of years. It just takes time to make progress. Look at other industries which have poor representation of men or ethnicities and see how many inroads they've made in say, the last five years compared to tech.
25
throwaway991823 3 days ago 0 replies      
Had to use throwaway here, sorry.

I attend and organise various conferences and its simple. You got to get Women. 10% minimum, more is better.

One of our own engineer (who is very mediocre but a women) got accepted to a major conference. She was told her talk is not so good, but they still want her and she should think of something.

End result? When I see a woman talking in a conference I assume she is there because of her gender, not skills.

Lose lose to all

26
kkelleey 3 days ago 0 replies      
Does anyone know if there is data somewhere that shows the male/female speaker proportion across different conferences?

Would be interesting to see how it compares across the different industries. I didn't see anything after some basic googling.

27
Mz 3 days ago 0 replies      
If someone approaches me to talk somewhere just because Im a woman, they havent done their job of finding what my expertise is. Therefore, I am going to insta-decline.

This is a really good policy. She's 100% right.

28
ktRolster 3 days ago 0 replies      
Because I am fine writing, but feel completely awkward displaying myself in public! (I'm speaking for myself here).
29
ivanhoe 3 days ago 0 replies      
Huge respect for this lady, both for the attitude and for all the interesting tech stuff that she does...
30
digitalpacman 3 days ago 0 replies      
I agree with this. I am male though so my opinion might not count.
31
shanemhansen 3 days ago 0 replies      
Reminds me of a response once given by Richard Feynman (I can't find the source so I apologize if I misremember it). Essentially he was to be included in a book of successful Jewish people. He declined because he considered his ethnoreligious background to be irrelevant to his accomplishments as a physicist.

To stave off the inevitable response that the experience of a white american man is irrelevant, let me leave this quote from the head of the princeton physics department: "Is Feynman Jewish? We have no definite rule against Jews but like to keep their proportion in our department reasonably small". So even in the face of systematic discrimination Feynman wanted to be known not as a Jewish scientist, but as a scientist.

32
skylan_q 3 days ago 5 replies      
If there is no gender parity, there is a problem.

No, there isn't. We don't have a "gender parity problem" in nursing, pre-k education, garbage collection, nor boilermaking. Just because it isn't 50/50 doesn't mean it's a problem.

33
cheez0r 3 days ago 2 replies      
That sure sounds like what the tech industry always seems to prefer: walk the walk, not talk the talk; software, not slideware; etc. Be a strong tech woman and you don't have to advocate for them; your actions speak louder than words.

Just ask Dr. Neil DeGrasse Tyson.

34
jondubois 3 days ago 2 replies      
I don't think women have it particularly hard in tech. Some aspects are harder but other aspects are easier.
35
sp332 3 days ago 3 replies      
A better title would be "I won't give talks about being a woman in tech". She's not against talking about it in general.
36
3 days ago 3 days ago 1 reply      
Thanks, I've fixed it.
37
mhurron 3 days ago 4 replies      
> I don't think those guys were bad people, I don't think they sat there in their minds thinking, "let's tear the woman down". I really don't think they thought about it at all

Yes, they are bad people, and the fact that they didn't have to think about going out of there way to tear someone down is part of why they are bad people.

They didn't have to think about "let's tear the woman down" because they were already at the position 'she didn't belong there.' And yes, there would most likely be something similar with a man and his daughter, it just would simply be along the lines 'she's never going to get married if you teach her to act like a boy.'

> I think there's a lot of unconscious, unconsidered, unthoughtful bias that they (and we) all carry around by default,

Yes there is, and it makes you a bad person. You should be derided for it every time it shows itself.

38
not_anit_woman 3 days ago 2 replies      
Your right, women get hired more easily and get paid more, so it is very much not symmetrical.
39
socalnate1 3 days ago 1 reply      
Because people's opinions should totally be discounted based on the color of their skin! Privileged or not, I will never support those who try to fight that privilege with discrimination and censorship.
40
sakaloda 3 days ago 8 replies      
Diversity in tech will never work. You are going against reality if you think everyone has the same aptitude to be good at math/engineering. Look at the data below.

SAT Math by race [1] :

- Asian/Pacific Islander = 598

- White = 534

- Mexican American = 461

- Black = 429

SAT Math by gender [2] :

- Men = 537

- Women = 503

[1] https://nces.ed.gov/fastfacts/display.asp?id=171

[2] http://www.fairtest.org/sat-race-gender-gaps-increase

41
not_anit_woman 3 days ago 2 replies      
Then why are we not allowed to have Tall well-built white men clubs?
42
clifanatic 3 days ago 0 replies      
> They need to do their homework, instead of reaching out to the first tech woman speaker they can think of

... unless that homework points to a man being the most appropriate speaker?

43
onetwotree 3 days ago 0 replies      
It's really bullshit to ask people from disadvantaged groups to take on additional responsibilities and "educate" their less disadvantaged peers.

To give an example that is, in my opinion, much more disturbing, but qualitatively similar, my friend, who's a trans guy, was living in a sober house a while back. The manager demanded that he reveal his gender identity (he passes like a pro, beard and everything) to his conservative, homophobic roommates. The manager seemed to be well intentioned and wanted to "teach the roommates a less about diversity", but hey guess what, that's not my friend's fucking job. His "job" in sober living, simply put, is to stay sober. He did so with flying colors, but someone with a more tenuous recovery could easily have been pushed over the edge by this bs.

Similarly, it's not the job of women in tech to represent other women and try to force cultural changes. It's their job to be good at tech.

44
grb423 3 days ago 1 reply      
This is very refreshing. Diversity is a key to strength but seeing a woman present tech stuff is my preference at a tech conference. I was at a conference recently and I heard a neckbeard, seeing a woman warming up for the presentation we just sat down for, say something dismissive about another diversity talk. He actually left without sitting down. His loss. The talk was pure tech gold.

As the OP says, the diversity presentations should be left to the diversity professionals.

19
Sculpture of Housing Prices Ripping San Francisco Apart dougmccune.com
387 points by dougmccune  2 days ago   206 comments top 24
1
corysama 2 days ago 3 replies      
Very beautiful, poetic, relevant to a lot of the audience of HN, and props for detailing the tech behind it.

Meanwhile... the highest and lowest hex I can spot is 1265 / 457 = a ratio of 2.76 with both endpoints having relatively steep curves compared to the rest of the histogram. With the graph's Z0 set to 457 and Zscale set to an arbitrary ratio, the sculpture-graph conveys the impression that there is a discrepancy of 5 or more times between the top of hill on the bottom tier vs the nearby flat area of the top tier. When the reality is something more like 1234 / 810 = 1.5

2
tomschlick 2 days ago 17 replies      
As an outsider, the solution to the SF housing issues seem pretty simple... Vote out the NIMBY politicians and replace them with people who will change zoning laws to allow for more building. I'd assume almost no one likes the current situation unless they are a landlord.
3
bsurmanski 2 days ago 1 reply      
The one thing that is missing from the article is a top view of the sculpture. Although I believe it, I would like to see the sculpture align to the outline of the city.
4
notadoc 2 days ago 7 replies      
Why is SF so opposed to building up? Build more high rises filled with apartments and condos, there are plenty of areas that could be rezoned to accommodate high rise living.
5
kofejnik 1 day ago 3 replies      
People keep saying 'SF should allow for more development' as if it's a universal human right to live in SF; while current residents are being mean by not wanting highrises on their block. SF is nice, more people want to live there than there's space, hence, not everyone can afford it. Why would the current owners/residents want to change that? I really don't get it.
6
audleman 2 days ago 1 reply      
Why hasn't this guy donned all black and put this piece for sale in a swanky gallery for $1mil? It's the perfect blend of art and social commentary, and beautiful to boot!
8
bborud 1 day ago 1 reply      
This is indeed poetic: sculpture bemoaning gentrification realized through the kind of technology that is celebrated by bearded hipster millionaires that drive up housing prices.
9
Cenk 2 days ago 0 replies      
Wow, this looks amazing. Id love to see more sculptures based on data.
10
abritinthebay 2 days ago 0 replies      
Lovely piece! A little misleading with it's z-axis (given the range isn't very much and it seems exaggerated or logarithmic at least) but that's more artistic license, which is fair (and pretty).

I would be interested to see what it would look like with a more linear scale though.

11
hoprocker 2 days ago 0 replies      
Lovely idea, fascinating way to visualize demographics.

One comment, and not at all a criticism of the art: I'm not sure if relative property values between neighborhoods really describes how SF is being ripped apart over time; for that, it might be more accurate to graph, say, proportional difference in median rent over the past n years, which might more closely hew to contested neighborhoods (ie, Pacific Heights doesn't usually catch headlines for how much it's changed in the last 5 years).

12
etrautmann 2 days ago 0 replies      
I love the aesthetics and implementation - very cool. In reality, however, I think of the rip more between everything that's there and what's not pictured. The difference between the top and bottom home sales is not all that much, though the social divide between those who could afford to stay and those who can't even live here at all anymore is the underlying purpose.

I guess I don't buy that the home prices are dividing the landed gentry in Noe valley from pac heights as much as the rich elite from everyone else who's not buying homes in SF.

13
bunkydoo 2 days ago 1 reply      
If you don't like the house prices, move out of San Francisco
14
whordeley 2 days ago 1 reply      
WOW, this is utterly profound! Like that time someone put an empty McDonalds cup on the floor of a fine art museum and everyone stood around proclaiming its genius.
15
rubidium 2 days ago 0 replies      
"but if neighboring areas are too far from each other I allow them to split, tearing the city along its most severe economic divides."

Interesting that, from an artistic standpoint, the high delta's in nearby neighborhoods house price leads to a more interesting sculpture. The city may be 'torn', but in this case it's a good thing... assuming you think mixed neighborhoods are better than the alternative (gated communities and slums).

16
SocksCanClose 2 days ago 0 replies      
This is actually -- and I'm sure the artist will agree, albeit with a heavy heart -- a really great representation of investment opportunities.
17
smoyer 1 day ago 0 replies      
I'm in awe ... it's so organic looking (but then I guess homo sapiens are technically organic).

The only problem I see with sculpture for data visualization is when your boss asks for an updated report next week.

Thanks for sharing!

18
thinkpad20 2 days ago 0 replies      
The sculpture is really beautiful, but is the real issue that San Francisco is "ripping apart"? Even the lowest numbers on that chart ($457/square foot) are more than double the average price per square foot in Chicago ($219 according to trulia.com), for example. Is SF ripping apart, or simply becoming so gentrified as to be only livable by the wealthy?
19
thatsso1999 2 days ago 1 reply      
you should post this on thingiverse! you'd get a lot of appreciation there for the impressiveness of the print itself (as a fellow 3D printerer, daaaaaaaamn), and honestly it's so gorgeous I wouldn't be surprised if it got featured.
20
johnwheeler 2 days ago 0 replies      
when i first saw the sculpture, i was like, hmm...

then, when i read the article, i was like... WHOA!!!

21
unstatusthequo 2 days ago 1 reply      
Maybe if he sells a lot of these he can afford a million dollar fixer upper or teardown.
22
swampthinker 2 days ago 1 reply      
Seems like the site went down, and Google Cache isn't helping.
23
khoury 2 days ago 0 replies      
Cool
24
largote 2 days ago 2 replies      
We detached this subthread from https://news.ycombinator.com/item?id=12139125 and marked it off-topic.
20
Verizon nears deal to acquire Yahoo bloomberg.com
291 points by kshatrea  1 day ago   201 comments top 26
1
chollida1 1 day ago 9 replies      
It's now being reported that Verizon and Yahoo are exclusively negotiating. That's as close as anyone has gotten since Microsoft made an offer years ago.

Verizon is really doing a big transition with this acquisition and their AOL acquisition. They've acquired alot of valuable web space to put adds on/monetize. This is a probably good news for Yahoo employee's as Verizon then has a vested interest in keeping the company running and not splitting it up into pieces like a PE firm may be more inclined to do.

The one interesting thing I've heard is that Verizon isn't interested in Yahoo's patent portfolio, which means it could still be up for grabs.

Hopefully its bought by a Microsoft/Google consortium and very liberally cross licensed rather than a private equity firm who will look to more aggressively monetize it.

I also heard that Tim Armstrong, formerly of Google with Mayer will lead the combined AOL/Yahoo company, which means that Mayer probably isn't coming along as part of this deal. I think most people expected this.

If this ends up going through for the reported 3.5 billion, then Verizon has bought a significant portion of traffic on the web for roughly 8 billion (AOL was acquired for 4.4 Billion).

This could end up looking like a very good acquisition in a few years!

2
utopcell 1 day ago 1 reply      
It is deeply sad that there's even a chance that Yahoo will be sold for ~$5BN to Verizon, which bough AOL for $4.4BN, especially considering that AOL has 0.9% of the US web search traffic while Yahoo still has more than 10% and amazing agreements in place that allow it to show the best search results between Google and Bing. How is this maximizing value for shareholders exactly ?
3
colordrops 1 day ago 2 replies      
So they dumped the great Fios on insanely incompetent Frontier, and are picking up the failing Yahoo. What's their strategy here?
4
chkuendig 1 day ago 1 reply      
So finally that merger with Aol is happening after all...
5
hodder 1 day ago 9 replies      
Marissa Mayer is a terrible CEO. Unbelievable the amount of money she has been paid to flush the remainder of Yahoo down the drain with no strategy while employees suffer. Yahoo (net of BABA) is less than worthless.
6
kartD 1 day ago 2 replies      
Yahoo did serve one purpose, it was the exit system for a lot of almost successful startups. Hope that continues under Verizon, though I doubt it.

On Mayer, well eh she put her hand up and suffered. but then Yahoo was pretty much past saving. Hopefully, she's learned from her mistakes and moves to something more productive.

7
grandalf 1 day ago 1 reply      
I'm hoping Yahoo's Smart TV platform business is sold to someone who will make it awesome. It has potential and a pretty massive installed base.
8
finid 1 day ago 2 replies      
That's just wow!

Fifteen years or so from now, some company will likely be picking up Facebook on the cheap. By that time, Twitter will probably be history.

Google?

The wheel of life...

9
joobus 1 day ago 2 replies      
> Verizon is discussing a price close to $5 billion for Yahoos core Internet business...

Yahoo's current market cap is ~$37b

10
Floegipoky 1 day ago 0 replies      
Is Verizon Wireless still injecting tracking headers into cellular data traffic? If so, how will these new acquisitions affect the way that these headers can be used?
11
mark_l_watson 1 day ago 0 replies      
I have mixed feelings about this. I still use Yahoo once or twice a week to read their news feed, and I am a long time Verizon customer (although my wife and I are considering saving a lot of money each month and switch to Google Fi when we need new phones).

I would hope that a media company like Yahoo could have existed independently from a parent company.

12
CodeSheikh 1 day ago 0 replies      
A bad marriage is taking place.
13
IamFermat 1 day ago 1 reply      
Wow, i gotta say this is like Time Warner/AOL. Perhaps the internet is more mature now to make this work but Im not holding my breath. How will they reconcile the 2 adtech stacks which are huge messes in and off themselves.
14
mathattack 1 day ago 0 replies      
This certainly seems to make the most sense. It does seem like Verizon is chasing the past with this acquisition, going for Internet 1996 and Internet 2000 rather than the mobile future. (I don't buy Mayer's contention that Yahoo is now a mobile company)
15
socrates1998 1 day ago 1 reply      
This isn't good for consumers. Verizon is looking to add web assets with a longer term strategy, locking in people to their web space.

As in, two different data tiers, they family of businesses and everything else.

If there were true net neutrality, the government would block this purchase.

16
branchless 1 day ago 4 replies      
My requests:

1. don't touch yahoo finance

2. bring back pipes

17
porsupah 1 day ago 0 replies      
From what I've read, it appears unclear as to what Yahoo defines to be its "core business". Are there any details on what specifically Verizon is seeking to acquire, and whether Flickr is included?
18
macspoofing 1 day ago 1 reply      
For 3.5 billion (rumored price), I'm surprised Google wouldn't take a flyer on Yahoo.
19
drdeadringer 1 day ago 0 replies      
I wonder what Jerry Yang thinks about all this.

I still remember the "Go 3.0" marketing liner.

20
awqrre 1 day ago 0 replies      
Somewhat unrelated but: Since the Government is already tapping into wired and wireless networks without consent, it would be nice if they would take control of all ISPs... we would probably get better cellphone service at least.
21
0xmohit 1 day ago 2 replies      
Curious what is Yahoo! worth sans stakes in Alibaba and Yahoo! Japan?
22
tomjen3 1 day ago 4 replies      
What, other than some japenese websites and Alibaba does Yahoo own that is worth anything?
23
bogomipz 1 day ago 0 replies      
"Ma Bell, proud owners of both AOL and Yahoo." Clinton running for president. You would be forgiven for asking if it was the 1990s. It's hard for me to be excited by any of this. Those are three really dull companies.
24
ArtDev 1 day ago 1 reply      
Ugh, isn't there a better company out there to buy Yahoo?
25
ilostmykeys 1 day ago 1 reply      
Hopefully this will sink Verizon.
26
Esau 1 day ago 1 reply      
"Verizon sees a complimentary set of businesses that could find a home alongside its AOL properties."

This tells you everything you need to know, because AOL should have died years ago as well.

21
Native encryption added to ZFS on Linux github.com
276 points by turrini  3 days ago   139 comments top 15
1
mrsteveman1 3 days ago 1 reply      
Related pull request with more details & discussion

https://github.com/zfsonlinux/zfs/pull/4329

2
jlgaddis 3 days ago 4 replies      
Slightly off-topic but if anyone has any resources on performing a clean (preferably, Ubuntu or Arch) Linux "root on ZFS" installation, please share.

I followed the instructions for Ubuntu 16.04 on the github.com/zfsonlinux wiki [0] a while back and (encountered a few little issues along the way but) got it working, although I experienced some MAJOR performance problems so something wasn't quite right (exact same hardware is blazing fast when running FreeBSD). I can't imagine it was just "how things are" with regard to the current state of ZFS on Linux (or Ubuntu specifically) -- it was like someone hit the laptop's "pause" button a couple of times per minute.

[0]: https://github.com/zfsonlinux/zfs/wiki/Ubuntu-16.04-Root-on-...

3
ryao 3 days ago 0 replies      
This is not native encryption that was committed. It is just the kernel cryptography framework required for native encryption. Native encryption comes next.
4
makmanalp 3 days ago 9 replies      
What's the stability of ZFS on linux like these days? Anyone have any positive / negative experiences to share?
5
black_knight 3 days ago 5 replies      
Why would each file system need a "native encryption"? What gains are there fromt this?

Encryption seems it would be more cleanly implemented transparently underneath the file system level.

6
knz42 3 days ago 2 replies      
So now thanks to this there are two implementations of AES in the linux kernel. Who's responsible for ensuring they are both correct?
7
emaste 3 days ago 0 replies      
Note that the commit linked here is a port of the Illumos Crypto Framework (ICF), which is a dependency but is not the change that actually brings native encryption.
8
jvehent 3 days ago 1 reply      
That is a terrifying amount of crypto code. Has anyone audited this, or plan to?
9
nixomose 2 days ago 1 reply      
Is there really a need to make zfs your root volume? You can reinstall your root volume in a few minutes from a flash drive. What you really want is your home directory to be zfs, and just do all your work in your home directory.Saves all the grief of trying to make it your boot volume and it works just as well.
10
mei0Iesh 3 days ago 1 reply      
I think this is most useful for cross-compatibility. Right now if a client uses FreeBSD ZFS, and you need to mount it to access project files on your Linux desktop, you can't if they used encryption. But after this is standard, you should be able to mount the same ZFS filesystem anywhere.
11
mrsirduke 2 days ago 1 reply      
> We cannot use the Linux kernel's built in crypto api because it is only exported to GPL-licensed modules.

What a strange choice by the Linux kernel.

12
Infernal 3 days ago 1 reply      
As someone who just last night set up an Ubuntu 16.04 server with the intention of using ZFS, should I wait until this hits the Ubuntu repos? Is it possible to enable this encryption on an existing filesystem?
13
DashRattlesnake 3 days ago 1 reply      
So what's the relation of this to OpenZFS? Is this currently just for the Linux port, and not yet pulled into OpenZFS for other platforms?
14
lmm 2 days ago 0 replies      
So does ZFS on FreeBSD support native encryption? Can I switch my existing pool?
15
zxv 3 days ago 0 replies      
native encryption with AES-NI support sounds awesome.When is likely to make it into ubuntu repos?
22
Building an Open Core Company: Interview with GitLab's CEO gitlab.com
287 points by Siecje  3 days ago   135 comments top 17
1
mrmondo 3 days ago 1 reply      
Hands down the best open source community with a commercial offering I've ever dealt with. The company really clearly gives a $&@"! about what people feel is right and wrong with the product components and has always heard me out - even if people disagreed with me (god forbid I might be wrong about something!).

I truly have grown to love GitLab and the team that develops and supports it, it's probably the fastest moving OSS project I've ever seen, and every release is clearly getting better (although there are some UI changes recently that bug me, but that might just be me).

Thank you yet again everyone involved with gitlab and it's varying components (especially gitlab-CI) both internally within the company and those contributing either code or conversations from around the world.

Gitlab has enabled us @infoxchange to move faster than ever before, with a low TCO and a responsive support team.

I salute you!

2
SEJeff 3 days ago 2 replies      
Gitlab is great stuff, but as a business model for open source, I always considered open core as somewhat toxic. When your community is trying to add features in your commercial implementation, you're "competing with yourself", which in this context, is actually a bad thing. I saw it wreck havoc in the ZenOSS community years ago.

David Neary does a most excellent post on why:https://blogs.gnome.org/bolsh/2010/07/19/rotten-to-the-open-...

3
pritianka 3 days ago 1 reply      
I've been impressed with the GitLab founder in every interaction I've had with him. He treats people well and makes an effort to connect with every dev/person interested in his product. He's building a great culture.
4
willow9886 3 days ago 3 replies      
In the article Sid mentions that "according to research, by 2019 80 cents of every dollar spent on software will go to on premises or single tenant software."

Does anyone know which research he's quoting?

5
OJFord 3 days ago 2 replies      
The interviewer asks if the community edition is a good funnel into the enterprise edition, but doesn't ask what I see as the converse; which I'd hoped for:

"Does making the source available restrict pricing options, or completely lose would-be users - or would self-hosters never be customers anyway?"

This has been bugging me and holding me off a project that I'd like to be open-source; I just can't help wondering what stops someone from thinking "well I'm just going to host your source code, and undercut you". I'm all for a competitive market - but not if I do all the work and we only compete on how to price it!

6
vonnik 3 days ago 1 reply      
> Open source is wind in your back.

So true. Open source lets small engineering teams punch above their weight. Thousands of extra eye balls on the code for QA make it so much better. And the project-user feedback loop is fast.

7
Sylos 3 days ago 1 reply      
> [Without telemetry] Do you have to rely on users asking for features then?

How does telemetry affect that? I can see that you would maybe prioritize work on improving different features depending on what gets used most or what crashes most, but how should telemetry help you decide what features to include?

8
eevilspock 3 days ago 0 replies      
The true core of GitHub (and GitLab) is Git itself. So both are open core where it really matters. Because both GitHub's and GitLab's essential operations are all done over the Git core, neither code and code history hosted on either system are locked in.

Where GitHub has slightly more lock-in and is less open than GitLab is its Issues system; One can only extract the data (via API) for import into an alternative system, whereas with GitLab you could run your own copy of the open source code (but you'd still have to do the data export-import as far as I can tell).

(copied my comment from a duplicate thread)

9
mperham 3 days ago 1 reply      
Great interview with a lot of good advice for anyone starting their own OSS-based business.

Making EE "open source" (not "Open Source") is a brave move but I think anyone who's done this work before knows that having the source available for viewing doesn't really matter much to actual customers with money - a packaged solution with long-term support and legal compliance are their main concerns.

10
iamcreasy 3 days ago 1 reply      
I was excited to read how open source philosophy is ingrained into Gitlab and how it helps them grow but the article rather skims over it.
11
rburhum 3 days ago 2 replies      
I was going to try it out after many months... and it is down :( https://twitter.com/gitlabstatus/status/755878196634087424
12
Myrmornis 3 days ago 1 reply      
Lots of companies use Jira for issues and Github/Gitlab for code. It's a painful divide. Do Gitlab and/or Github have plans to rescue such companies from that situation?
13
Siecje 3 days ago 1 reply      
Any chance GitLab will add Mercurial support?
14
grabcocque 3 days ago 6 replies      
Here's the open source definition:

https://opensource.org/osd-annotated

Gitlab EE, which is how Gitlab makes its money, comes nowhere close to meeting that. It's pretty much straight up corporate dishonesty to pretend to be an open source company when you're following a proprietary software model to make money.

You're no more an "Open Source Company" than Apple or Microsoft, who also have some open source projects separate from their main revenue streams.

15
NickBusey 3 days ago 2 replies      
Lot's of griping in here about whether or not GitLab is really 'open source'. Typical HN vitriol. I think GitLab is doing fantastic work pushing forward open source philosophies with real world execution. They're putting a lot of their money where their mouth is. Hell, you can clone their ENTIRE corporate website. On top of that, the product they offer (yes, their free, Open Source community edition) is _fantastic_ and continues to improve with each release. I am constantly impressed with how this company operates, and am always trying to roll in more of that to my own company (their Handbook for example is fantastic.)
16
paxcoder 3 days ago 3 replies      
In the interest of objectivity, I'd change the title to read "Open Core" instead of "Open Source"
17
grabcocque 3 days ago 2 replies      
Gitlab are not an open source company. Their primary revenue generation stream is proprietary software.
23
Programming Language Rankings: June 2016 redmonk.com
249 points by adamnemecek  3 days ago   183 comments top 24
1
lkrubner 3 days ago 14 replies      
Since I've been interested in how the Functional paradigm might help us developers deal with concurrency, there are 2 items here that really strike me:

1.) Clojure fell back

2.) Elixir, though small, is still moving forward

I've been a fan of Clojure since 2009. It grew rapidly for a few years, from 2009 till at least 2014. It has stalled out. This makes me sad because I've loved working with it and I think it a great community, in most ways. But it is true that the community has been unable to answer some of the criticisms leveled against it. The reliance on Emacs has meant it is not easy for beginners. On the other side, the elite (the purists?) have wanted to see Clojure be more like Scheme, or more like Racket, or more like Haskell. Clojure has offered up many interesting ideas, but perhaps hasn't quite built a complete eco-system that makes it broadly appealing.

Elixir, though very small, still seems to be moving forward, and perhaps has a chance to answer some of the demands that people made of Clojure (more pure? easier syntax?). Maybe Elixir is the language that make the Functional style the dominant style in computer programming?

2
flohofwoe 2 days ago 1 reply      
When looking at github-provided data it doesn't make sense to differ between C++, C and Objective-C/C++ since the detection for these languages is still completely broken. For instance this project doesn't have a single line of Objective-C in it, yet github says there's 9% Obj-C in it: https://github.com/floooh/yakc. Similar oddities in my other C++ projects.
3
acangiano 3 days ago 5 replies      
Not perfect, but it feels significantly more realistic than the TIOBE index (http://www.tiobe.com/tiobe_index) or PYPL: https://pypl.github.io/PYPL.html.

The trouble is that I make this statement from a place of intuition, conscious of my own biases, so I could be wrong.

4
machbio 2 days ago 1 reply      
"GitHub language rankings are based on raw lines of code, which means that repositories written in a given language that include a greater amount of code in a second language (e.g. JavaScript) will be read as the latter rather than the former."

This has huge impact on the ranking - it does not seem right to me..

5
fermigier 2 days ago 0 replies      
AFAICT (based on years of poking around various projects, and based on the data they provide on my own projects), the data provided by GitHub are largely broken.

Making analyses and comments on these data is futile, as I have already argued directly (and with vigour on my side) with several Redmonk employees in the past. This didn't end well, since I'm pretty passionate about these things in particular, and "the truth" in general (vs. "opinions").

I'm sure real scientists could do some interesting work on this subject, but Redmonk's methodology is anything but scientific, and in consequence, their results are just content marketing, not something that you should base business or engineering decisions upon.

6
jballanc 3 days ago 3 replies      
Hmm...I wish they had taken a bit more time to dig into the situation with Julia than simply to note it moved from position 51 to 52 on the ranking (is that even a significant shift?). Having just come from giving a presentation at this year's JuliaCon, I am more optimistic about Julia's future than at any point in the past. I think what we're seeing here is that Julia is making a concerted push toward v1.0. It was announced at JuliaCon 2016 that they are targeting JuliaCon 2017 for the v1.0 release. It should be expected, then, that the break-neck pace of innovation has taken a back seat to a focus on stability, but based on what is already on the roadmap for Julia v2.0, I don't see Julia moving anywhere but up.

(Also, I think the comparison to CoffeeScript is rather unfortunate. In my mind, CoffeeScript was primarily addressing shortcomings in JS that JS has, since, done quite a bit to rectify. Julia is not trying to be a better "X", so there's no "X" to steal away its momentum.)

7
vegabook 3 days ago 1 reply      
I love this ranking because while all rankings have flaws, this at least is very transparent on how it works, it's parsimonious, it is entirely numerical, easily reproducible, and it gives us at least a small amount of cross validation via two (mostly) orthogonal variables. Now all we need to do is to animate the plot over time.....
8
SwellJoe 2 days ago 5 replies      
It's interesting how stubbornly reliable old things stick around, despite popular negative opinion of those reliable old things.
9
flukus 3 days ago 1 reply      
So apart from swift and go, things have been extremely static for ~5 years?

I'm guessing that VB uptick was from VB devs finally discovering git?

10
na85 3 days ago 1 reply      
Misleading title. It's entirely github and SO-centric and as such misrepresents C and probably C++ as well, since those languages have scads of code elsewhere.
11
cm2187 2 days ago 1 reply      
Unrelated but still annoying. This website is a prime example of responsive design gone wrong. Trying to read it on an iphone, but the chart with the trends overflows out of the screen, and the website locked the ability to zoom out.

In this example as often, the website would be more readable without responsive design.

12
gavinpc 2 days ago 1 reply      
TeX has its own stackexchange site [0], and so does Emacs [1].

While both TeX and Emacs Lisp questions are still treated on StackOverflow, it probably explains why TeX in particular appears to be such an outlier. (It's sure not because people aren't having trouble with it. The stackexchange site is indispensable.)

[0] http://tex.stackexchange.com/

[1] http://emacs.stackexchange.com/

13
rezashirazian 2 days ago 2 replies      
Haskell is more popular than Swift? That's surprising.
14
insulanian 2 days ago 1 reply      
TypeScript is really getting traction. I think it will explode when AngularJS 2.0 is finally released.
15
themihai 2 days ago 3 replies      
I was hopping for more from Rust :|
16
asdfzxc 3 days ago 2 replies      
I wonder why Hardware Description Languages don't make the cut here?
17
jmnicolas 2 days ago 3 replies      
No mention of Dart, is it dead ?
18
eevilspock 2 days ago 0 replies      
> There is no movement, in fact, among languages ranked within our Top 10.

> the difficulty of growth is proportional to the rankings themselves as one rises, so does the other.

OF COURSE this will be the case when your analysis is looking at total cumulative usage instead of current usage (i.e. deltas in cumulative usage)!

For example, say language X has been around 20 years, and language Y 5. Language Y could be 10 times more popular than language X today, but if the analysis is counting 20 years of accumulated code in X, X will easily rank higher.

I hunted for their methodology, and found none. Nothing they say indicates they are doing delta analysis. Since such analysis would not be trivial and be a lot more expensive, I'm pretty sure they'd mention it if they were doing it.

To corroborate my criticism, the TIOBE Index, which measures signals where accumulation is less a factor, shows much more movement over time, with languages rising and falling as one would expect: http://www.tiobe.com/tiobe_index.

19
0xmohit 2 days ago 1 reply      
Wish it was possible to zoom http://redmonk.com/hist-rankings-full.html
20
TickleSteve 2 days ago 2 replies      
Since when was "Arduino" a language??

Its C++, just hidden.

21
IshKebab 2 days ago 0 replies      
It would be interesting to compare the number of stars of different language projects on github, rather than the number of projects.
22
dingleberry 1 day ago 0 replies      
it seems that the interesting languages fall more on github side

the popular languages seems to fall more on stack overflow side

those on the line are vanillas

23
habosa 2 days ago 1 reply      
Somehow there are more Github projects in FORTRAN than Elixir. This blows my mind.
24
andyidsinga 2 days ago 0 replies      
having only looked at the graph, it seems like this is could also be interpreted as discussion rate vs implementation(usage) rate (??)
24
Practical Guide to Bare Metal C++ gitbooks.io
282 points by adamnemecek  2 days ago   80 comments top 7
1
vvanders 2 days ago 6 replies      
Nice to see some concrete examples of where you don't want exceptions, rtti or dynamic memory.

Quite a few developers believe that if you don't have these features you aren't writing "real C++" but it's common practice in quite a few industries where size/performance is critical.

2
Taniwha 2 days ago 3 replies      
He's talking about real time code here and completely missing one of the main reasons why you have to use great care using C++, and even parts of the C libraries ... all those hidden mutexes in new/malloc and other parts of the standard libraries (stdio too)

I'm sure we all know what happens when your ISR quietly does a new while some other part of your code is holding one of the locks deep in malloc.

But far more important is avoiding priority inversions when a low priority thread is holding a lock in a library somewhere, they result in high priority threads missing real-time deadlines - the sort of heisenbugs that are pretty impossible to find .... and are best to avoid by design.

3
Kenji 2 days ago 3 replies      
The usage of single throw statement in the source code will result in more than 120KB of extra binary code in the final binary image. Just try it yourself with your compiler and see the difference in size of the produced binary images.

What?? 120KB? I have to try that out right away.

Edit: That turned out to be completely wrong. Simple program without throw: 71'218 bytes. With throw (one additional line that throws a plain integer): 71'814 bytes. GCC 4.8.1 on Windows. Would have been surprised if that was true.

Edit2: Okay, my bad, I should have linked statically. The statement sounded so absolute, but of course it is conditioned on the bare metal environment which is the subject of this article. Thanks for pointing out where I went wrong.

4
devbent 2 days ago 2 replies      
If you go down to micro-controller levels, you often times are stuck with C++ 2003 and a vendor specific compiler, which means you will lack many of the niceties in the article.

I agree with him that removing the entire standard library is needed. Of course you then need to copy an implementation of printf from somewhere, and likely set it up to only work in your debug builds. Then you quickly figure out that the standard library has a lot of things you didn't even realize you depended on. Math functions typically pop up next, you'll likely end up using a vendor library (by which I mean ARM's), but if you are doing a bunch of math heavy work, and even more so if your chip has some limited FP capabilities (assuming it has an FPU at all), you may also find the need to re-implement some functions based on your performance needs.

Embedded is fun. :)

All that said, I would kill for access to proper lambdas.

5
amaks 2 days ago 3 replies      
Risking being down voted, I doubt that c++ is the right language for bare metal development at all with its constraints like exceptions and RTTI. Rust seems to be the modern and safer version of C that is more appropriate for bare metal development.
6
Animats 2 days ago 0 replies      
The Arduino development environment is really GCC C++ with a platform-specific library. All the compile-time C++ features work. Some features that need run-time support may not link.
7
_pmf_ 1 day ago 0 replies      
Template oriented programming is a tremendously useful technique for working with overhead-less abstractions for embedded software, but C++ makes it so hard. Most large users of template oriented development (automotive) use external code generators for what should ideally be supported by the language itself.
25
Approaching Almost Any Machine Learning Problem kaggle.com
310 points by Anon84  1 day ago   32 comments top 9
1
glial 1 day ago 5 replies      
Interesting post, but there are several errors, or at least suggestions that don't make sense to me:

> Doing so will result in very good evaluation scores and make the user happy but instead he/she will be building a useless model with very high overfitting.

Testing on your training data doesn't in-and-of-itself lead to overfitting, but it will hide overfitting if it does exist (and is a terrible practice for that reason).

> At this stage only models you should go for should be ensemble tree based models.

Not sure why this should be the case. Many ensemble models are very memory hungry & slow, Random Forests being a good example. They are flexible and have minimal assumptions, sure, but that doesn't mean you shouldn't try any other modeling technique, especially if you have domain knowledge.

> We cannot apply linear models to the above features since they are not normalized.

Not true at all. You won't be able to meaningfully compare coefficient magnitudes, but you can certainly apply linear models.

> These normalization methods work only on dense features and dont give very good results if applied on sparse features.

Since normalization is done feature-by-feature, there is no reason why this should be true.

> generally PCA is used decompose the data

If your latent features add linearly, ok, but do they? Is it meaningful to have negative values for your features? If not, consider using something like sparse non-negative matrix factorization.

Lastly, using this approach means you have a HUGE model & parameter space to search through. Because of this you will need a ton of data to get meaninigful results.

He seems to be treating machine learning methods as a bag of tricks. That's ok so far as it goes, but in my experience it's much more valuable to try and understand your data, and the process that generates it, and then build a model that tries to reflect that data generation process.

2
vonnik 1 day ago 3 replies      
This is a great post as far as it goes. While Abhishek mentions Keras for neural networks (and Keras is a great Python library), he doesn't really go into the cases where deep neural networks are more useful than other algorithms, and how that changes a data scientist's workflow.

DNNs are really well suited to unstructured data, which isn't the kind he highlights. One reason for that is because they automatically extract features from data using optimization algorithms like stochastic gradient descent with backpropagation. What that means is they bypass the arduous process of feature engineering. They help you get around that chokepoint, so that you can deal with unstructured blobs of pixels or blobs of text.

Because unstructured data is most of the data in the world, and because DNNs excel at modeling it, they have proven to be some of the most useful and accurate algorithms we have for many problems.

Here's an overview of DNNs that goes into a bit more depth:

http://deeplearning4j.org/neuralnet-overview.html

3
lqdc13 1 day ago 0 replies      
Approaching (Almost) Any Machine Learning Classification Problem.

If you are doing sequence labeling, learning something about the data, tackling partially unlabeled data or time-varying data, you generally have to take a different approach.

4
slv77 23 hours ago 1 reply      
Throwing out a couple of stupid questions since we're talking about rules-of-thumb..

1) How much does feeding irrelevant features have on a model? For example if you added several columns of normally distributed random numbers?

2) How much impact does having several highly correlelated feature have on a model?

3) If you had limited time and budgets would it be better spent on cleaning data (removing bad labels, noise in data); feature engineering (relevant features) or feature selection (removing irrelevant or redundant feature)?

5
pesfandiar 1 day ago 2 replies      
It's a very insightful article about nitty-gritty details of working on ML problems. However as an outsider, I can't decide if some of very specific statements without any reasoning (e.g. good range of value for parameters) are highly valuable wisdom coming from years of experience, or merely overfitted patterns that he's adopted in his own realm.
6
danvayn 1 day ago 0 replies      
Awesome post and site, my only complaint about this blog is that you'd think the top left would be a link and 'No Free Hunch' would not be one, but in fact it's the opposite..
7
cmdrfred 1 day ago 0 replies      
I've been looking for a place to begin with machine learning, thanks.
8
elgabogringo 1 day ago 1 reply      
Awesome post. Too hungover to read all that today though. Definitely Saturday morning over a cup of coffee - assuming I'm not in the same shape then.
9
greenpizza13 1 day ago 1 reply      
Great post, but how am I supposed to find it credible when he uses quotation marks for emphasis?

> The splitting of data into training and validation sets must be done according to labels. In case of any kind of classification problem, use stratified splitting. In python, you can do this using scikit-learn very easily.

26
China is challenging the idea that censorship thwarts online innovation washingtonpost.com
213 points by JumpCrisscross  2 days ago   222 comments top 49
1
stickfigure 2 days ago 16 replies      
Article provides only two specific examples:

You go on Facebook and you cant even buy anything, but on WeChat and Weibo you can buy anything you see

A more recent trend: live-streaming sites where people pay real money to reward performers with virtual gifts. (You sang beautifully, heres a digital Lamborghini, dear.)

I might grant the first (with reservations), but the second is laughable (and was tried without success here in the early days of social networking). I can't help but think this article should be titled "Chinese internet companies flourish in Chinese market". Well duh - even aside from the "virtual protectionism" of the GFW, Chinese companies are more likely to understand the preferences of Chinese consumers than non-Chinese companies. This is important, but it isn't exactly tech innovation.

How many HN startups are localizing to Mandarin? Language alone provides a massive market barrier that leaves opportunity for regional companies to thrive. But that barrier works both ways - what Chinese internet companies are thriving in the world marketplace? The best example I can think of is Alibaba, but that is strongly tied to Chinese manufacturing. Is anyone using WeChat or Weibo outside mainland China?

I personally am very excited and encouraged by the emergence of China as a modern, educated populace. Great things will come from bringing another 1 billion humans online and contributing to the world marketplace - tech is one of the great positive-sum games. I'm looking forward to it! But this article is a pretty poor illustration.

Also, as a Norteamericano, I'm deeply offended by statements like "America want's to believe China can't..." - we're not all xenophobic Trump supporters, not even most of us.

2
acd 2 days ago 5 replies      
China may follow a similar development history path to Japan.First the Japanese copied the west and made cheap bad quality items.Then the Japanese innovated and made very high quality items.

http://asq.org/learn-about-quality/history-of-quality/overvi...

"In the 1950s and 1960s, Japanese goods were synonymous with cheapness and low quality, but over time their quality initiatives began to be successful, with Japan achieving very high levels of quality in products from the 1970s onward."source: https://en.wikipedia.org/wiki/Quality_management

Top500 super computer list, the fastest super computer now has a Chinese CPU Sunway SW26010 which has 260 cores.https://www.top500.org/lists/2016/06/https://en.wikipedia.org/wiki/SW26010

List of inventionshttps://en.wikipedia.org/wiki/List_of_Chinese_inventions

3
spodek 2 days ago 3 replies      
Where there are factories and competition, innovation will happen. When we started manufacturing in China, we had experience building and they didn't so we had to show them how to do everything. Now they manufacture without our oversight, have the relationships with suppliers, and figure out how to do things faster, cheaper, and higher quality because it directly benefits them.

Meanwhile, we don't have the experience building things that they do with their equipment and people.

The only problem is believing that other people innovating hurts us. It doesn't.

4
halfelf 2 days ago 0 replies      
Though as a Chinese programmer, I have no doubt we will achieve some great innovation here, the author of this report apparently doesn't know internet industry well, for choosing a bad example. Xu Dandan himself is just a joke here, and himself is widely considered as a bragger.
5
Unklejoe 2 days ago 3 replies      
Perhaps the reason there's a stigma of China not being able to innovate is because they're so widely known for blatantly copying existing products and designs (and selling them for a much lower price). In some cases, its a direct rip-off of an existing product (for example, search AliExpress for TIAL Wastegate), and in other cases, its a similar, but often cheaper (in both cost and quality) design.

The problem is that people love to generalize, and the fact of the matter is that when many people think of Chinese products, they think of cheap knock-offs being sold on eBay. Obviously, this is a pretty bad generalization considering many of the legitimate products we use every day were made in China, but were talking about perception amongst the general population. [Criticizes China for making knock-offs while typing on an iPhone]

The point that needs to be made is that making cheap knock-offs does not preclude them from innovating. There are a lot of people there, and Im sure the entire country isnt composed of mindless assembly drones.

Its a shame that this stigma exists, and it will take a while before it completely fades away.

I realize that some people might be offended by this (and my use of cheap knock-offs), but if you dont believe me, just go ask around for yourself.

6
jayadevan 2 days ago 3 replies      
I was at the Alipay headquarters a few weeks ago. What caught my eye was a large IBM machine kept outside the office with lots of signatures on them. When I asked them what it was, they said this was the last piece of American technology the company used and they'd kept it as a trophy. They've replaced every thing else in the company with Chinese tech. That's quite something for a company which processes millions of transactions in a day.
7
pzh 2 days ago 2 replies      
When I saw the word innovation, I was expecting something a bit more than web/mobile chat and e-commerce apps. Not saying that the Chinese startup scene is in any way worse than SV, but I think we've put the bar for what constitutes innovation way too low everywhere...
8
hourislate 2 days ago 1 reply      
In Marc Goodman's book "Future Crimes" he discusses the largest transfer of Human wealth to ever occur.

Through China's hacking efforts they have stolen Trillions of Dollars of US Tax Payer money that has gone into research and development of anything the Government has done including the F 35 fighter development.

They have also attacked US Industry and stolen trade secrets and software. Recipes for Carbon Steel, Stainless Steel, etc. Decades of R&D and 100's of millions of dollars gone. Chinese State Owned Sinovel Wind Group stole AMSC's computer code that was developed to run power generating Wind Mills. It cost the company almost a 1 billion dollars a year in lost revenue and all the money it took to develop it.

Here is an article regarding the F 35 Program.

http://news.nationalpost.com/news/canada/canadian-politics/o...

Is it really innovation when all you do is steal and copy from the west?

9
antoniuschan99 2 days ago 2 replies      
I think we need to figure out how to bring manufacturing closer to North America. For example, getting a prototype batch of PCB's from Advanced Circuits cost ~$400 for 5-10 pieces. Whereas it costs ~$30 from Seeed Studio.

The quality is obviously better from Advanced Circuits, and it doesn't take a whole month as it would from Seeed (I've heard good things about OSH Park and they're US Based).

The quality of Chinese products is improving, and the parallels with the history of Japanese tech are strikingly similar.

In terms of software expertise, I think the talent is still in North America. The hardware expertise though is somewhat non-existent. I'm surprised there's no city like Shenzen here.

10
Retric 2 days ago 1 reply      
The problem with innovation in China is the same problem it has in most areas namely corruption. In an actual free market innovation has huge dividends, but when success is arbitrary and often based on outside connections it's far less useful. That's not to say the people are not innovative and there is plenty of innovation in smaller more competitive markets.

In the end they will continue to innovate, but as long as it's fighting both the government and social norms things are going to be bottleneck.

11
whack 2 days ago 0 replies      
> Theres this strange belief that you cant build a mobile app if you dont know the truth about what happened in Tiananmen Square, said Kaiser Kuo, who recently stepped down as head of international communications for Baidu, one of Chinas leading tech companies, and hosts Sinica, a popular podcast. Trouble is, its not true.

There's something that's hilarious, delightful, and depressing about the above statement, all at the same time.

12
Zenfinch 2 days ago 1 reply      
Chinese companies are certainly innovating and they are picking up R&D teams anywhere on the planet.

For example you have Andrew Ng as Chief Scientist at Baidu Research in Silicon Valley even though Baidu pretty much only operates solely in the Chinese market.

13
chvid 2 days ago 4 replies      
Blocking Facebook and Google may have been brutalist censorship at first. But as a policy for encouraging innovation it is pure genius.

Will be interesting to see when/if some of the Chinese social networks become popular outside China.

14
keenerd 2 days ago 2 replies      
Let's play write the article that should have been written!

Here's my favorite example of Chinese innovation: the ESP8266. It is original hardware and 1/10th the price of the equivalent offering from TI (their CC3000), with more speed, more features and better reliability.

15
kosmic_k 2 days ago 1 reply      
Of course Chinese entrepreneurs can innovate. That being said, the CCP has been making it more difficult. As of this month every single mobile game needs government approval at cost to the developer.
16
fumplethumb 2 days ago 0 replies      
> The United States wants to believe that the scourge of censorship thwarts online innovation, but China is challenging the idea in ways that frighten and confound.

As an American, I do believe that a culture of censorship and phenomena like the Great Firewall stifle an economy's capacity to innovate. That's not to say that innovation is impossible in such an environment, just more difficult.

> It doesnt matter how the car is capable of traveling. Once it gets on the highway, you can imagine what the end result will be, he said.

> The implication is that Chinas government is happy to have companies build shiny, fast things as long as regulators can put up roadblocks as they please. So far, theyve mostly targeted foreign firms.

Exactly, so far! What happens when the state determines that your blooming startup threatens their agenda?

17
Nokinside 2 days ago 0 replies      
Kleiner Perkins has currently offices in four cities: Menlo Park, San Francisco, Beijing and Shanghai http://www.kpcb.com/china

China has markets, opportunities and innovative people. Things are just all somewhat different.

18
Symmetry 2 days ago 0 replies      
In general a nation's ability to copy technology is a very good indication of its ability to innovate. Back in the day the USA copied British technology and art shamelessly. Later Japan was known for copying technology. Both went on to be great innovators.

Copying well, figuring out what to copy and what isn't needed, is hard.

19
GFK_of_xmaspast 2 days ago 0 replies      
I've been hearing people say "the Chinese can't innovate" for many many years now, and I've seen "the Japanese can't innovate" in older sources, and once I saw a reprinted claim from a Brit in the 19th century that "the Germans can't innovate".
20
jokoon 2 days ago 0 replies      
I think that's why Chinese leadership want to separate themselves from the west, because they don't want western-style capitalism to influence their country, and end up negotiating economic advantages with developed countries. That's what state-capitalism is all about.

I think they want to have both the benefits of isolationism, and the benefits of exports.

Ultimately it's very easy to play with anti-US views. Much easier to blame the US than to blame the chinese government. For example the vietnam war, iraq war or japan nukes might sound much worse than the tiananmen square for a chinese.

But in the long run, a country like china should be able to catch up with how late they have developed. I believe they have excellent human capital. Not sure if they will be able to compete in very high tech fields, but they might one day.

21
awt 2 days ago 0 replies      
America's service economy where we "own" IP and everyone sells each other backrubs is a joke.
22
throwanem 2 days ago 2 replies      
tl;dr Internet fails to turn China into America; Americans frightened, baffled.
23
jsonmez 2 days ago 0 replies      
I'm here in Beijing and I just experienced this first-hand. Amazing.

Came over here for the launch of the Chinese translation of my book, Soft Skills, and I am amazed by the tech community here.

24
Vampires123432 2 days ago 1 reply      
The article fails to touch on what I regard the major barrier to Chinese innovation. Specifically, Chinese cultural norms are prohibitive towards the creative enterprise. They just don't think through a problem the way hippie American kids do; and why should they? The creative process NP-complete.

Chinese society is ingrained with the "get ahead by besting your peers". That sort of mentality is not conducive to creativity. It is stifling. It forces one to take the shortest path all the time without any opportunity for making constructive mistakes.

And I don't buy the mentality of Chinese kids coming to US universities and solidifying their critical thinking skills. They congregate into throngs of Chinese students which perpetuate the Asian version of the "keeping up with the jones'" lifestyle. Trendy t-shirts and designer jeans, flocking to the basketball courts to ape Yao and JLin, nary an independent thought lest they should offend the echo chamber. "Why do we do what we do? Because we are told. Sure I can recite that the uninspected life is not worth living... But do I understand it?"

Furthermore, you see in Chinese pockets a lack of respect for their fellows should one fall by the wayside, whether it's the victims own fault or not. Those with weakness (or being different) are cast aside like lepers. It disgusts me. And to my original point, it stifles creativity.

The Japanese at least have Murakami. What celebration is there in China for revolt and indignation?

The Chinese competing with American innovation? Get real, and go F yourself (the preceding inserted as a provocative demonstration of American anti-normism, vote me down, square peg).

25
jakub_g 2 days ago 3 replies      
I get a big popup "To keep reading, please enter your email address", any simple way to remove it other than removing nodes from DOM [1] and manually removing "drawbridge-up" CSS class from `<html>`?

[1] using https://addons.mozilla.org/en-US/firefox/addon/hack-the-web/

26
waterphone 2 days ago 0 replies      
Similarly, with regards to the common western belief of poor manufacturing quality from Chinese factories, I find that this is primarily true when western companies push the Chinese manufacturers to cut costs. In contrast, high-end Chinese-designed and -built products that I've bought have been excellent. Which is no surprise, and yet so many people associate "Made in China" with poor quality.
27
infinity0 2 days ago 0 replies      
It's insanely ignorant to even hold such views in the first place. It's not as if American media doesn't effectively self-censor already.
28
pipio21 2 days ago 5 replies      
Chinese have not innovated yet in anything in the present day. The examples given are just taking individual parts, and copying them all in one system.

The main difference is that in China there are not software and business patents. There are patents but in practice you can do whatever you wish. Also the Government can and actually does whatever pleases them. Someone knocks your door and tells you: We have decided you have to teach those guys what you do so they can do the same you do.

It is not innovation, but forced collaboration. Almost any technology is bought from outside, they will offer 10, 20 times(or whatever necessary) more salary(than what they actually earn) to key workers outside China with tech experience in order to go China and train Chinese replicating products in China. Once in China they will teach other companies to replicate the technology. This process is totally natural and periodic in Middle Land.

It has nothing to do with innovation. In fact the System makes it really hard to really innovate in China: Only traditional products like Silk, Porcelain and tea are protected from counterfeits. The rights of the individual is always less than the collective. The education teaches you submission and Confucian values, not risk taking and disruption.

Never forget that disruption and innovation is synonymous with change, and the first thing they will want to change is their Government.

Their system have worked fine for them until now, coming from total poverty(most people in China actually remembers the famine that killed millions, you see people wasting food as a symbol that they are "rich" enough to waste it),the advances have been impressive, but is not a model for developed countries.

29
perseusprime11 2 days ago 0 replies      
I've also heard Foxconn drives a lot of what is possible in the future versions of iPhones and advises Apple on component level innovations.
30
toomanythings4 2 days ago 0 replies      
>whats more revealing is how Chinese firms have taken the best tech and adapted it.

Only felt like skimming the article but it seems that quote is at its base. There's a difference between "adapting" other people's work and "innovation". There is also a difference between manufacturing products and creating the original idea for those products and making them.

31
chrisper 2 days ago 0 replies      
Interestingly, the German part of Seafile just stopped working with the Chinese part because they got greedy and did other weird things they did not like.

Here is the source: https://seafile.de/en/about-the-future-of-seafile/

32
criddell 2 days ago 0 replies      
Big Chinese tech firms may not always be the first to come up with a good idea, but they certainly are able to refine an idea (which is often innovative). They also will run with bad ideas.

There's a story on TechDirt today [1] showing how the large Chinese firms have been putting together massive patent portfolios and have started litigating... in East Texas.

[1]: https://www.techdirt.com/articles/20160718/06573135006/just-...

33
msl09 2 days ago 0 replies      
Doesn't Chinese techies circumvent the great firewall of China constantly? Also, I think there was an article recently about how the government officials turn a blind eye at breaches that didn't involve political matters.

Somewhat on topic: http://time.com/4283248/china-great-firewall-fang-binxing-ce...

34
tmaly 2 days ago 0 replies      
I use WeChat along with WhatApp. Mainly I use it to communicate with some foreign friends. I like seeing the different features it has, it gives me ideas of what is popular in other regions.

Its nice to have that insight as to what works outside of the US market, I hope my side project could work in China.

35
known 2 days ago 0 replies      
Recently Jack Ma claimed that China is delivering "low cost- better quality" products than USA;
36
mtgx 2 days ago 0 replies      
> Theres this strange belief that you cant build a mobile app if you dont know the truth about what happened in Tiananmen Square,

I think people want to find out what happened in the Tiananmen Square for other reasons than "wanting to build a mobile app".

37
ryanmarsh 2 days ago 0 replies      
Innovation. I don't think it means what you think it means.

Seriously this is such an over used and hence watered down word. Every new thing isn't and innovation, novel maybe but not innovation.

The assembly line was an innovation. Ride sharing was an innovation. Pokmon go was a novel use of tech.

38
LiweiZ 2 days ago 0 replies      
Innovating takes time. There is way much less room for things that need more time to happen in China. The operation there is simply stimulated by different direct drivers. Now, with this taken into account, it's probably easier to have a rough idea of the reality.
39
Vexs 2 days ago 0 replies      
Bunnie of chumby/novena fame has this pretty great blogpost on how cheap stuff spurs innovation.

https://www.bunniestudios.com/blog/?p=4297

40
otaviokz 2 days ago 0 replies      
It's amazing the lengths western people will go in order to convince themselves that competing political/economical systems will never pose any real challenge...
42
dang 2 days ago 0 replies      
We changed the baity title to a representative sentence from the article. If anyone suggests a better (more accurate and neutral) title, we can change it again.
43
ksk 2 days ago 0 replies      
Replace America with Britain and China with America and travel back a few hundred years.
44
awt 2 days ago 0 replies      
I think we should censor articles about how censorship is good for China.
45
andyidsinga 2 days ago 0 replies      
"Silicon Valley may be powered by organic kale"

sheesh.

46
liveoneggs 2 days ago 0 replies      
I wonder who is paying for this shill.
47
bionsuba 2 days ago 4 replies      
> we're not all xenophobic Trump supporters, not even most of us

If one wishes not to be generalized, that person should abstain from doing it to others.

48
untilHellbanned 2 days ago 0 replies      
So what are the actual innovations? Opening coffee shops and having lots of e-commerce transactions != innovation.
49
nxzero 2 days ago 2 replies      
>> "In April, the U.S. government officially named the Great Firewall a barrier to trade."

That's absurd.

27
Google Cuts Its Giant Electricity Bill with DeepMind-Powered AI bloomberg.com
231 points by runesoerensen  4 days ago   78 comments top 15
1
bytefactory 4 days ago 5 replies      
Very frustrating read. Read more like an advertisement than an informative article.

The claims made would've made for a very interesting tech-dive into a novel use of machine intelligence, but no details were provided.

2
tdaltonc 4 days ago 5 replies      
The next step, of course, is to let the AI know which servers in the server-farm itself is running on so that it can optimize for self preservation.
3
fovc 4 days ago 2 replies      
They saved "several percentage points" off of 4.4M MWh, so maybe 250M KWH, which might be $10-20M. At 30x earnings [1], they just made back most if not all of the purchase price of Deepmind [2]

[1] https://ycharts.com/companies/GOOG/pe_ratio

[2] https://techcrunch.com/2014/01/26/google-deepmind/

4
honkhonkpants 4 days ago 1 reply      
Very light on details. What is the baseline of the savings? For example, was there a water pump that was always running at a fixed worst-case setting, and the machine learning system now ramps it up and down? If so, what is the marginal benefit over alternatives like a rudimentary closed-loop electronic control? Would like to know more about the system that was replaced, instead of these bare claims.
5
tdaltonc 4 days ago 3 replies      
They need to find some medium sized city that will let them play with their traffic lights.
7
zhanwei 4 days ago 1 reply      
"Now that DeepMind knows the approach works, it also knows where its AI system lacks information, so it may ask Google to put additional sensors into its data centers to let its software eke out even more efficiency."

Sounds like active learning to me. It's a type of machine learning where a learner pro-actively ask for interesting data points to be labeled so that he can learn more about the system. :)

8
zitterbewegung 4 days ago 0 replies      
This is very similar to DART where DARPA was able to recover all of its investment to AI and a testament to its pragmatism. Also both are logistical problems. See https://en.wikipedia.org/wiki/Dynamic_Analysis_and_Replannin...
9
sowbug 4 days ago 2 replies      
OT: it's frustrating to pay Google Contributor to skip ads, and still get nagged by bloomberg.com for using an ad blocker (which I've never done).

https://www.google.com/contributor/

10
iamleppert 4 days ago 3 replies      
How exactly is this better than standard PID control? I'm thinking if you actually look at what it came up with, is probably some form of PID control on systems that previously didn't have it. Think fans that are simply left on all the time.

We're talking about simple physics. Heat transfer. Cooling systems. They should have been installed, operated and programmed correctly using very simple techniques.

It's an interesting application but I'm thinking this is a prima facie example of over-engineering.

11
Aelinsaar 4 days ago 0 replies      
This is kind of thrilling... the somewhat generalized use for an AI with such a concrete benefit.
12
jjp 3 days ago 0 replies      
https://deepmind.com/blog - some more details.
13
T-A 3 days ago 0 replies      
Not exactly a new idea. Here's a random book which I happened to be looking at just yesterday: https://www.crcpress.com/Artificial-Intelligence-in-Power-Sy...
14
polskibus 3 days ago 0 replies      
Is this similar to what WalMart has been doing with their energy efficiency in shops for ages? I mean centralized power management, early warning system, lots of sensors, etc.
15
knowThySelfx 3 days ago 1 reply      
At some point will AI start to have questions like "Who am I? Who made me? Whats the purpose of my existence?" etc.We will have atheist AI's and theist/deist AI's and what not. I guess it will be time for some AI philosophy.

Makes me wonder if there's a creator, will He/She be amused by our attempts at answering "Who am I" and such questions.Will be fun :D

28
Show HN: Riko A Python stream processing engine modeled after Yahoo! Pipes github.com
277 points by reubano  2 days ago   63 comments top 15
1
reubano 2 days ago 2 replies      
`riko` is pure python stream processing library for analyzing and processing streams of structured data. It's modeled after Yahoo! Pipes [1] and was originally a fork of pipe2py [2]. It has both synchronous and asynchronous (via twisted) APIs, and supports parallel execution (via multiprocessing).

Out of the box, `riko` can read csv/xml/json/html files; create text and data based flows via modular pipes; parse and extract RSS/ATOM feeds; and bunch of other neat things. You can think of `riko` as a poor man's Spark/Storm... stream processing made easy!

Feedback welcome so let me know what you think!

Resources: FAQ [3], cookbook [4], and ipython notebook [5]

Quickie Demo:

 >>> from riko.modules import fetch >>> >>> stream = fetch.pipe(conf={'url': 'https://news.ycombinator.com/rss'}) >>> item = next(stream) >>> item['title'], item['link'] ('Master Plan, Part Deux', 'https://www.tesla.com/blog/master-plan-part-deux')
[1] https://web.archive.org/web/20150930021241/http://pipes.yaho...

[2] https://github.com/ggaughan/pipe2py/

[3] https://github.com/nerevu/riko/blob/master/docs/FAQ.rst

[4] https://github.com/nerevu/riko/blob/master/docs/COOKBOOK.rst

[5] http://nbviewer.jupyter.org/github/nerevu/riko/blob/master/e...

2
Fuzzwah 2 days ago 1 reply      
I was a heavy user of pipes and I'm now a heavy user of python. I have built my own dodgy simple replacement for some of the things I used to rely on pipes for. I'm very eager to see what you've got here, at first glance it seems like an excellent fit for my needs.

Thanks!

3
tanlermin 2 days ago 1 reply      
Can you consider Dask integration? http://distributed.readthedocs.io/en/latest/queues.htmlhttps://github.com/dask/dask

It can handle parallel and distributed parts for you.

https://github.com/dask/dask

4
raimue 2 days ago 1 reply      
I am still a user of Plagger [1], but development halted quite some time ago. Maybe this could be a good replacement.

[1] https://github.com/miyagawa/plagger

5
ecesena 2 days ago 1 reply      
This is really interesting. Have you looked at Apache Beam? What I think is interesting about Beam -in this specific context- is that it has a standalone runner (java), that similarly as riko let you write pipelines without worrying about a complex setup. But then, if you need to scale your computation, Beam is runner-independent and you can take the same code and run it at scale on a cluster, wether it's spark, flink, or google cloud. You can read more here [1].

As for riko more specifically, Beam will have soon a python sdk, but I'm unsure if there will be a python standalone runner. Maybe this is something to look into...

[1] https://www.oreilly.com/ideas/future-proof-and-scale-proof-y...

6
oellegaard 2 days ago 1 reply      
If you're looking for a stream processing engine more close to Storm, etc. but also simple, check out Motorway: https://github.com/plecto/motorway :-)
7
tudorw 2 days ago 3 replies      
if someone can spin up a usable gui, charge enough to make a living without compromising on performance, promise some longevity and a way to export of my stuff I would probably pay for that, I loved pipes, the GUI was a big deal for me.
8
ewindisch 2 days ago 1 reply      
Sweet. I put together something similar for NodeJS which is now called 'turtle' (because it's turtles all the way down...). There's a bit of a focus on AWS Lambda & other FaaS solutions as a means of building Lambda architectures, but it can be used by itself.

https://github.com/iopipe/turtle

9
mxuribe 2 days ago 0 replies      
While I didn't use yahoo pipes too often, I loved it. Having this as a python library (I'm trying to get deeper into python), is great! Kudos and good luck!
10
aioprisan 2 days ago 2 replies      
Is there anything like this available that's based on node.js with a decent GUI?
11
svieira 2 days ago 1 reply      
Also in this space (and worth looking at for inspiration, especially for other potential sources and sinks of data) - Apache Camel [1].

[1]: http://camel.apache.org/

12
pastaking 2 days ago 1 reply      
Also might want to check out http://concord.io, it's a bit more work to set up, but it's much faster than most stream processing systems
13
DyslexicAtheist 2 days ago 1 reply      
This is absolutely beautiful. Love the fact that it's using RSS for this.
14
et2o 2 days ago 1 reply      
Looks interesting. What kind of applications do people use this for?
15
satai 2 days ago 1 reply      
Looks nice.Are there any plans for twitter support?
29
Reducing Adobe Flash Usage in Firefox mozilla.org
235 points by _jomo  3 days ago   114 comments top 20
1
jhatax 3 days ago 8 replies      
I was hoping that Shumway, Mozilla's effort to render swf files using JS (like what PDF.js is to Adobe Reader), would be released at some point. It looks like the project has been added to the Firefox Graveyard [1]. I don't have Adobe Reader installed on my Mac any more, and don't really miss it.

While Chrome's proposal to white-list the top-10 domains is a good start at curbing the loading of Flash on my laptop, I prefer the approach being considered by Safari to report that Flash (and other legacy plugins) is not available on the platform even if it is installed. [2]

Safari's approach will ensure that most users see HTML5 content and won't really miss Flash. Folks who use sites like Twitch that insist on Flash will know how to force Safari to load the content they want to view.

Unfortunately, Safari's user share outside of Mobile is very low. We need Chrome, Firefox and IE to adopt a similar approach (or agree on an approach for all vendors) if we are to really rid ourselves of Flash.

1. https://bugzilla.mozilla.org/describecomponents.cgi?product=...

2. https://webkit.org/blog/6589/next-steps-for-legacy-plug-ins/

Edit: Moved links to the end of the post.

2
niftich 3 days ago 2 replies      
I'm very torn on this subject. I'm always wary when browser vendors force the hand of users, programmers, and everyone else.

I fully understand that Flash has had an outsized share of vulnerabilities 'affecting browsing' over the years; I fully understand that Adobe has deprecated Flash for new content production; I fully appreciate that the 'web platform' has acquired new APIs and capabilities over the last four years, making it a more potent platform than the days when people opted for Flash or Silverlight because an external runtime was the only way to reliably deliver the experiences those developers wanted.

But in a world where a HTML webpage from 1991 [1] still loads and renders fine, I'm worried about the sheer amount of content that exists in Flash from the 2000s that will be made inaccessible. Sure, those developers should have known that developing on a proprietary platform is a risky bet, but this was back when Javascript was awful, browsers were racing to implement not-yet-final enhancements to CSS3 with vendor prefixes, and powerful vendors were bickering about which formats to support in a proposed <video> tag. These developers of course should've known better, but they had no other choice.

What Mozilla is doing here is actually quite reasonable, but they're under pressure from Google Chrome who can unilaterally decide to ban flash from all but the top 10 sites, and get away with it due to their control of multiple platforms and their unwillingless to compromise.

If Mozilla's tactics stray too far from Google's, they risk being seen as followers, rather than policy drivers; furthemore they answer to a divided fanbase that on one hand wants an open, independent web (in which Flash has no place), and on the other hand, wants a refuge from the incumbent browser maker's unilateral policies (currently Google, previously Microsoft).

[1] http://info.cern.ch/hypertext/WWW/TheProject.html

3
_jomo 3 days ago 4 replies      
I don't have Flash installed at all anymore and it works quite well. For the few sites that don't work without Flash these days, I either don't care or use youtube-dl -g [0] or livestreamer [1] and open the direct video link in Browser or VLC.

Twitch is one of the popular sites that don't have a working HTML5 player for the masses (it does work without Flash using the methods above). There's Beam.pro which has some interesting approaches to live streaming with HTML5 [2]. The only thing I haven't found a great solution for are the big Music streaming sites, which all rely on Flash (the others shut down). Some people told me Google Play Music may or may not work with HTML5 but I haven't tried that yet.

Also, a great number of websites will ask you to turn on Flash when installed but deactivated and only use the HTML5 player when it's not actually installed. I guess it's a design flaw that Browsers report disabled or click-to-play plugins to websites.

0: http://rg3.github.io/youtube-dl/

1: http://docs.livestreamer.io/

2: https://forums.beam.pro/topic/168/where-we-re-at-with-html5-...

4
rcconf 3 days ago 5 replies      
If you Google 'top facebook games', and you browse to each one, you will find a majority of them use Flash. Here are a few of them:

- Candy Crush (50,000,000+ monthly users)

- Dragon City (10,000,000+ monthly users)

- Criminal Case (10,000,000+ monthly users)

- Angry Bird Friends (1,000,000+ monthly users)

I'm currently working on a Flash game with a large player base. Firefox's suggestion of adopting HTML technologies is not simple when the game is 9 years old! I think many Facebook games are going to run into a similar issue.

It's getting scary now tho, it seems like Firefox and Chrome are aggressively trying to get rid of the usage of Flash. We've essentially decided that we're going to convert this 9 year old game to C++ (via Emscripten) in the next year. Good luck to everyone else who is going through the same thing as we are.

5
rcthompson 3 days ago 0 replies      
> We categorized SWFs as fingerprinting SWFs if they were smaller than 5x5 pixels

Coming soon: 6x6 fingerprinting/tracking SWFs?

6
verisimilitude 3 days ago 1 reply      
It is interesting to contrast this discussion today with the discussion Jobs' "Thoughts on Flash" spurred 6 years ago: https://news.ycombinator.com/item?id=1304310
7
white-flame 3 days ago 0 replies      
> Over the past few years, Firefox has implemented Web APIs to replace functionality that was formerly provided only by plugins. This includes ... fast 2D and 3D graphics

Just a friendly reminder that the 2D graphics functionality of Flash is still not replaced for a massive chunk of graphics and games built with a vector-based visual style.

Canvas 2D vector graphics still do not properly antialias adjacent edges (shows garish seams and unexpected transparencies), whereas Flash would render them properly and with high quality.

8
ars 3 days ago 6 replies      
Any plan to reduce/remove flash needs to address the HUGE amount of small flash based web games. Just look for online playable games for kids and you'll see how many there.

"Websites that currently use Flash or Silverlight for video or games should plan on adopting HTML technologies as soon as possible."

This is utterly unrealistic, these games are 10 or more years old sometimes, and still played in large numbers, with no money available for the developer to rewrite them.

Only an automatic transpiler of some kind has any chance here.

9
nix0n 3 days ago 2 replies      
Now that HTML5 is gradually replacing Flash, has anyone seen a good Flashblock replacement for blocking HTML5?
10
supergreg 3 days ago 3 replies      
The only use for Flash I have these days is for streaming sites like Twitch. Once that's tackled, I'll be more than happy to remove the plugin.

That said, it used to be easy to block annoying stuff by having Flash enabled on demand.

11
jlebar 3 days ago 0 replies      
I know this is unimportant, but I have to say, I strongly dislike this green trendline that they have fitted to the graph.

It clearly does not fit. The graph flattened out at Jul 2015.

12
amelius 3 days ago 0 replies      
I really wonder if one day (perhaps in a distant future) HTML will end up on a graveyard, just like Flash, and what we can do now to make this event less painful.
13
Animats 3 days ago 1 reply      
Mozilla needs to do outreach to the porno industry to get them to convert.
14
codazoda 3 days ago 1 reply      
Good. They're going slow, starting with fingerprinting and supercookies, which is nice for users. I welcome the end of Flash.

I personally killed flash from Chrome about a year ago. I've seen a few sites that use it, which I just leave, but I haven't seen anything I can't live without.

15
ComodoHacker 3 days ago 0 replies      
>The criteria for adding content to the blocklist are:

>* Blocking the content will not be noticeable to the Firefox user.

>* It is possible to reimplement the basic functionality of the content in HTML without Flash.

There are three classes of content in the block list: Fingerprinting, Supercookie and Viewability. While I'm heard of various fingerprinting techniques besides Flash, I'm curious how "to reimplement without Flash the basic functionality" of supercookies, given its main feature is persistence despite of user's effort.

16
Endy 2 days ago 0 replies      
That's funny. I guess there's a reason why I'm being forced into using old browsers rather than supporting any of the rabid anti-Flash nonsense. Then again, I'm anti HTML5 & WebDRM (now under the more innocuous title of EME)
17
ivanhoe 3 days ago 0 replies      
I had it disabled in Chrome for the last 6 months or so, and very rarely needed to temporarily re-enable it, like maybe twice in all that time. And even that is not a big deal, you just go to chrome://plugins and switch it on and back off later, it takes 2-3 clicks to do it.
18
nfriedly 3 days ago 0 replies      
I don't install flash these days. I usually browse in Firefox and don't really miss it. On the occasions when I do need flash for something, I'll fire up Chrome because it has flash built in.
19
bobajeff 3 days ago 1 reply      
I think Chrome's propsal to whitelist the top ten domains and block all the other sites by default would be more effective at curving the web's dependency on flash.

Edit: Turns out Firefox is planning on blocking all sites by default. So Firefox's approach looks more promising.

20
fulafel 3 days ago 1 reply      
It's crazy that the perpetual security disaster hasn't been enough to disable Flash so far. (Goes for Chrome too, but at least they have reasonable sandboxing for it)
30
France orders Microsoft to stop tracking Windows 10 users theverge.com
231 points by abhi3  2 days ago   142 comments top 14
1
satysin 2 days ago 10 replies      
Is there any accurate analysis of exactly what Microsoft collects in Windows 10? I understand you cannot totally disable the telemetry (with the exception of an Enterprise version) but when put down to the "basic" level is there any capture of what is being sent? Everything I have seen is bullshit anti-Microsoft fairy tale stuff.

I understand text and voice data will be captured and sent if you use Cortana but that is pretty obvious, the same is true of Google, Bing, Siri, etc. [0]

What I want to know is when I put things at the lowest setting possible what do MS get and how often?

Edit: [0] I mean captured and sent for processing. I expect (perhaps wrongly) for it to be deleted from Microsoft's servers as soon as my request has been answered. Unlike Google which stores everything you say to Google Now for example.

2
bad_user 2 days ago 1 reply      
I would like laws that would force companies to disclose the bad side of what they are doing.

When they ask if you want to share what you type or say with them, in order to improve the experience and for you to get more relevant suggestions or more accurate spell checking or whatever, they only focus on the positives.

But that's not enough. I want them to say that your delicate and private conversations might leak and be used for nefarious purposes by disgruntled employees, state agencies, hackers or future owners of that data, because that's the truth.

Much like how cigarettes packs have graphical warnings on them. I'd like that very much, because as an ex-smoker I can tell you that those work. But of course, it would hurt their business to admit it, so they'll never do it willfully.

3
Taek 2 days ago 1 reply      
People are giving special attention to the things that Microsoft is doing, but history shows that they will eventually accept it and live with it.

Facebook has been doing this for a long time, to extremely high degrees of invasiveness. Google as well, and pretty much every single web startup in existence. Collecting data is how you compete in modern business.

If you think this Microsoft stuff is a big deal you should have another look at the entire foundation of modern tech.

4
jld89 2 days ago 3 replies      
Are there other countries as active as France is concerning the enforcement of user privacy laws and data protection?
6
acd 2 days ago 3 replies      
I switched to Ubuntu because of Windows 10. Using Linux as a primary operating system works great as a developer.

There is a saying which says if a product are free "you are the product". Microsoft made the upgrade to Windows 10 free a guess so that they can mine data about you and your habits. That data is valuable for marketing purposes.

Wireshark traffic dumps show a lot of data going to Microsoft telemetery.

I choose to say no to that data collection, instead wanting to keep a bit of privacy.

Have some security wiz MITM the Microsoft telemetry server with their own cert to inspect the data collection traffic?

7
serge2k 2 days ago 0 replies      
> the four-character PIN system used to access Microsoft services is insecure, because there is no limit on the number of attempts a user can make.

I just tried logging in with my pin.

After a handful of tries I was given a string to enter before I could try again. I did that. After another try I got told to restart the device before I could try again.

So it doesn't look like 10 tries and locked out forever, but rather increasing penalties for incorrect attempts. Which is fine.

oh and my pin is 6 characters long.

If they don't have this right why should we believe them about any of their other claims?

8
zamalek 2 days ago 1 reply      
> Microsoft: users are in control with the ability to determine what information is collected

> Microsoft: so enterprise customers will be able to completely turn off telemetry if they choose[1]

Which is it, Microsoft?

[1]: http://www.techrepublic.com/article/windows-10-now-lets-you-...

9
ionised 2 days ago 1 reply      
This is a step in the right direction, but no fine they can levy will be sufficiently punitive.

Companies like this will continue on and consider things like this simply the cost of doing business.

Kind of like banks. They don't give a fuck.

10
0xmohit 2 days ago 0 replies      
With LinkedIn [0], Microsoft has much more in its arsenal.

That said, who cares. I've hardly seen anyone use uBlock Origin, Ghostery or Privacy Badger. OTOH, people love tools [1] that read your email and notify about due bills and the like.

[0] https://twitter.com/darylginn/status/590664399041519617

[1] Google Now

11
72deluxe 2 days ago 2 replies      
I have LittleSnitch on my Mac and observe the requests that my Windows VM makes. I believe you can use an equivalent tool on Windows, such as GlassWire, or also the very useful tool O&O ShutUp10 with which you can disable telemetry settings.
12
VOYD 2 days ago 0 replies      
Good luck with that.
13
dogma1138 2 days ago 1 reply      
Windows 10 petite edition - coming soon.
14
sievebrain 2 days ago 4 replies      
Ah, CNIL.

What counts as "excessive"? Apparently whatever someone at CNIL thinks is excessive. I can imagine that Microsoft learning what apps you download is inevitable given their reputation based malware detection scheme: no way for that to easily work except by IE checking in with Microsoft to find out if a program is known malicious or not. And figuring out if a program is actually interacted with or not seems like a pretty good signal to determine if a new, unknown program is a silent botnet or not.

"4-PIN limit is insecure, because there's no limit on the number of accesses" is exactly the kind of bureaucratic central-planning nonsense that France has so many problems with. You do not need absolute counted limits on a password/PIN system to make it secure. You just need to take other steps to make brute forcing infeasible, like throttling the rate of attempts. Why is CNIL attempting to micro-manage the code for the Windows authentication systems, something they are clearly not qualified to do? The details of Microsoft's security system is their concern alone: if users dislike the way Microsoft do it, then they have other alternatives they can easily switch to.

I suspect Microsoft may do what other big companies do and simply ignore CNIL completely. They can only hand out relatively small fines and it's easy for big companies to just pay them off to make them go away. Their rulings have a long history of being completely unreasonable so it's usually the easiest path.

       cached 24 July 2016 04:11:01 GMT