hacker news with inline top comments    .. more ..    23 Jan 2017 Best
home   ask   best   2 years ago   
1
Google Has Started Penalizing Mobile Websites with Intrusive Pop-Up Ads scribblrs.com
873 points by sply  3 days ago   379 comments top 54
1
Smirnoff 3 days ago 6 replies      
I really would like to see Google penalize websites that force you to login after google showed these websites in the results.

1. Take Linkedin for example: you search for a person on google; google shows a linkedin result; you go to linkedin but you are greeted with giant popup asking you to login to view info. Ridiculous.

2. Same with Quora: they come in results with basic info, but when you go to their page, they forward you to registration/login page.

These practices are not ok in my book. Surely, they can do whatever they want on their websites but if Google indexes you and shows some info in search results, then you better show that info on your page without forcing me to register.

PS: To be clear -- this behavior happens on mobile version of their websites. Not sure how it plays out on desktop.

2
Animats 3 days ago 16 replies      
What's really stupid are sites from which you can buy things, but then pop up an ad for something else. Fandango, which sells movie tickets, does this. As you're trying to get to the "buy ticket" page, they shove movie trailers for other movies in your face.

I mentioned a site earlier today which sold plumbing supplies.[1] They pop up a "gimme your email" box which 1) cannot be dismissed, and 2) isn't even theirs, it's from "justuno.com", a spamming service.

These outfits have lost sight of what their web site is for. They're putting obstacles in front of a customer who's about to give them money. This is usually considered a big mistake in retail.

[1] https://www.tushy.me/

3
ageitgey 3 days ago 12 replies      
Most users hate these pop-ups and cheer this move from Google. But let me add a little context to why these ads are so prevalent and why some companies view this move as Google abusing their power.

If you visit to any "guide" website like TripAdviser, Yelp, etc, these days on a mobile browser, you'll notice that the sites often barely let you do anything without downloading the native app. They all but refuse to let you see content and throw up "Download our app!" pop-ups everywhere.

By traditional logic, that seems insane. Why are they putting so many roadblocks between the user and the content? Surely that must be driving away users, right?

The reason for this behavior is that Google is systematically destroying the SEO traffic of these sites by adding their own competitive features to search result pages that appear above organic results.

If you search for a restaurant / hotel / flight on your phone, Google will often show its own custom widgets above the organic search results. It's not unusual that zero organic search results are visible "above the fold". The more Google does this, the more the share of clicks goes to them instead of to organic search results in these types of searches.

That means that even if these guide companies have #1 search rankings for every possible search term, they are seeing their SEO traffic plummet every month because they can't compete with Google's "above #1 result" placement. So as a defensive move, some companies are basically giving up on SEO traffic in the long term and trying to forcefully convert many visitors as possible into users who visit directly via a native app (and thus bypass Google). They know that every web user who doesn't download the native app is ever less likely to ever find them again via a search result page.

So to these companies, they see this change from Google as another anti-competitive move because Google is taking away one of their last remaining lifelines for user acquisition.

Personally, I find those full-page ads super annoying and hate them too and think they should go away. But like anything complicated, this isn't a simple black and white move to benefit users. It's also a strategic move that helps Google and hurts some competitors.

4
netinstructions 3 days ago 2 replies      
Funny, because Google Adsense offers "Page-level vignette ads" which are full page interstitial ads shown for mobile devices.

The penalty must not apply because:

> They're displayed when the user leaves a page, rather than when they arrive on one, so the user doesnt have to wait for them to load

https://support.google.com/adsense/answer/6245304?hl=en

5
FreakyT 3 days ago 0 replies      
Good. Those have been becoming increasingly prevalent to the mobile web's detriment.

I don't mind a few ads, but many of these interstitials are downright maliciously designed, making the entire page load consistent on hitting a tiny "x" target, presumably designed with the intention of facilitating accidental clicks on the ad.

6
aresant 3 days ago 4 replies      
You mean intrusive like the AMP header on every !@$! mobile page now that's not only annoying but breaks the standard UX?
7
alphonsegaston 3 days ago 4 replies      
I'd really like to see them work on improving relevancy instead of swinging their corporate weight around at whatever "benevolent" end they decide is important this week. Considering how much time I have to spend nowadays tweaking queries and futzing with the search tool options to get relevant results, I'm starting to look at all of these moves much more cynically. Taking on anti-patterns is great, but not when your search experience is rapidly becoming one.
8
quadrangle 3 days ago 0 replies      
"Google Has Started Penalizing Mobile Websites with Intrusive Pop-Up Ads"

I totally read this as "Google Has Started Penalizing Mobile Websites [by using the penalty of imposed] Intrusive Pop-Up Ads" instead of "penalizing those websites that use Intrusive Pop-Up Ads"

9
jrochkind1 3 days ago 3 replies      
Why only "mobile websites"? I hate em just as much when I'm viewing on the desktop.
10
nhumrich 3 days ago 2 replies      
While I applaud Google for doing this, it's also very scary that Google has that much power that they can basically make anyone on the web do anything by threatening ranking blackmail. Google is starting to use more grey area tactics to control things (such as disabling accounts for those who resold a pixel). Makes me start to actually worry about the power Google has.
11
chinhodado 3 days ago 3 replies      
For me, the most annoying thing while browsing on mobile is the vibration ads/fake alarms. It's horrible. I haven't even seen a single good use of this vibration API, as it is only ever used for things like "Your phone haz virus click here now".

Why isn't there an option to disable it in Chrome is beyond me.

12
quadrangle 3 days ago 3 replies      
Use uBlock Origin people! In Firefox on Android. Don't see any pop-up ads! I can't believe how much pain people subjective themselves to needlessly!
13
makecheck 3 days ago 0 replies      
The only thing a web site should need to measure is how long its visitors stay. I know that the instant I see any pop-up garbage, I immediately leave: I dont care what the pop-up might say, I dont care where they mightve placed a little "X" to dismiss their message, I simply go BACK and I DONT return.

Do not allow yourself to be bullied. Yes, services have some nonzero value but your time also has tremendous value and you should not undersell it by putting up with stupid crap. Any site that shoves things in your face is being disrespectful, it is wasting your time, and it is costing you, which is not OK. Let those sites die out.

14
iwlbebnd 3 days ago 1 reply      
I've largely stopped using chrome on mobile because of the lack of ublock. When I do use chrome it's with JavaScript disabled.

Despite a few UI differences switching to Firefox on mobile with ublock has been excellent.

15
bogomipz 3 days ago 0 replies      
The article states"

"In short, if a web page puposely hides content behind an ad or forces interaction with an ad"

Does this mean that it doesn't include those nauseating "follow us/sign up for our newsletter" light boxes that plague the web now?

Also why would this only be for mobile? Are they any less of a scourge for desktops?

16
StuieK 3 days ago 0 replies      
If users hate these, shouldn't google's ability to rank the best pages already take care of this problem without special casing it?
17
evolve2k 3 days ago 6 replies      
A client has just asked me to add a pop-up "whatch this vid, join our newsletter", when the user scrolls to about half way down the homepage for their SAAS startup. Further the pop up is not to reappear for 90 days.

They got the approach from attending an online marketing workshop that suggested this increases their list.

Felt like a bit of an anti-pattern to me.

Anyone have advice as to if this is effective or if it will be affected by today's announcement?

18
chmars 3 days ago 2 replies      
What's about intrusive cookie warnings?

(They are apparently mandatory in the European Union and Google made them part of the Adwords rules some time ago.)

19
thebspatrol 3 days ago 0 replies      
Kind of scary that Google is so much of a gatekeeper to the entire internet that they can essentially decide which websites get visited and coerce them into submission.
20
therealmarv 3 days ago 0 replies      
So can we get rid of forbes.com quotes finally?
21
natch 3 days ago 0 replies      
This is great news... I'm normally not always a fan of Google's every move but when they use their position to encourage a better web, their power is awesome and we can look forward to the effects.

The headline was confusing. Google has started using intrusive pop-up ads to penalize mobile sites? This writing has clarity issues.

>Web pages need ads to operate, but...

Plenty of web pages (websites?) are operated without ads, out of love for a topic, desire to build a name, or other reasons. Web pages don't NEED ads. Well some have been built to rely on them, and can only survive with ads, but certainly not all web pages.

Aside from that and the headline, great level of detail in this article about the exceptions, the general sizes, and the rollout.

22
bradlys 3 days ago 0 replies      
This title confused me. I thought Google was penalizing mobile websites by injecting intrusive pop-up ads.
23
valine 3 days ago 0 replies      
I wonder if forbes will have to remove that annoying "Quote of the day page" or if this only applies to JavaScript popups.

I also wonder how google distinguishes between things like floating nav bars and elements that obscure content.

24
beefsack 3 days ago 1 reply      
I wonder how much of the internet userbase, like myself, just close a site the moment a pop-up appears which takes me away from what I want to be doing.

Has anyone here done any AB testing on it and have some numbers?

25
em3rgent0rdr 3 days ago 2 replies      
How about Google let me install extensions on Chrome in Android?
26
webartisan 2 days ago 0 replies      
The problem doesn't seem to be related only to web. These dark interstitial patterns are present in every platform.

Recently used Ola's (Uber's main competitor in India) native app on android, and right when you're about to book a pool ride, at times, they'll show you a full page interstitial advertising pool rides. And if that wasn't ridiculous already, they provide no way to cancel the popup. The only way to proceed is by clicking "Try share".

And when you do so, it throws a generic "Uh Oh, Something went wrong error", and you're basically stuck without a ride.

27
amelius 3 days ago 0 replies      
I want sites penalized when they take part in user-tracking.
28
digitalmaster 3 days ago 0 replies      
This is actually pretty impressive for a ad company to take steps that are overwhelmingly better from a UX perspective but also directly target online ad revenue models. #bold #impressive #hardProblems #thumbsUP
29
shadowSeeker 3 days ago 0 replies      
Many of these pop ups strategically block navigation options in the end adding up to unnecessary hits on some 2ndry linked page(via intrusive ads) or making sick stickies keeping us away from main material for which we were initially there making us dependent on AdBlockers. These sites find that out and stop access to their content(their concern maybe genuine but ugly process) until we stop AdBlockers taking away our freedom altogether. I have stopped going to most News Websites for this.
30
antihero 3 days ago 0 replies      
I think the most annoying trend is those redirect-to-store type ads. I think they're probably seen as malicious but there needs to be more done to prevent scummy advertisers doing this. You get to a point where you literally cannot use sites because the second after the page load you're redirected through a whole load of dodgy looking servers and eventually to the store to get some garbage app.
31
chimpscanfly 3 days ago 0 replies      
Look, popups are great for marketing, but this penalization isn't bad from a marketing and UX standpoint.

Popups, while effective, are being overused, which means they will become less and less effective.

On top of that, too many are poorly created and don't work on mobile, making it difficult to impossible to close out. This is unfortunate.

I've long held that we need a less intrusive "popup" that nudges instead of disrupts users. Basically a Hello Bar style that while catching my attention is something I can easily ignore.

32
JumpCrisscross 3 days ago 0 replies      
Looking at you, Forbes...
33
NinjaViktor 18 hours ago 0 replies      
I guess watching porn would be more enjoyable now.
34
notatoad 3 days ago 0 replies      
what about the websites that implement adsense's "please fill out this survey to view the content" ad? do they get penalized.
35
eumenides1 3 days ago 3 replies      
I wish Google would penalize websites with pay walls
36
webartisan 3 days ago 0 replies      
What's disheartening is that many websites have started finding workarounds to show these interstitials after a delay or in between screen transitions.

The inherent problem is that app experiences somehow lead to better conversions, and no company would want to lose revenue due to dropped install numbers.

37
bborud 3 days ago 0 replies      
Good. Fuck'em. This is the most eloquent response these sites deserve. They know this is annoying so there is no excuse for doing it while claiming it adds value or somesuch bullshit.
38
aphextron 3 days ago 0 replies      
I've never understood this pattern anyways. I swipe right and look for another website the moment I see a full screen popup on mobile. I can't imagine they are effective.
39
Transisto 1 day ago 0 replies      
Brave browser for mobile has only benefits. (the one that is based on chrome)
40
herbst 3 days ago 0 replies      
I hope this also penalizes facebook. They are using half of the screen to force me to a account. I know it is supposed to be less, but try with 1600x900
41
mikelbring 2 days ago 0 replies      
The worst one by far are those that also vibrate your phone. It should be illegal, or impossible.
42
apercu 3 days ago 0 replies      
This thread is hilarious. I wish I had time to upvote each comment.
43
amirmansour 3 days ago 1 reply      
How will this affect UI patterns like Modals?
44
t_fatus 3 days ago 1 reply      
if they could stop showing results in GoogleNow which, once clicked, immedialtely redirect me to some kind of creepy scary vibrating ringing page telling me I've a virus on my 'NEXUS 5X - Orange SAS' that would be even better
45
TwoBit 2 days ago 0 replies      
Can we penalize web sites that beg people to use their app instead?
46
jbicha 3 days ago 0 replies      
But will they penalize mobile apps with intrusive pop-up ads?
47
khana 3 days ago 0 replies      
Welcome news.
48
bluetwo 3 days ago 0 replies      
So... all mobile sites?
49
dhp1161 3 days ago 0 replies      
they need to penalize imgur for doing this
50
agumonkey 3 days ago 0 replies      
additional ideas for google:

- penalty for non no-js fallback

51
gagginaspinnata 3 days ago 0 replies      
Finally
52
st3v3r 3 days ago 2 replies      
53
st3v3r 3 days ago 2 replies      
54
serge2k 3 days ago 4 replies      
Oh good, Google abusing their power again.

I guess as long as it's for "good" reasons.

edit: would any downvoters care to explain how google being able to arbitrarily dictate web content is a power they should have?

2
NHTSAs full investigation into Teslas Autopilot shows 40% crash rate reduction techcrunch.com
801 points by fmihaila  3 days ago   314 comments top 25
1
Animats 3 days ago 4 replies      
It's interesting how vague this is. There's an NTSB investigation still pending into a specific Tesla crash.[1] The goals are different. NHTSA asks "do we need to do a recall?" NTSB asks "exactly what, in detail, happened here?" NTSB mostly does air crashes, but occasionally they do an auto crash with unusual properties. Here's the NTSB report for the Midland, TX crash between a train and a parade float.[2] That has detailed measurements of everything. They even brought in a train and a truck to reconstruct the accident positions.

It took a combination of problems to cause that crash. The police lieutenant who had informed the railroad of the parade in previous years had retired, and his replacement didn't do it. The police marshalling the parade let it go through red lights. They were unaware that the traffic light near the railroad crossing was tied in to the crossing gates and signals. That's done to clear traffic from the tracks when a train is approaching before the gates go down. So ignoring the traffic signal took away 10 seconds of warning time. The driver thought the police had taken care of safety issues and was looking backwards at the trailer he was pulling, not sideways along the track. People at the parade were using air horns which sounded like a train horn, so the driver didn't notice the real train horn. That's what an NTSB investigation digs up. Those are worth reading to see how to analyze a failure.

[1] https://www.ntsb.gov/investigations/AccidentReports/Pages/HW...

[2] https://www.ntsb.gov/investigations/AccidentReports/Pages/HA...

2
uncoder0 3 days ago 4 replies      
After looking at the report it looks like Tesla ran into the same issue we did in the 2007 DARPA Urban Challenge. The trailer was higher than the front facing sensors. We and most other teams had all assumed 'Ground Based Obstacles' meant that any obstacles on the test track would make contact with the ground in the lane of travel. DARPA decided to put a railroad bar across the street and expected cars to back up and do a U-Turn when they encountered it. The bar was too high off the ground for our forward LIDAR to see it so we collided with the bar at nearly full speed.[1] The sad part about this is that when we were drinking after dropping out of the challenge our team leader said something along the lines of 'At least we know no one will ever die now from the mistake we just made.'

[1] https://www.wired.com/2007/10/safety-last-for/

3
snewman 3 days ago 8 replies      
Tesla comes off extremely well in this report. For one thing, the 40% statistic cited in the headline appears to be well supported by the NHTSA report (section 5.4) and actually manages to frame the incident in a very positive light:

ODI analyzed mileage and airbag deployment data supplied by Tesla for all MY 2014 through 2016 Model S and 2016 Model X vehicles equipped with the Autopilot Technology Package, either installed in the vehicle when sold or through an OTA update, to calculate crash rates by miles travelled prior to and after Autopilot installation. Figure 11 shows the rates calculated by ODI for airbag deployment crashes in the subject Tesla vehicles before and after Autosteer installation. The data show that the Tesla vehicles crash rate dropped by almost 40 percent after Autosteer installation.

I had hoped to see more information about this specific incident. For instance, any data on whether the driver had his hands on the wheel, what steps the car had taken to prompt his attention, etc. But that doesn't seem to be included.

4
xenadu02 3 days ago 1 reply      
For those who don't want to signup to Scribd just to download a publicly available PDF: https://static.nhtsa.gov/odi/inv/2016/INCLA-PE16007-7876.PDF
5
stale2002 3 days ago 7 replies      
Oh, hey, will you look at that.

The imperfect, incomplete, beta, level 2 self driving cars that were supposed to be the "dangerous" area of self driving are ALREADY better than human drivers.

Can we stop the politics and deploy all the real self driving cars to the road immediately, since the government has proven that even the shitty variety is safer than humans?

6
sxp 3 days ago 3 replies      
The 40% number isn't very informative. The report has multiple notes about it:

ODI analyzed data from crashes of Tesla Model S and Model X vehicles involving airbag deployments that occurred while operating in, or within 15 seconds of transitioning from, Autopilot mode. Some crashes involved impacts from other vehicles striking the Tesla from various directions with little to no warning to the Tesla driver.

ODI analyzed mileage and airbag deployment data supplied by Tesla for all MY 2014 through 2016 Model S and 2016 Model X vehicles equipped with the Autopilot Technology Package, either installed in the vehicle when sold or through an OTA update, to calculate crash rates by miles travelled prior to[21] and after Autopilot installation.[22] Figure 11 shows the rates calculated by ODI for airbag deployment crashes in the subject Tesla vehicles before and after Autosteer installation. The data show that the Tesla vehicles crash rate dropped by almost 40 percent after Autosteer installation.

21 Approximately one-third of the subject vehicles accumulated mileage prior to Autopilot installation.

22 The crash rates are for all miles travelled before and after Autopilot installation and are not limited to actual Autopilot use.

So the actual rates of crashes for Teslas using Autopilot vs Teslas not using Autopilot aren't reported.

7
randomstring 3 days ago 2 replies      
Waiting for the headline: "Human Fails to Prevent Accident, Outraged Public Calls for Banning of all Human Drivers"

The obsession with perfection in self-driving cars is misplaced, they just need to be demonstrably better than humans.

This is obviously the future.

8
huangc10 3 days ago 5 replies      
Can anyone who is in the industry comment on how Autopilot performs in poor weather (ie. flash floods, thunderstorms, snowstorms etc...)

All I can find from the article about weather was in section 3.1:

> The manual includes several additional warnings related to system limitations, use near pedestrians and cyclists, and use on winding roads with sharp curves or with slippery surfaces or poor weather conditions. The system does not prevent operation on any road types.

9
cbr 3 days ago 1 reply      
This is really good news. A major worry with driverless cars has been that companies would be harshly punished for accidents, even if there was a dramatic reduction in crashes overall.
10
lz400 3 days ago 2 replies      
I think there's sometimes a lot of marketing and hand waving in this type of argument "crashes go down with autopilot". Most car accidents are caused by drunk or old people, and they drag the average up. if you tell me a Tesla autopilot beats a drunk guy it won't surprise anyone. Now as a non-drunk, young(ish) driver but experienced and careful, my statistics don't look anything like the average, they look at lot better. You have to convince this demographic, not beat the averages otherwise it's not rational for me to buy the feature. I'm guessing this goal post is a lot harder.
11
brilliantcode 3 days ago 0 replies      
For your reference, a level 4 automated car will look something like this:

https://www.youtube.com/watch?v=jhUX8qWFGc4

Imagine you are too fucked up to drive. Your car will be able to pick you up. Do you need Pepto Bismol too? Your car will pick it up from a drive through billed through your license plate. I'd give roughly 15~20 years for this to take place.

12
brighton36 2 days ago 0 replies      
The truck was visible for at least 7 seconds prior to the crash in the full report - another article here:

http://arstechnica.com/cars/2017/01/after-fatal-tesla-crash-...

Strangely enough, giving more people autopilot would probably be better than letting people drive. I think Tesla's picked the right time to enable it, since the cross-over point between autopilots being better than humans in general use cases has been reached.

Call it a beta if you want, but it's a pretty damn promising beta.

13
bcaulfield 3 days ago 0 replies      
So I'm far less likely to crash if I use this, and I have something to blame if I do. Everybody wins! (Except the engineers).
14
themgt 3 days ago 0 replies      
I don't always like Gladwell, but his piece on the Ford Pinto and the NHTSA philosophy towards auto safety more generally is quite worth the read [1]. I hadn't considered the intersection of this and self-driving car tech, but I wonder if NHTSA will basically take the position that as long as self-driving tech saves lives overall, a few "bugs" where the car kills the driver are an acceptable trade-off.

[1] http://www.newyorker.com/magazine/2015/05/04/the-engineers-l...

15
tn13 3 days ago 1 reply      
40% figure is meaningless unless the absolute numbers are reported. How do we know if this difference is statistically significant ?
16
em3rgent0rdr 3 days ago 0 replies      
I only want to use open-source code for something that my life depends on. That way it can be open to any one to inspect some can independently determine if the code is behaving as desired.
17
mrtron 3 days ago 1 reply      
What other car company can even recover the airbag deployment rate per mile?
18
ChuckMcM 3 days ago 0 replies      
That is a pretty remarkable report. It essentially holds Tesla up as an exemplar of the standard other car makers will be expected to achieve.
19
ridiculous_fish 3 days ago 0 replies      
This means that autopilot must be engaged at least 40% of the time (Amdahl's law!). Tesla owners, is that realistic?
20
schraitle 3 days ago 0 replies      
Does anybody know what the "Population" field indicates at the top of the report?
21
zekevermillion 3 days ago 0 replies      
Impressive. 2/5 reduction is a lot of lives saved.
22
NamTaf 3 days ago 0 replies      
I've railed on about the safety issues of autopilot before and how I'm not entirely comfortable with the pace they've developed compared to the considerations of human-machine interfaces and driver attentiveness, particularly given my (moderate) exposure to these sort of problems in other industries. Thus I'm particularly interested in that section of the article.

What I found interesting is that figure 10 shows that as you jack up the independence of the machine, the level of driver distraction accordingly increases. Adaptave Cruise Control (ACC) shows a significantly higher percentage of shorter-duration off-road glances than Limited-Ability Autonomous Driving Systems (LAADS). Additionally, countermeasures help to alleviate some but not all of that increase in distraction. Importantly, this is coupled with the point that the duration in which drivers have to react to most impending is under 3 seconds. This may seem obvious but it's a critical set of data to help objectively demonstrate the risks involved with losing or even reducing alertness.

It goes on to say that Tesla has addressed the risks of mode confusion, distraction, etc. and has implemented solutions to address this unreasonable risk, which they in turn define as abuse that is reasonably forseeable. In this, they're talking about the reasonably forseeable risk of eg: the driver not understanding if they're in autopilot or not. It goes on to mention that Tesla has also changed its driver monitoring strategy to promote driver attention, which I take to mean detecting hands on the steering wheel.

Either way, Telsa's main approach to dealing with driver alertness by testing for hands on the steering wheel. My concern is that this doesn't consider the alertness of the driver to their surroundings, particularly other vehicles that may be approaching them or the process of anticipating hazards (approaching an intersecion where there's a blind corner and adjusting focus to pay more attention to what may come from it, for example). I don't see how Tesla's countermeasures address this.

The physical act of manually driving causes drivers to maintain alertness not only to where they're going, but also the situational alertness of what's around their vehicle. Specifically, it's the process of random actions that requires an taking input, making a decision and executing the appropriate action that maintains this alertness. If the driver isn't having to make those random decisions and take action then their alertness drops. Autopilot, even with hands on the wheel, eliminates much of that random decision-making and reacting.

When you drive, you mentally note the vehicle over your shoulder that is in the lane next to you, and subconsciously consider that they may do something insane. You consider those blind corners as you approach them and that vehicles may spontaneously appear out from them. You see a truck on the road which is approaching a bend and give it a wider berth because its centre throw may cause it to cut the corner into your lane slightly. These are all tasks that you do, that you may not do as well or at all when autopilot is steering, because you are not as engaged with the driving process.

Critically, I don't see how ensuring hands are on the steering wheel causes these alertness tasks to continue as frequently as manual steering. The driver may be in the physical location to quickly take over, but they may not be in the mental location to do so. This is the major issue I have with the rapid autopilot development based on my experience in related areas where maintaining situational alertness proved to be very difficult when the person was engaged with only a limited scope of requirements to prove their conscious presence.

I feel like the report doesn't really drill in to this as much as it needs to. It begins to touch on it around Figure 10 but sort of hand-waves it away saying 'Telsa considered it discharged their responsibility to make sure drivers stay focused by implementing countermeasures', but I believe it's more nuanced than that. It investigates the extent to which Tesla's system is good at ensuring drivers are physically present (that is, their hands aren't on the passenger seat making breakfast) but it doesn't really look at the mental presence that delivers situational alertness.

That mental alertness is the major sticking point for me. I don't really have a solution beyond "drive manually" which isn't reasonable, because this technology is here to stay and will continue to grow, but it's why I've always been bearish about the rapid pace of rollout of these driverless technologies, particuarly when advertised as 'beta'. As I've said before, no amount of disclaimer and 'hey, you should do this' really changes how drivers behave once the equipment is placed in their hands.

23
sandworm101 3 days ago 3 replies      
Great. There is no doubt that driver assists cut down on crashes. But what tesla has on the road is far from a total eyes-closed autopilot. That is an inflection point with this tech that nobody has dared to test on the public road. I remain unconvinced pending those trials.

Also, still havent seen any autodrive handle off-road driving such as boarding a carferry or navigate a construction zone manned by an inattentive flag person.

24
battlebot 3 days ago 0 replies      
I don't completely trust the NTSA and I'm skeptical about auto-piloting cars but accept that more and more of those will be on the roads. I will never ride in a vehicle that lacks an override mechanism.

In general, I think we are moving way too fast towards these self-driving vehicles because certain factions want to try and replace long and short haul truckers with robotic systems that are cheaper and damn the consequences.

25
dkonofalski 3 days ago 3 replies      
I don't really know why this is surprising. Computers are already better than humans at most tasks that involve a limited set of behaviors and they have infinitely better response time than humans (and continue to get better). How could anyone think that a report like this was going to end up any differently?
3
I Had My Electronics Seized by U.S. Customs and Border Protection vc.gg
876 points by stryk  1 day ago   534 comments top 42
1
Steeeve 1 day ago 13 replies      
As an American citizen who travelled extensively up until TSA made me too miserable to continue, I am absolutely embarrassed about the travel situation in the United States.

I grew up in the land of freedom, and grew up hating overzealous and pointless "security" associated with the iron curtain.

We have freedom of speech, which inherently means freedom of thought. At one point we were the bastion of freedom and hope, and a leader in this regard. Nobody should fear harassment because they happened to speak at DEFCON anymore than one should fear harassment because they spoke at Jesus camp, or the science fair for that matter.

We SHOULD be encouraging people to come here and talk about things that might get them in trouble in their home countries. We SHOULD take a position of leadership and protect the basic inalienable rights of all people regardless of their citizenship, alliances, or thoughts.

I am absolutely appalled at the state that our government has arrived at. I am stunned that after such a strong revocation of the previous leadership that Obama's generation sought to increase security theater, decrease freedom, and decrease the rights afforded to our own citizens and our visitors.

The enemy of our freedom isn't thousands of miles away in some terrorist training camp. It's the people we freely elect who have seemingly no forethought and no historical understanding of our fundamental values.

A leader isn't someone who looks to cover his ass at every opportunity. A leader is someone who stands up, tells people there is something wrong, and pushes us back to the values that we were raised to be proud of. You don't have to be president to be a leader. Every police officer, every border agent, every engineer, janitor, and unemployed citizen can stand up at some point and see enough is enough. People need to stop arguing about politics and stop worrying about the boogey man. People need to start caring about freedom and they need to stop the culture of fear.

Sorry... This whole discussion hit a sore spot for me. I have always been a proud american and this kind of behavior just flies in the face of the values I was raised to embrace.

2
rwoodley 1 hour ago 2 replies      
Hmm... coming from the Chaos conference. Refuses to answer questions. Has a ton of equipment. Resides in Romania. And, get this: already involved in a terror hoax that shut every school in LA!! http://www.dailymail.co.uk/news/article-3367044/Maine-colleg...

What a sense of entitlement.

Seems like the TSA was doing their job.

3
vowelless 1 day ago 1 reply      
This is ridiculous!

Some colleagues and I get back from Barcelona in December and while I didn't have any problems [1], my colleague who is Indian born, but with a green card, was questioned for 30 or so minutes. Gladly he didn't have his electronics seized but as a permanent resident of this country, one would have expected some better treatment.

American border security has been among the most draconian one since I set foot into this country in 2007 [2]. I don't have high hopes from the next 4 to 8 years either. Conferences should start moving outside of the US. I don't see a point in non-Americans wanting to vacation in the US either -- there are a lot of other great places in the world.

[1] I am born in the Middle East, with a name that raises all the alarm bells for the DHS and was on the NSEERS list. I was detained in 2010 for a few hours in DC on my way back from the Middle East (everything short of a full cavity search was done). As soon as I got my green card, I applied for Global Entry and now don't have to interact with any border security agents on my way back into the country.

[2] In my early days, my boarding pass would have SSSS either hand written or printed on it: https://en.wikipedia.org/wiki/Secondary_Security_Screening_S...

4
ploddington 1 day ago 4 replies      
It would be fun to start a meme of carrying antiquated electronic waste devices, and getting them confiscated as a means of disposal.

Encrypt them with spacefiller data and start crossing borders, and then refuse to provide passwords and abandon them for sheer amusement.

Way more fun than recycling old laptops and batteries. And better than planning to travel empty handed, in anticipation of getting ripped off by pesky customs guards.

5
2ion 1 day ago 5 replies      
I've always been guarding against such things whenever I passed through a non-free or otherwise questionable country's air ports: Laptop contents were pruned of everything, leaving just the base OS, and data was stored redundantly on dedicated servers connected to the internet. I then would connect via SSH or VPN, unlock the encrypted partitions and download whatever I needed.

The only countries where I got treated in an appropriate, respectful manner at airports were Japan (both entry and exit as a foreign citizen) and Germany. UK (Heathrow) and VAE just had idiots and bullies for security and were my worst experiences. Which I'm assuming is entirely intentional. Going for the headphones even is really a bit stark I'd say.

Needlessly to say, I don't fly to or via these countries whenever I can avoid it at a good price.

6
rdslw 1 day ago 2 replies      
Might be slightly off-topic, but I'm still curious about one experience:

On return flight from US I refused to use full body scanner. I got 'thorough pat' (called that way probably to scare us, although it wasn't so bad), also my carry-on was checked in detailed way.Interesting things happened further, I have two stage flight.

1. On first landing in Germany, while moving by foot to next connection, I was picked by Germany border officers from all passengers going through corridor, and examined for about 5 minutes, also my carry on was scanned. Nothing serious, but happened first time ever (try of 20 I would say)

2. on final destination, while I picked my checked bag, it was marked with stickers, and I was picked by border people of my country, also asked, both bags checked.

3. At home I saw TSA notice inside of my checked bag.

Last thing wasn't unusual, I would say I had it on 30% flights from US, but all three items in a row connected somehow (?) to the full body scan refusal, were a little bit too much for a pure coincidence.

7
109981283091 1 day ago 2 replies      
The bizarre thing is that once upon a time these "contraband" checks were a thing at the East German (communist) border.

You would be checked for unwanted newspapers, literature etc.

At the time the U.S. was vehemently against the practice ...

8
un-devmox 22 hours ago 0 replies      
For those that don't know who don't know who Vincent Canfield is (I didn't), it is worth reading up on the man as he is a fascinating if not controversial person with a flair for raw/abrasive humor and a confrontational manner. He faced extradition and criminal charges in Germany [0] and his email service, cock.li, was at the center of the terrorist hoax against US schools [1].

[0] https://twitter.com/gexcolo/status/715640881651564544?lang=e...[1] http://motherboard.vice.com/read/the-story-of-cockli-the-sit...

I would like to respond to a few comments:

> Why do we only hear stories about this from the US and not from China or Russia? https://news.ycombinator.com/item?id=13454966

As a proud US citizen, I understand the slight you are feeling. It's as if we are continually being picked on and singled out for every misstep. In these types of thread I always see comments like:

> USA: make East Germany great again. https://news.ycombinator.com/item?id=13454689

> Yip. Traveling to the US is ironically the one place where I'd be concerned about my rights & freedom. Their tourist harassment force really needs to relax. https://news.ycombinator.com/item?id=13455194

Which infuriate me! But, we (as US citizens) should be held to the highest of standards because at times what we preach as a country is contradicted by our actions. It's not good enough to say, "Well, this is an isolated incident," or "Look at what x_country does." To justify abusive over reaching actions by our government based on what another country does is not logical or constructive. What we need to do is improve ourselves as a country and by doing so maybe set an example for others.

9
mirimir 1 day ago 2 replies      
The safest option is traveling light. Just a cheap notebook and phone, containing no sensitive data, which you will abandon if compromised/confiscated. That way, you attract little attention at borders. Keep your data online, securely encrypted, and buy equipment "anonymously" for cash.
10
ChuckMcM 1 day ago 4 replies      
Sadly this is feeling more and more like the new 'normal'. Carry a copy of your essential data, heavily encrypted, in an 'attachable' hard drive. Leave the expensive display and keyboard (aka laptop) at home.

Now I'm wondering if I can patch Asterix so that if I text it a particular challenge and response will it take the number that is texting it and route my phone number to it.

11
em3rgent0rdr 23 hours ago 0 replies      
The porn on the burner phone trip back seemed to defuse the agents. So maybe a strategy could be to have all your important work be encrypted, but have the a decoy boot which you reluctantly unlock after acting all nervous which contains legal porn in an open browser.
12
rwmj 1 day ago 2 replies      
Good job they've seized those dangerous headphones and USB cables ...

It makes no rational sense because if you think the headphones are more than they appear to be (eg. they have a secret memory card), then why not seize every other solid item -- pens, toothbrush, even the suitcase itself could be suspicious. So the conclusion is this is just designed to intimidate and inconvenience.

13
ergot 1 day ago 1 reply      
Worth listening to Jake Appelbaum's 'digital anti repression workshop' [1]. In this he explains why he takes the hard-drive out of his laptop and just uses a TailsOS thumb-drive for his computing. It would be actually hilarious when staff ask to peruse the contents of your computer for contraband, only to discover the laptop doesn't have a hard-drive.

[1]: part 1 https://www.youtube.com/watch?v=HHoJ9pQ0cn8

[#]: part 2 https://www.youtube.com/watch?v=s9fByRmAHgU

14
dirkg 6 hours ago 0 replies      
So if a friend or family member from Europe wants to visit, they shouldn't travel with their personal laptops, phones etc?

All this talk of 'keep your data on a private vpn server and ssh' is so far outside of what a normal person can do its ridiculous. I understand the concerns but 99.999999% of people aren't even aware of these things.

And yet they can be arrested, detained indefinitely, and much much worse, for no reason whatsoever, and have their devices confiscated at the very least - and this isn't just a possibility now (as in its possible a black hole will spontaneously emerge and swallow us all) but a very likely probability.

So what do we do? Not have any visitors, never travel?

15
Havoc 1 day ago 1 reply      
Yip. Traveling to the US is ironically the one place where I'd be concerned about my rights & freedom. Their tourist harassment force really needs to relax.
16
walrus01 1 day ago 3 replies      
For those not in the loop, this is the guy who runs http://cock.li
17
marak830 1 day ago 5 replies      
These officers refused to identify themselves to me at all, which is interesting "

Is that even legal?

18
jimnotgym 1 day ago 3 replies      
Wow a page full of horrifying border control experiences. Perhaps all those people who like to say Schengen free movement is a bad idea might want to reconsider?
19
nraynaud 1 day ago 0 replies      
USA: make East Germany great again.
20
dirkg 7 hours ago 0 replies      
TSA, HS and Border protection are a bunch of unqualified, uneducated, underpaid thugs put in a position of absolute power with zero consequences, and rewarded for every 'criminal' they catch. What exactly do you think will happen?

These are people whose dream job is becoming a cop and being allowed to wield lethal force on anyone with zero questions.

The best part is most everyone in this country is so brainwashed they'll actually support this because it keeps us 'great'.

21
anondon 1 day ago 1 reply      
Genuinely curious if anyone here experienced something similar (being asked to unlock all your devices) and if so how often?
22
aaronmdjones 1 day ago 1 reply      
I was wondering why I could not resolve cock.li from where I am right now; and thought this may be related, but it turns out the DNSSEC signatures have expired[1] so my resolver is rejecting the answers.

[1] http://dnsviz.net/d/cock.li/dnssec/

23
loydb 2 hours ago 0 replies      
We're heading to Italy in the Spring (from the US). I'll buy a cheap chromebook the week we leave, never use anything but incognito mode, and we'll rent phones in Italy once we're there.
24
aestetix 1 day ago 4 replies      
Why do we only hear stories about this from the US and not from China or Russia?
25
beezischillin 16 hours ago 2 replies      
Oh, awesome.

I'm planning to go visit my business partner in Texas in October, now I'm having serious doubts about the whole trip. (I am dual nationality, Hungarian and Romanian and I have a Hungarian passport which, theoretically, allows me to enter under the Visa Waiver program). By the way, I always travel with my Hungarian passport, because people at the border DO treat you differently. If you're on Romanian papers, you face way more harassment, more waiting and in general it's a lot more of a struggle.

I find it incredibly obnoxious that the only who are affected by this entire BS are the people who, I'm quite sure, are not planning to do acts of terrorism, in fact, has anyone noticed that the ones who do are the only ones this sht never managed to stop?

Btw, travelling within the EU has gotten a bit more annoying, too. When I left the UK at the beginning of April, last year, I pretty much got through without any issues, they haven't even bothered to look at my passport. I've not been searched, harassed, stolen from by border officials, etc., however in November, I've harassed, scanned, made to take my shoes off (your poison, brother), scanned to sht, and to top it off, the agent who was scanning my backpack stole out of it (2 PS4 video games, worth 20 quid, really?).

I'm planning trips to US, Germany, Austria, Ukraine and France this year, however based on experience from last year and now this, I'm pretty much questioning if I should by now.

I would've wanted to bring some of my photography equipment with me on these trips, so how risky would you think it would be to travel (for example to the US), if I had a mirrorless camera on me (Sony A6000), two batteries, 5 lenses, a tripod, a 13" laptop and two phones (always prefer to have a backup while traveling abroad)?

Background info: I have no convictions, I don't do drugs, I haven't been arrested, I don't attend security conferences and in fact, I'm generally a nobody.

26
mhomde 1 day ago 3 replies      
Question, say that I travel with an reset telephone, can you be forced to "install" it, complete setup and enter account information so it syncs it with your icloud/google/microsoft acocunt?
27
orbitingpluto 18 hours ago 1 reply      
Why would you cross any border with electronics with data on them? Especially if you went to 33C3? Especially in this climate?

Why didn't you just SSH everything? You can easily resync your contacts/data on your phone after you cross the border. You can keep the few you must have on the recently reset phone. If you travel often, why don't you have a second phone for just this to save you time?

I'd think most of us have tech junk piling up. It's less effort to reset or have a 'travel drive' than top go through all the paperwork to (maybe) get stuff back.

Maximum laziness. If customs asks you to unlock your phone, nothing gets you through quicker then quick unquestioning compliance. (Of course the current climate is stupid and needs to change.)

28
finid 1 day ago 1 reply      
All of my devices are encrypted. Though I'll be doing some key rotation as a result of this, I'm confident none of the devices will be able to be decrypted.

That's what I always advice folks to do.

29
nodamage 1 day ago 1 reply      
Is this person a US citizen?
30
pjc50 19 hours ago 2 replies      
Reports of people being denied entry for political reasons even though they should be guaranteed entry: https://www.dailykos.com/stories/2017/1/21/1623180/-Canadian...

(yes, it's dailykos, I'm sure there will be a claim of fake news)

31
paradite 1 day ago 1 reply      
Related discussions on border control:

U.S. government begins asking foreign travelers about social media

https://news.ycombinator.com/item?id=13242620

32
pcardh0 1 day ago 3 replies      
Is there something more to the story? I travel with laptops and iPads all the time. Never been stopped.
33
tmaly 21 hours ago 1 reply      
I was flying from Indiana back to New York last week. I got flagged for a search because they said my bag had some residue.

Utter BS I have travelled with the bag for over 20 years.

On the plane I was talking with the person next to me. I told her it was all security theater and she could not believe I could say that.

34
WildUtah 17 hours ago 0 replies      
This story inspired me to get my own @cocaine.ninja email address. I'll be putting it on my resume and all official bank or government correspondence from now on.
35
Fiahil 17 hours ago 0 replies      
I wonder, would it be possible to ship sensible devices through another medium (fedex?) instead of taking them through border security? Sure it would be expensive, but at least you don't get confiscated $2k worth of electronics because your name sounds funny.
36
dman 22 hours ago 1 reply      
Wonder if I can get TSA to do data recovery for some HDDs that I have that crashed.
37
london888 15 hours ago 0 replies      
US CBP and others often seem to see themselves as a kind of Special Forces detachment.
38
RichardHeart 1 day ago 3 replies      
I don't foresee any event which would cause the right to be free from unreasonable searches at the border to be acquired. I guess the only hope is the supreme court?
39
arnon 1 day ago 0 replies      
Terrifying
40
known 1 day ago 0 replies      
Facism?
41
isuckatcoding 14 hours ago 0 replies      
CBP is cancer.
42
lolikoisuru 1 day ago 1 reply      
That title is editorialized. Cock.li is not an anonymous email server, it's a regularemail service.
4
Container Tabs mozilla.org
821 points by malikNF  2 days ago   210 comments top 39
1
rlpb 2 days ago 8 replies      
I would like to be able to configure my browser to open every URL in a domain-specific "container", unless I say otherwise.

Say site www.a.org includes an image from www.evilcorp.org, and www.evilcorp.org sets a cookie. When I then go to www.b.org and it includes an image from www.evilcorp.org, I don't expect the cookie to be sent back.

In other words, the cookie should be tied to www.a.org, even though it actually came from www.evilcorp.org. It should only be sent if my URL bar says www.a.org AND the image is coming from www.evilcorp.org.

I feel that this is how browsers should have been designed in the first place. I welcome this Container Tabs feature, but I don't think it quite goes far enough to restore my privacy.

2
nikcub 1 day ago 6 replies      
This is a neat idea but it doesn't implement the main reason I use separate profiles in Chromium - different security contexts based on how much you trust a site.

Example: my main general browsing profile has flash, PDFs and all plugins disabled, absolutely all handlers switched off, all hardware access off, WebGL switched off, no account logins and uBlock Origin set to aggressively block most third-party requests.

My second most used context is for personal sites I login for - with access to third-party cookies (for those sites) and running most third-party requests with standard uBlock rules.

I have yet other contexts (Chrome profiles) where Flash is enabled if I need that, then a separate browser for Java etc.

I'd like to see these security levels built into browsers where the contexts are built around permissions and site trust rather than access to the user store (which is also important)

I don't think it's realistic for most users to do this right now with profiles, as it requires a lot of discipline - it needs to be in the UI.

Browsers have become as sophisticated as operating systems and we're accessing more and more of our personal data using them, yet the model of 'every site has access to everything' be it a site you trust, like Gmail, or a random URL you're clicking on - has somehow survived.

It's the equivalent of the old days where everyone would run their local systems with an admin account for everything. I really think browsers need a rethink along these lines where websites are treated like apps and you can apply a trust group to each.

3
JohnBooty 2 days ago 7 replies      
I am a Mozilla supporter and FF is my "daily driver" browser. Very interested in this feature.

Chrome has had it for a years, and it's a killer feature for many developers. It's very very useful to have multiple browser windows open, each logged into the same site as a different user. A lot of people do this by opening multiple browsers (FF, Chrome, Edge, Safari, etc) but that has its limits and it just adds another variable my poor brain would prefer not to handle.

Also very useful for home/work separation. One browser account for work and one for home. And also maybe one for porn. So that when you're screensharing in a meeting and you type a URL into the browser, you don't get autocomplete suggestions for your favorite porn sites popping up. Happened to an old (married) boss of mine once while displaying his screen on the projector... typed "a" into the URL field and the browser helpfully suggested he navigate to AdultFriendFinder. Right in front of some clients. :)

Firefox's "container tabs" implementation may be slightly confusing. Chrome's implementation is dead simple. One identity per window, and the identity name is always displayed in the upper right.

With FF's container tabs, I'll have one identity per tab, and I can see they're color coded, but that means I'll have to mentally map colors to identities. It's more flexible than Chrome's implementation but there's more cognitive overhead involved.

Also, what's up with the name "container tabs?" That tells me nothing about what they do. All tabs... contain things. I think they need to rename it to "identity tabs" or something. How on Earth would anybody ever guess that "container tabs" is related to identities and data sharing?

We'll see how it plays out though. I'm excited to try it and I am continually grateful for Mozilla's efforts. In fact this reminds me I haven't donated to them in a while....

4
x1798DE 1 day ago 3 replies      
Right now, what I'm doing is that all my standard browsing happens on Firefox, with uMatrix blocking JS and just generally locked down. For anything where I want to log in or have a persistent identity (stackoverflow, gmail, etc), I set up separate Chrome profiles and manually open each site in the appropriate profile.

Honestly, this whole thing is a bit of a pain - my ideal solution would be one where I can set up "domain groups" matching on the URL in the URL bar, each fully isolated from one another (different extensions, settings, caches, histories, forms, cookie stores, etc), and clicking links that go from one profile to another, all referer information is stripped out. Anything not matching one of the domain groups would go to the "default session" (which I would configure to be completely locked down and ephemeral).

Additionally, I'd want a context-menu item "Open link in group <x>", which would open something matching another domain group in the domain group of my choice, so that I could do things like visit gmail in two different groups.

5
altano 2 days ago 0 replies      
This is awesome. Microsoft's identity system is a nightmare so switching between my Office 365 email account and my OneDrive/music accounts is always annoying. I'd love to be able to contain each and stay logged in to both accounts.

At work I test lots of user accounts on the same site and make heavy use of Chrome profiles for that. This would fill a similar role.

But while I'm glad to have them, no average user would ever understand any of these concepts as presented in these screenshots.

6
znpy 2 days ago 6 replies      
I've been doing this for years using both Firefox' and Thunderbird's multiple-profile features.

Just run "firefox --no-remote -ProfileManager" and here you go.

So the serious question is: how is this any different from using multiple profile?

Multiple profile also have the pro/con that they are actual different processes, so there's no information leak between profiles whatsoever (well, unless some serious hacking happens).

Edit: being different processes with different profiles, they also have different configuration folders, different cookie sets, different password storage locations etc...

7
sirn 2 days ago 1 reply      
This looks amazing!

I have been using Self-Destructing Cookies[1] for few years and while I think the extension is great, I always feel there's not enough isolation between tabs. For example, if I have Twitter logged in in one tab, and other tab contain Twitter button, then the other tab can still have access to my Twitter cookie. (Because Twitter tab is still active, so SDC would not destroy the cookie.) I know this is solvable using tracker blocker, but something like SDC but worked on tab container level would be very welcomed.

(Other side effect of using SDC is I seems to get the harder ReCAPTCHA that make you click an object until all of it disappear, with new ones popping up after clicking. Usually took about 5-10 clicks. Very annoying.)

[1]: https://addons.mozilla.org/en-US/firefox/addon/self-destruct...

8
grenoire 2 days ago 2 replies      
I would love this feature and it would actually get me to switch to Firefox in a heartbeat. I'm currently using Chrome just because of the the (subjectively) better developer tools, but this is a feature that would make my life so much easier!
9
liminal 1 day ago 0 replies      
I like where this is heading but their pre-selected categories don't make sense. Container isolation should be based on security requirements rather than site content. E.g. shopping and banking have similar security requirements that are different than following a click-bait link on facebook.

I'd like a container per Google account since trying to switch users in their apps is a disaster that forces me to run multiple browsers.

I'd also like to tie sites to specific containers. So supposing Banking stays its own category, that should mean that any sites that open in the Banking container will never open in another container. Similarly it should be possible to whitelist a set of sites for a container so e.g. only specific banking sites will launch in the Banking container.

Each container should have its own set of security permissions.

I'd like to have disposable containers. I want a safe space where I can open a sketchy link and not have to worry about that page doing anything to the rest of my environment.

10
nkkollaw 4 hours ago 0 replies      
It would be cool if there was a shortcut to hide all other tabs but a certain group.

I would love if I could use this to organize my 100 tabs I always have open.

11
nmy 1 day ago 0 replies      
This is why I'm using Firefox nightly. It is a killer feature to keep open my many AWS and GCP accounts (one container per client). It still needs to be polished though.
12
thewisenerd 2 days ago 4 replies      
why isn't "Saved Passwords" and "Saved Search and Form data" separated between containers?

There have been autofill/form-data attacks in the past[0] and there was a story recently on HN's front page showing the same[1].

I'd like to point out that mozilla already has a configuration option to disable form data saving on https sites, 'browser.formfill.saveHttpsForms'. Why?[2]

> Right; the idea is to eliminate "opportunism". If my laptop is stolen, Firefox's current behavior makes it easy for a thief to find a https: site in my history, go to it, check out, and then just let autocomplete hand them my complete credit card details.

[0] https://news.ycombinator.com/item?id=12171547[1] https://news.ycombinator.com/item?id=13329525[2] https://bugzilla.mozilla.org/show_bug.cgi?id=252486

13
aplaice 1 day ago 0 replies      
I don't want to be too greedy (considering that the presence of this feature on the desktop is already great), but is there any chance that this will be coming to Firefox for Android? It might be a challenge to implement this UX-wise; however, it would also be extremely helpful since it would help in isolating mobile "web-apps" while still using a decent browser (Firefox for Android instead of Chrome), especially as profiles, which exist on the desktop version of Firefox, are not available on Android.

(Googling does not seem to have produced any relevant hits.)

14
newscracker 1 day ago 0 replies      
This is really awesome!!! I used to manage this kind of separation by using private windows and using different browsers. I don't really want to manage these by window or create user profiles, as may be the case in other browsers.
15
jagtesh 1 day ago 0 replies      
This is a great idea! I love the fact that I can have both contexts in the same window. What would also be pretty cool is being able to move all windows in one context into a new window, if I want to separate things in a new OS workspace.
16
_andromeda_ 1 day ago 0 replies      
This is really good. I might actually find myself using FF more now. Chrome supports multiple profiles but it's quite tedious make the switch(you'd need to create multiple accounts). The best solution that I'd been using was opening an incognito session, regular session, guest session(easier than having multiple profiles but hardly sufficient when you want many more separate sessions).

Edit: I love the color coding feature that distinguishes the distinct containers you have open.

17
TazeTSchnitzel 2 days ago 0 replies      
This would be nice for using more than one Twitter account without needing to open one in private browsing (TweetDeck exists, and I do use it, but I prefer Twitter Web). I hope it makes it to the release channel.
18
jedisct1 1 day ago 0 replies      
This is really neat.

I became a huge fan of Opera Neon's interface, though. And it would be a perfect fit for "containers". Drop icons into folders, and done. Folders represent containers.

19
cpeterso 1 day ago 0 replies      
How would you design a user interface for container tabs that non-technical users can understand?

Something like container windows, isolating different browser windows instead of tabs, might be a clear way to visibly show the separation to the user. In one window, they can log into their work Gmail. In another window they can log into their personal Gmail without any Google cookie confusion.

20
Tajnymag 2 days ago 1 reply      
Lack of history separation seems quite pitty to me imo :-/
21
mozillauser2017 1 day ago 0 replies      
I've been using the Private Tab addon to open separate logins in the same window for years: https://addons.mozilla.org/firefox/addon/private-tab/
22
jbverschoor 1 day ago 4 replies      
I'm long searching for a solution to do this is a much broader way.. On the OS level.

PrivateWork AWork BWork C

Ideally it would seem like a different user. (Filesystem, cmd-tab).But easily accessible like the three finger swipe.Fast user switching doesn't cut it.

I even tried logging in on my local machine using VNC or remove desktop.. Is having 4 VMs the only way?

23
chrisper 1 day ago 0 replies      
This is pretty awesome. Very useful.

My school's security is a joke. You cannot log out except by closing your browser. The session also never expires.

Not only that, but now they moved to a "single-sign on." If I sign in on one app, it signs me in for all apps.

24
raimue 1 day ago 0 replies      
Nice to see this integrated natively. I was using MultiFox before to get this functionality. For example, this allows me to manage multiple twitter accounts, without logging out and back in all the time.
25
therealmarv 2 days ago 3 replies      
Tech question: Does Firefox has technical process seperation of tabs nowdays? This is one of the main features of Chrome since 1.0 and just want to know if Firefox has something similar finally.
26
muddysky 2 days ago 1 reply      
Slightly OT: This page reads like Mozilla's Developer Network (MDN) online documentation, I had to read the first sentences few times until I got what container tabs are about.

However, very nice feature.

27
anigbrowl 1 day ago 0 replies      
A welcome development, though long overdue. I don't understand why browser UI is so boring and unimaginative.
28
j_s 1 day ago 0 replies      
I use the Qupzilla browser incognito mode which is a separate session per-window by default. It uses the Chrome guts and has a few rough edges.
29
StevePerkins 1 day ago 0 replies      
Awesome. Coming up with legitimately innovative features and publicizing them will do more good than a dozen new logo campaigns.
30
teekert 1 day ago 0 replies      
Nice, up until now I used Chrome as the browser logged into everything (fb, Twitter, Google) and FF as the main browser.
31
espeed 1 day ago 0 replies      
An official linux container tab with shell access (for dev, emacs, etc) would be a killer feature for Chromebooks.
32
zyxzkz 1 day ago 0 replies      
Been wanting something like this for years.
33
alkonaut 2 days ago 4 replies      
Why do I even need this? Isn't it by default so that site1.com can't see cookies from site2.com for example?
34
lmedinas 1 day ago 1 reply      
Anyone know when this feature is planned to hit Beta or Stable ?
35
drc0 1 day ago 0 replies      
the thing I blame is that ctrl+t on a container opens a new tab of the default container and not the one that is currently on focus.
36
WillyOnWheels 1 day ago 0 replies      
37
hosh 1 day ago 0 replies      
Cool. I had been using multifox plugin for testing websites, but it won't work with the new multiprocess engine and the plugin author had no intention of porting it. Glad to see an alternative.

I use Firefox for testing my dev work but reading about the privacy use-case, I might seriously consider switching from Chrome as my main browser.

38
DyslexicAtheist 1 day ago 0 replies      
not needed if you're running Qubes-OS :)
39
hkjgkjy 2 days ago 3 replies      
As a user, I actually just want my browser to contain less features. Vendors add and add and add features. If I want different user profiles, I already have many users on my OS - I just switch between them.

When Chrome came out, I and many others switched to it just because it was lacking so many features. It was great!

5
Chrome 56 will mark HTTP pages with password fields as non-secure googleblog.com
825 points by vladootz  22 hours ago   341 comments top 35
1
lucideer 20 hours ago 3 replies      
Firefox has started to do this recently and it's been fantastically informative and helpful.

It's the one new browser feature I never really considered wanting/needing before, that's really stood out to me as being incredibly valuable since I've started to see the warnings pop up.

2
foota 20 hours ago 13 replies      
Pm - "why is this page insecure"

Developer - "chrome labels password fields as insecure over http"

Pm - "what if it wasn't a password field"

3
no_wizard 23 minutes ago 0 replies      
I'm going to go ahead and make another shameless plug, since a lot of folks who are hesitant about this new HTTPS stack are worried about deployment, and thats for the fantastic folks over at Caddy. They make an Apache/Nginx alternative that has built in letsencrypt renewal support and automatically encrypts your site by default and serves over https/2.

https://caddyserver.com/

I am not an affiliated developer, but I am a user, and have recommended this to others as well, its a solid product.

4
KirinDave 15 hours ago 2 replies      
I've got to say though, that this is a wee bit frustrating as a developer. SSL libraries are terrible, bug ridden, hard to work with, and there are huge sacrifices using a pass-through proxy to offer SSL.

The brittleness of SSL libraries manifests not just in the form of security exploits, but also in the form of delaying the next generation of HTTP technology. Node doesn't support natively support HTTP/2 due to HTTP2 fitting issues [https://github.com/nodejs/NG/issues/8]. Jetty was delayed for Java SLL changes. Same with Go.

If Google wants to make the whole web secure? That's great. But we also need to work on making it simple to secure. So much research goes into novel ciphers and optimal ways to defeat timing attacks, and etc etc, but the spike in complexity means that we're reaching a point where almost no individual or group can approach a correct implementation.

It worries me that we're approaching a point where we're utterly dependent on a security standard no one can understand.

5
polygot 19 hours ago 5 replies      
I hope they do this for CC numbers too, because I know of a website I had to use that passed your Name, address, CC number, CC exp, amount; the whole shebang over plain ol' http to do a payment shudder.
6
alex_dev 17 hours ago 1 reply      
I've been paying rent with www.rentpayment.com which unfortunately serves up their home page with multiple logins over http. Naturally, emails and tweets to their support go ignored. Maybe they'll finally respond after more people ask them why they're "non-secure".
7
ktta 20 hours ago 1 reply      
Mods, can we please get the "?m=1" part of the url removed? I think the current link is for mobile.
8
no_wizard 13 hours ago 0 replies      
Alright, for what its worth everyone, if you haven't seen this already, here it is! The Cert bot from EFF!

Get that HTTPS motor running. This really does make it easy.

https://certbot.eff.org/docs/intro.html

9
myf01d 20 hours ago 2 replies      
Advancing HTTPS is one of a few good things Google made in the recent years. Thanks Google.
10
jvehent 20 hours ago 1 reply      
Firefox has a similar feature enabled in dev edition: https://blog.mozilla.org/security/2017/01/20/communicating-t...
11
throwaway6845 18 hours ago 2 replies      
Countdown until a JS extension that takes a normal <input> field and uses &bull; characters to make it look like a password field without tripping Chrome's detector...
12
kyledrake 13 hours ago 0 replies      
There should be an HTTP Header (or a CSP directive) to allow servers to set sites as "Not Secure" manually. That would help a lot of people dealing with phishing attacks on web hosts.

It would function in the same way - if Chrome detects CC/password forms, it labels the site as Not Secure.

13
SilasX 19 hours ago 2 replies      
Stupid question: Is the warning going to show up for localhost i.e. using chrome to see the local dev version of your website?
14
egberts1 3 hours ago 0 replies      
Needs to start blocking form fields that have no corresponding input text box because...these unused fields still get autofilled with cached but personalized info.
15
meta_AU 20 hours ago 4 replies      
What should be done for routers and printers that are accessed by their IP address?
16
Sarkie 5 hours ago 0 replies      
I hope me trying to push this on G+ and Twitter for years helped.

This was always my first install on a new Chrome.

https://chrome.google.com/webstore/detail/unsecure-login-not...

17
stilliard 8 hours ago 0 replies      
Working on a new community site to help people move to HTTPS: https://blog.movingtohttps.com/dedicated-to-simplifying-the-...
18
ryandrake 16 hours ago 0 replies      
Not a Chrome user, but this is a great feature, and is at least moving things in the right direction. Really they should go farther though. The UI treatment is almost un-noticable, even if they went with the "red triangle" version. How about a red-background interstitial page or a modal with a clear "Get Me Out Of Here" and "I Know What I'm Doing" choice for the user?

And for all those "small businesses" that are going to get affected by this? It's hard to muster up much sympathy at this point. It's 2017, and you're still horsing around with vanilla http?

19
vladootz 22 hours ago 0 replies      
I know the article is older, but it's January 2017, just a reminde.The message will appear in the address bar.
20
HappyTypist 16 hours ago 0 replies      
I'm in an A/B test group where all pages are marked either green 'Secure' or red 'Not Secure', password or not.

I like it.

21
LogicX 14 hours ago 1 reply      
It's great that Google wants to move more sites to https, and I'm in support of this, but it also creates challenges for security vendors such as myself.

Currently DNSFilter and others Man in the Middle traffic destined for sites our customers have decided to block. This works great for http, but not https, as certificate warnings are presented.

The standard work around is arguably less secure: adding a third-party CA to all end-points. This can still present problems with HSTS and certificate pinning.

I'd like to work with Google to create a standard where vendors can either be on a whitelist or have new recognized SSL cert fields, not to MITM traffic, but just to present users with a friendlier message explaining whats happening, and providing a separate https:// url to visit for information from the vendor about the block.

Implementing such a standard in browsers would further increase user security, and provide a viable method for filtering on guest networks where there is no end-point access.

22
ifelsehow 16 hours ago 0 replies      
Will this also apply to data URIs? Thinking of the recent data URI phishing exploits [1]

[1]: https://www.wordfence.com/blog/2017/01/gmail-phishing-data-u...

23
tehlike 16 hours ago 1 reply      
What happens if the page is insecure, but the attacker places an iframe in the page with HTTPS url, which then tricks the user into sending their credentials (unsuspecting users will think they are logging into the site).
24
agumonkey 12 hours ago 0 replies      
Do they send a letsencrypt notice to these domains ? notifying users is awesome, helping "late" hosts into HTTPS would be perfection.
25
idbehold 19 hours ago 2 replies      
How does it handle password inputs that are added to the page with JS?
26
clamprecht 18 hours ago 3 replies      
What if the <form> submits to an https page but the page is served up on an http page? The form submission will be secure, correct? Will Chrome still mark as insecure?
27
iceman_w 15 hours ago 0 replies      
Insecure websites could get around this by not marking the fields as password fields but using javascript to make them appear so to the user.
28
ams6110 20 hours ago 6 replies      
It should just label HTTP pages as "not secure", full stop. Because they aren't secure. Or at least, any page with a form. Never mind if it's a password field or not.
29
thowfaraway 13 hours ago 2 replies      
So if we have an http page with a password field that posts via https, it will be marked non-secure?
30
johndoe4589 18 hours ago 0 replies      
> A substantial portion of web traffic has transitioned to HTTPS so far, and HTTPS usage is consistently increasing. We recently hit a milestone with more than half of Chrome desktop page loads now served over HTTPS

Well OBVIOUSLY when the traffic is increasingly going to the same top ten sites like Faceboo, Twitter and Co.

31
daryltucker 19 hours ago 0 replies      
I'd like to see this with Adobe Flash and third party scripts. (Pandora!)
32
rectangletangle 19 hours ago 0 replies      
About time, this is an excellent feature.
33
ai_ja_nai 19 hours ago 0 replies      
Thank God
34
malikNF 17 hours ago 2 replies      
This is such a dumb idea on google's part (and mozilla's) because people are now going to program dumb workarounds for this.

Google seriously has to stop trying to police the god damn web.

35
Elrac 20 hours ago 0 replies      
"Thanks, Captain Obvious!"

Is there an option to turn this off, for those of us who feel we need it like a hole in the head?

6
Removing Python 2.x support from Django for version 2.0 github.com
714 points by ReticentMonkey  4 days ago   387 comments top 26
1
Flimm 4 days ago 3 replies      
The next release, Django 1.11, will be a long-term support release, and the one after that, Django 2.0, will no longer support Python 2.

https://www.djangoproject.com/weblog/2015/jun/25/roadmap/

I've grow to highly respect the Django project for its good documentation, its healthy consideration for backwards compatibility, security, steady improvements and all round goodness.

2
yuvadam 4 days ago 3 replies      
This call has been made a while back, and it makes perfect sense. Python 2 is slowly being EOL'd and if you're starting a brand new Django project there's no reason on earth you should choose Python 2 anymore.

Sure legacy projects still need support and for that they get the 1.11 LTS, but otherwise it's really time to move on.

3
rowanseymour 4 days ago 0 replies      
I'm glad they are making a clean break from Python 2 and I hope this pushes other projects in the ecosystem to fix those remaining libraries without Python 3 support. It does get a bit frustrating when things break between Django releases, but they have a good system of deprecating things for a couple of releases beforehand. And at the end of the day, Django is for people who want to build websites, not life support machines... and I think they're doing a decent job of striking a balance between breakage and stagnation.
4
nodamage 4 days ago 8 replies      
I have a Python 2.7 project that has been running smoothly for many years now and I'm having trouble finding a reason to upgrade to Python 3. The project uses the unicode type to represent all strings, and encodes/decodes as necessary (usually to UTF-8) when doing I/O. I haven't really had any of the Unicode handling problems that people seem to complain about in Python 2.

Can someone explain what benefit I would actually gain from upgrading to Python 3 if I'm already "handling Unicode properly" in Python 2? So far it still seems rather minimal at the moment, and the risk of breaking something during the upgrade process (either in my own code or in one of my dependencies) doesn't seem like it's worth the effort.

5
stevehiehn 4 days ago 5 replies      
Good. I've been getting into python a bit because i have an interest in datascience. I'm mostly a Java dev. I have to say the python2/3 divide is a real turn off. Many of the science libs want to use seem to be in 2.7 with no signs of moving.
6
oliwarner 3 days ago 0 replies      
A whole pile of people complaining about upgrading Django highlights two things to me:

Not enough people are using tests. A decent set of tests make upgrade super easy. The upgrade documentation is decent so you just spend 20 minutes upgrading broken things until it all works again.

People pick the wrong version. I've seen people develop and even deploy on -dev and it makes me cry inside because they'll need to track Django changes in realtime or near enough. Pick an LTS release and you get up to three years on that version with security and data-loss upgrades and no API changes.

7
misterhtmlcss 3 days ago 1 reply      
Is anyone going to talk about what this means for Python and Django? I read the first 30-40 comments and they are all about off topic stuff related to Django, but still the core premise is the committed move to Python 3.x going forward.

What do people think of that?! I'm a newer dev and I'd really really love to hear what people think of that and what it means for the future rather than side conversations about how bad their API is, how good it is, how good their Docs are and how bad they are.... Blah blah.

Please!! This community is filled with some of the most brilliant minds and I for one don't want to miss out on this chance to hear what people think of this change.

Please please don't reply that you disagree with my POV. That's irrelevant, but please do if you are interested in the initial topic. I'd be be very excited to hear your thoughts.

So Django moving to Python 3.X Go :)

8
erikb 3 days ago 0 replies      
There are only two possible opinions here:

A) You mostly have Python3 projects: Then you like it because you know more ressources will be spent on your pipeline and having more Py3 packages is also helpful.

B) You still have Python2 projects: You hate it, because it pushes you out of your comfort zone.

But I have to say, we want our langauges to develop as well. We want our packages to get attention. And there was lots of time to switch and experiment with switching. Ergo, it should happen. Even if you don't like it as much, that's where things are heading. Deal with it, move on. Let the community help you, if necessary.

9
gkya 4 days ago 0 replies      
This is a nice patch [1] to review for Python coders. Seems to me that most incompatibilities are provoked by the unicode transition.

[1] https://patch-diff.githubusercontent.com/raw/django/django/p...

10
karyon 4 days ago 0 replies      
The related django issue is here: https://code.djangoproject.com/ticket/23919

there are lots of other cleanups happening right now. It's a real pleasure to look at the diffs :)

11
myf01d 4 days ago 1 reply      
I hope they just find a way to support SQLAlchemy natively like they did with Jinja2 because Django ORM is really very restrictive and has numerous serious annoying bugs that have been open since I was in high school.
12
ReticentMonkey 4 days ago 0 replies      
13
Acalyptol 4 days ago 2 replies      
Time to introduce Python 4.
14
gigatexal 4 days ago 0 replies      
This is great news. It will help move people off their python 2 code bases even more. Kudos to the Django team.
15
gojomo 3 days ago 0 replies      
Because incrementing version numbers is free, Django might as well bump the Python-3-requiring version number to Django 3.0.

Lots of beginners and low-attention devs will find "Django 3 needs Python 3" easier to keep straight than "Django 2 needs Python 3".

16
karthikp 4 days ago 2 replies      
Oh boy. And here I am still using Py2.7 with Django 1.6
17
mark-r 4 days ago 1 reply      
I was surprised to see the elimination of the encoding comments, I thought that the default encoding would be platform dependent. After a little research I found PEP 3120 which mandates UTF-8 for everybody, implemented in Python 3.0. It also goes into the history of source encoding for 1.x and 2.x. I wonder why there aren't more problems with Windows users whose editors don't use UTF-8 by default?
18
romanovcode 4 days ago 1 reply      
Good, it's about time this nonsense ends.
19
ReticentMonkey 4 days ago 1 reply      
Can we expect the async/await introduced from Python 3 for async request handling or maybe some heavy operations ? Something like sanic: https://github.com/channelcat/sanic
20
hirokiky 3 days ago 0 replies      
Say good bye to django.utils.six. yay
21
alanfranzoni 4 days ago 7 replies      
So, after a poor evolution strategy that lead the Python world to be split in two and forces maintainers to offer two versions for the same library, and upstream maintainers to offer support for two different python versions, the same is happening for Django!

I speculate that the latest Django 1.x will remain used - and possibly the most used - for a lot, lot of time.

22
daveguy 4 days ago 1 reply      
Seriously? The entire change to "unsupport" the majority of Python code is a mass delete of from __future__ import unicode_literals and utf-8 encoding? Is that really the extent of the "too difficult to maintain" code? There will be a split.
23
scrollaway 4 days ago 2 replies      
Oh my god stop. You're all over this thread. What bit you?

This is the price you pay for staying on an old version. You do not get to stick to an old version AND demand that others do too.

You CAN stay on Python 2. You CAN stay on Django 1.11. It's LTS. So is Python 2.7. You get to use both until 2020 with no issues. After that, not upgrading is a technical debt that will start to accrue, faster and faster as you can no longer use recent versions of various software.

You are free to make your infrastructure immutable; you then become responsible for it of course. And the money you're not willing to spend porting to Python 3 today will be money you spend on costs related to being on outdated infrastructure, years in the future. That's a tradeoff. Banks do it a lot I hear. A bunch of companies still use ancient hardware and technologies nobody would think of starting a business with today. These companies make billions.

You know what the employees of these companies aren't doing? They're not bitching on HN that the tech they're using is no longer supported.

24
jdimov11 4 days ago 3 replies      
25
belvoran 4 days ago 0 replies      
A VERY GOOD NEWS!!!

Yea, I know, shouting is not the best thing, but this is a really good news.

26
jonatron 4 days ago 0 replies      
Django was designed for making content based sites and CMS's quickly. It wasn't designed for webapps and REST APIs, and it can be used in those cases, but it's not great. I'd look at other options.
7
Lavabit Reloaded lavabit.com
616 points by ycmbntrthrwaway  2 days ago   223 comments top 28
1
mvip 2 days ago 4 replies      
If you really want secure email, having it hosted and owned by a U.S. company is a recipe for disaster. Since we know that the U.S. gov't will gladly issue gag orders and blackmail, why even bother? It's great that Lavabit is innovating but Protonmail is already ahead by simply not being in the U.S..
2
bigbrooklyn 2 days ago 13 replies      
If you NEED encryption, don't use email.

From: https://blog.fastmail.com/2016/12/10/why-we-dont-offer-pgp/

What's the tradeoff?

If the server doesn't have access to the content of emails, then it reverts to a featureless blob store:

 Search isn't possible Previews can't be calculated If you lose your private key, we can't recover your email Spam checking on content isn't possible To access mail on multiple devices, the private key needs to be shared securely between them
update:want->NEED

3
tinkersec 2 days ago 1 reply      
Code for Magma Mail Server: https://github.com/lavabit/magma

Code for DIME (Dark Internet Mail Environment):https://github.com/lavabit/libdime

4
codehusker 2 days ago 5 replies      
Is there any person as trustworthy as Ladar Levison for a service like email or chat?

To my knowledge, he is one of the few that has gone to the mat for his users.

5
jimnotgym 2 days ago 0 replies      
Whatever did or didn't happen in the past, I for one am pleased to see another organisation attempting to make email more secure. Especially when governments have gone surveillance crazy. Goodluck Lavabit
6
MaymayMaster 2 days ago 2 replies      
>Lavabit believes in privacy and will always ensure your digital freedom.

>Asks for your credit card information on the same page.

Wew, at least let us use buttcoin, Levison.

7
MichaelGG 2 days ago 1 reply      
Last I looked, DIME was just org level trust. That is, your domain determines what level of verification you get as far as knowing you have the right key for the recipient.

So if you used, say Gmail and they did DIME, you'd still be trusting them totally. Am I misunderstanding?

And still no admitting he was selling a fundamentally critically flawed service in the first place. If that's not even being mentioned, it really removes confidence from their new service.

As far as hardware HSM, that's cool. I very much enjoyed reading about how an HSM, the Luna CA3, was cracked:

http://www.cl.cam.ac.uk/~mkb23/research/Unwrapping-the-Chrys...

8
akerl_ 2 days ago 1 reply      
Trustful seems like a strange way to refer to the insecure mode. It is indeed full of trust, but not in the way a normal read would suggest: it requires full trust in Lavabit's hosting provider and administrator.

If you're going to operate in "trustful" mode, lavabit isny offering any real security wins over any other mail host.

9
kijin 2 days ago 0 replies      
What I want is an open-source proxy that I can install on localhost to provide IMAP/SMTP access on the one side, and talk to the encrypted remote data store on the other side.

All of the encrypted email services I've seen so far, including Protonmail and now Lavabit v2, require using a special client (app or webmail) instead of common email software. This fails the very first test that I apply when trying to decide whether or not to use an online service: can I get all my data out of it on short notice, in a standard format through an automated process?

For email, this means IMAP access so that I can use standard tools like imapcopy to back up and migrate my mailbox. I don't care how secure your product is if it leads to vendor lock-in. I want both good encryption and an exit strategy, and the latter is much more important because if you screw up, I can always move to someone who does it better.

10
coretx 2 days ago 3 replies      
Sensible choices in a nutshell: If you live in a 5-eyes nation, don't use or buy services hosted or operated from a 5 eyes nation. If you don't live in a 5 eyes nation, only use services hosted and operated from Iceland or Switzerland.( Nation states are the #1 threat, and your own nation is always the most dangerous one. )
11
tptacek 2 days ago 2 replies      
In August 2013, I was forced to make a difficult decision: violate the rights of the American people and my global customers or shut down. I chose Freedom.

Shouldn't that "or" be an "and"?

12
mike-cardwell 2 days ago 1 reply      
So they're using a HSM to protect the SSL key this time. Makes me wonder how many HSMs out there are already backdoored.
13
smoyer 2 days ago 1 reply      
How do we know who's controlling the Lavabit domain?
14
macmac 2 days ago 2 replies      
Why would they ask for name, address etc?
15
OJFord 1 day ago 1 reply      
Why does their server need your private key? (Except "paranoid" level - I'm much more concerned about handing my private key over to them than anything to with email, why's that paranoid?!)

Why can't they just receive, encrypt with my public key, let my client decrypt with private?

16
advisedwang 2 days ago 1 reply      
The explain document doesn't describe how key distribution works. How do I get a public key for somebody that I want to email, and how can I know that I am getting the right key?

This is the hard part of an modern cryptosystem and the usual source of weakness.

17
zymhan 2 days ago 4 replies      
Any reason I shouldn't sign up right now?

edit: Signed up. Half off for life is a sweet deal.

18
betolink 2 days ago 0 replies      
I consider this article relevant to this discussion: "Hackers can't solve surveillance" http://www.dmytri.info/hackers-cant-solve-surveillance/
19
daveheq 2 days ago 0 replies      
Naming it the "Dark Internet Mail Environment" is not going to get the average person's sympathy or interest, and will be an easy target for politicians.
20
chadcmulligan 2 days ago 1 reply      
If I was a government spook I'd set up an email service, then make a big show of closing it down because the government. Then decide to make a big show of 'No, Security is paramount' and reopen my mail service.

Not saying this is what happened of course but without legislation all 'secure servers' must be considered corrupted or corruptible. There isn't a technical solution to trust.

..or even going into extreme tinfoil hat mode - how do we even know this is the same person. Again no technical solution

Edit: why the down vote? - perhaps a counter argument would be better, I'd like to be proved wrong.

21
Arallu 2 days ago 1 reply      
What's the difference between Standard and Premier?
22
grecy 2 days ago 0 replies      
> Today is Inauguration Day in the United States, the day we enact one of our most sacred democratic traditions, the peaceful transition of power

Sitting here in West Africa, watching the news, we didn't see much peace during the rioting in the streets in (I assume) Washington

23
newsat13 2 days ago 1 reply      
There is no way I would trust lavabit again given it's past...
24
truebosko 2 days ago 0 replies      
Is this the right space to ask for opinions about Fastmail and its privacy? I just switched on trial after being on Gmail. I'm happy but I switched primarily to get part of my life away from Google.
25
DKnoll 2 days ago 1 reply      
I can finally get my old mail back. :)
26
wjd2030 2 days ago 0 replies      
this smells funny.
27
satysin 2 days ago 0 replies      
No trial is a shame.
28
tastythrowaway2 2 days ago 3 replies      
this vs protonmail.ch?
8
Amazon Web Services in Plain English (2015) expeditedssl.com
646 points by apsec112  3 days ago   76 comments top 22
1
michaelbuckbee 3 days ago 15 replies      
Hey, Author here. This is old and I haven't added some of the new services that AWS has released since I first wrote it.

Whenever this list comes up there's generally a group of people that dislike it for trying to be at least mildly humorous (The whole concept for it started with my developer friends and I joking about some of the names and how opaque they were, so not sure what I'm supposed to do).

There were a couple substantial edits I made to it where a few funny lines were cut in favor of better explaining what/how something worked.

I also started fleshing out some of the services with slightly more in-depth articles about them (such as this discussion of AWS Buckets where I compare Amazon's CTO to a character from 28 Days Later - https://www.expeditedssl.com/aws-s3-buckets-of-objects

I've sometimes thought that I should try and make it into an ebook or something, but there's always been something more interesting to work on. Thanks to everyone who has enjoyed it, shared it with their friends and hopefully took their first steps to messing around with AWS.

2
dcw303 3 days ago 2 replies      
This is really useful for a layman like me who doesn't have a lot of exposure to AWS.

Anything similiar for Azure? I would really like to understand the difference between the different types of app services, and especially how they relate to the project templates in Visual Studio.

3
beefsack 3 days ago 1 reply      
Calling S3 "FTP" is a bit misleading, I would have just called it "File Storage" and explained it along the lines of FTP instead.
4
pyreal 3 days ago 0 replies      
I just discovered a bunch of interesting stuff that I had no idea AWS provided thanks to the cryptic names. Most notable is Elastic Beanstalk - had no idea that was a PaaS!
5
velodrome 3 days ago 0 replies      
GCP does not really need one of these. It is a lot easier to understand. The only time it gets confusing for people are the papers or projects they were based on (e.g. StackDriver, BigTable, etc).
6
tobych 2 days ago 0 replies      
This answers one of my big pet-peeves, which is the useless word salad written by marketing pinheads that you have to decypher whenever you're comparing products or evaluating which tier of some service that you need, e.g.: "Upgrade from Basic to Ultimate if your stakeholders need to leverage content analytics and optimize dynamic competencies". No customer can ever sue you for misrepresentation if no one can ever quite figure out what you were claiming your software could do.
7
dsmithatx 3 days ago 0 replies      
I would highly recommend this resource for further reading.

https://github.com/open-guides/og-aws

8
tharibo 3 days ago 1 reply      
Do we have the same for Microsoft products?Even as a developer, I can't understand half of what they're proposing. Like what the hell is Sharepoint anyway?

Or what is SAP?

9
ankurdhama 3 days ago 1 reply      
Does anyone know who are the bunch of geniuses that come up with the names and decide upon what to use? World deserve to know about them.
10
noer 3 days ago 1 reply      
It's worth noting on the SES explanation it says:

>You could use it to send a newsletter if you wrote all the code, but that's not a great idea.

You actually can use a self hosted solution like Sendy to send marketing emails & newsletters via SES & only pay for the emails you send using SES

11
cygned 3 days ago 2 replies      
Still wondering why AWS does not provide a solid PaaS solution like Heroku does (they are on AWS, though) - or am I just overlooking it? I would like to host a few node.js/Clojure apps but I don't want to have the hassle with virtual machines/IaaS.
12
z3t4 2 days ago 0 replies      
13
dangle 3 days ago 0 replies      
Thanks. <3 Can you also rewrite all tutorials please?
14
sukruh 3 days ago 1 reply      
It would be cool if someone did a similar thing for the Apache Big Data projects.
15
roomey 3 days ago 0 replies      
Wonder what the vmware service on Amazon will be called - maybe Elastiware
16
frostymarvelous 3 days ago 0 replies      
This is very annoying to read on my S6 in chrome 55.0.2883.91
17
a012 3 days ago 1 reply      
> Code Deploy> Should have been called: Not bad

The one service name that's self declared.

19
kasparsklavins 3 days ago 1 reply      
Broken on mobile.
20
dustinmoris 3 days ago 0 replies      
Someone needs to teach expeditedssl.com how to build a website in 2017 that can also be consumed on mobile without content being cut out.
21
howfun 3 days ago 0 replies      
Machine LearningShould have been calledSkynet
22
slightlyCyborg 3 days ago 1 reply      
Updoot, because I was thinking just the other day how absurd Amazon's naming scheme is. Did an engineer think of that sh#!? If Amazon was run by Musk instead of Besos, a harsh email would have been sent to the employees to cut that sh#! out.

Source: Acronyms seriously suck https://twitter.com/davejohnson/status/602951117413216256

9
How to Avoid a Post-Antibiotic World nytimes.com
402 points by jseliger  1 day ago   262 comments top 29
1
lbarrett 1 day ago 4 replies      
The overuse of antibiotics in large-scale animal farming is particularly terrible; many are given low doses of antibiotics as part of their food [1]. This gives bacteria lots of time and selective breeding to become resistant, and it's probably the worst thing we could do for the long-term health of humanity.

[1] https://en.wikipedia.org/wiki/Antibiotic_use_in_livestock#Un...

2
milesf 1 day ago 8 replies      
From this website: http://seqclinic.com/chinese_medicine.html

 Historically, a Chinese Medicine doctor was paid a retainer to keep their patients healthy. If a patient became sick, the doctor would not be paid until the patients health returned. In a similar vein, a doctor that resorted to surgery was considered an inferior doctor. If he/she did their job correctly and helped their clients stay healthy, there would be no need to perform surgery.
Perhaps the current incentive for drug companies is the problem. They don't care if we stay healthy (which is what we all want), they only treat illness. In fact you could argue it's in the best interest of drug companies to keep us sick!

Maybe there's an alternative way to fund them, or an alternative way to deal with infections in the first place.

3
csl 1 day ago 0 replies      
Alexander Fleming gave the following warning, way back in his 1945 Nobel speech:

 It is not difficult to make microbes resistant to penicillin in the laboratory by exposing them to concentrations not sufficient to kill them, and the same thing has occasionally happened in the body.
From page 93 in https://www.nobelprize.org/nobel_prizes/medicine/laureates/1...

BBC did a very good radio segment about the penicillin discovery in their "50 Things That Made the Modern Economy" radio series: http://www.bbc.co.uk/programmes/p04pfn2z

4
jakobegger 1 day ago 3 replies      
A few important points about antibiotic resistance (as far as understood by a lay person that likes to read articles on the topic):

1) Resistance is inevitable. It doesn't matter if everyone finishes their prescribed treatment or not, bacteria will develop resistance to antibiotics one way or another.

2) Antibiotic resistance comes at a cost to the bacteria. In the absence of antibiotics, bacteria will lose their resistance. It is pretty unlikely that you will get infected with antibiotic-resistant bacteria when you bruise your knee in the dirt.

3) Multi-resistant bacterial infections mostly occur in a clinical setting, where people are especially vulnerable to infections (people on a respirator, people with a central venous line). You can prevent multi-resistant infections just like you prevent normal infections: Use sterile gloves, isolate people, follow procedure protocols precisely, etc.

Multi-resistant pathogens are a problem, but it's far from the doomsday scenario painted in those "it's the end of antibiotics" articles.

5
gleb 1 day ago 1 reply      
We could use a solution from electricity markets.

Electricity generation companies are paid not just for the electricity they produce, but also for reserve capacity. Because that's the only way to keep the lights on reliably.

Seems like this would work for antibiotics and snake antivenoms.

http://www.theenergycollective.com/adamjames/237496/energy-n...

6
halestock 1 day ago 5 replies      
In its apparent desire to blame Congress for the end of antibiotics, this article misses a big reason companies aren't developing new antibiotics anymore: they just don't work like they used to. Every new generation of antibiotic is effective for a shorter period than the previous.
7
cstejerean 1 day ago 2 replies      
Time to stimulate research into phage therapy instead of granting longer patents to antibiotics. The field seems promising with a long history in Russia but mostly ignored in the west until recently. https://en.m.wikipedia.org/wiki/Phage_therapy
8
sengork 1 day ago 0 replies      
For those that would like to visualise how the bacteria vs anitbiotics evolve in an experiment over 11 days, have a look at this video from Harvard:

https://www.youtube.com/watch?v=plVk4NVIUh8

9
Animats 1 day ago 0 replies      
The parent article is by two lawyers. Here's a better introductory article by a biochemist.[1] A key point is that there are a finite number of small molecules which can potentially be used as antibiotics. Throwing money at finding them may not work against a depleting resource.

[1] http://fire.biol.wwu.edu/cmoyer/zztemp_fire/biol345_F10/pape...

10
nradov 1 day ago 3 replies      
In evolving resistance to multiple antibiotics do bacteria tend to become weaker or less fit in other ways? I would think there must be some trade offs involved if they can no longer use certain molecules in their cell walls or metabolic pathways. Or to put it another way, if there are resistant and non-resistant bacteria in a particular environment will the non-resistant population tend to out compete the resistant population in the absence of antibiotics?
11
whack 1 day ago 2 replies      
The article seems to have thrown in the towel with regard to existing antibiotics, which seems like a waste since there's so much more we could do. Restrict antibiotics the same way we restrict morphines. Censure doctors who over prescribe antibiotics, when they aren't needed. Do not allow them to be used in animal feed for any reason. We're talking about a vital resource that can save millions of lives. Allowing this resource to be depleted by frivolous use, is downright criminal.
12
lunchladydoris 1 day ago 3 replies      
At the individual level, I've always thought of antibiotic misuse as (at least in some part) a symptom of our resistance to authority. People end their antibiotic course early (and then usually use the remaining pills another time) because, of course, they know better. I've run into people like this many times.

Now, perhaps doctors should take part of the blame for not explaining why you should finish your course of antibiotics, but users should take some of the responsibility too.

13
botexpert 1 day ago 1 reply      
It's such a shame we used them all on livestock. We raise around 60 billion land animals every year and probably more than 90% is being given last-line-of-defense antibiotics for faster growth. Perfect pool for some unbelievable evolution of antibiotic resistant bacteria, and yet we want to ban antibacterial soaps and other minor things.

But, at least the stakes and muttons and whatevz can be enjoyed.

14
Ericson2314 1 day ago 0 replies      
Honestly drug bounties seems so much better than patents across the board. Patents are better when you're inventing the market too, but even then are no panacea as it may take a long time to build the market after the initial invention, eating into the patterns lifespan. True innovative is often bewildered.

Here not only is the market clear cut, but this benefits new entrents as the immediate windfall allows for more risk-taking. It's liquidity for innovation.

Finally, the costs for patients / emergencies can be dirt-cheap, which is good because just as one should have insurance to cover future emergencies, so should society budge the R&D up-front.

15
toodlebunions 1 day ago 1 reply      
This is a global health crisis waiting to happen. Incentives need to be created for pharma to create many new broad spectrum antibiotics. Longer patents, financial or tax incentives, even subsidies, whatever it takes.
16
umberway 1 day ago 1 reply      
There may be a lot of antibiotics present in the soil which haven't been exploited yet because it's hard to grow in the lab the organisms which produce them. However techniques are being developed to circumvent this problem and here's an early discovery:

https://en.wikipedia.org/wiki/Teixobactin

Hopefully other and fundamentally newer ways to fight infection will be found. In the mean time getting fit and healthy now seems like a slightly wiser choice than it already was.

17
jvanderbot 20 hours ago 0 replies      
https://www.hhs.gov/about/news/2016/07/28/hhs-forges-unprece...

HHS has a joint initiative with UK-based research foundations.

I really wish an easy non-profit foundation existed that I could throw some change at.

Here's some more information: http://www.openphilanthropy.org/research/cause-reports/antib...

18
amorphid 1 day ago 1 reply      
It'll be interesting to see if/when a non-antibiotic solution to killing bacteria comes along. I read about an approach a novel approach a couple months ago that uses a polymer to destroy bacteria.[1] I'm not qualified to comment on the merits of the science.

[1] http://inhabitat.com/student-discovers-a-way-to-destroy-supe...

19
jahbrewski 1 day ago 5 replies      
Okay, I've read enough articles on superbugs to be seriously frightened. My question: is there anything I, personally, can do to help?
20
marktuff1 17 hours ago 0 replies      
Doesn't it strike anyone that perhaps of constantly trying to kill everything in sight that perhaps we should figure out a way to add super pro-biotics to our micro-biome to create little beneficial bacteria armies to fight the war against bugs instead of nuking our whole flora which then results in zero immunity and ability to fight anything off.
21
grahamm 1 day ago 1 reply      
The problem is anti-biotics are no longer being developed. They could be developed but it is not finicially worth a large pharma to do so.

Consider this, development takes years (>10) and costs billions due to failed attempts (over 9 in 10 fail), testing, etc. They then go to market and no government will buy them because the price is high to recoup the billions spent developing over ten drugs which only one made to market. In the end the governments force the companies to sell them at a loss should there be an epidemic. Then to cap it all some company in another part of the world where IP is not honoured rips off the drug and floods the most needed places with a cheap knock off.

Where is the incentive for a pharma to go through this.

22
mtdewcmu 12 hours ago 0 replies      
An even more direct way for the government to push discovery of new antibiotics would be for the government to do the R&D itself. Of course, that would offend certain ideologues.
23
necessity 1 day ago 0 replies      
The woman died because the antibiotic that could save her life was not approved in the US by the FDA, not because there was no anti-biotic that could possibly save her.

https://www.theatlantic.com/health/archive/2017/01/a-superbu...

24
chrisgd 1 day ago 0 replies      
How are we able to create vaccines against bacteria and why couldn't we do it for more strains i.e.,MRSA?

My son spent some time in the hospital fighting a pneumonia. Now his doctors are looking into why his body didn't get immunity from the pneumococcal vaccine (Prevnar13). I was always under the impression only viruses could be immunized against.

25
thephyber 21 hours ago 0 replies      
This was the subject of this week's "50 Things that Made the Modern Economy", a BBC podcast:

http://www.bbc.co.uk/programmes/p04pfn2z

26
vivekd 1 day ago 0 replies      
>Although the patent system is good at producing new blood-pressure medications and cardiovascular drugs, its not the right fit for antibiotics.

I don't know if I want to get into an arms race with microbes where we keep trying to discover new antibodies only to have them develop resistance. Maybe the better solution is to limit antibiotics only to life threatening situations and let people build up their natural resistance.

27
known 1 day ago 0 replies      
Can we use a virus that eats/kills that bacteria and later we'll kill that virus.
28
diminoten 1 day ago 1 reply      
I've never really bought these doomsday scenario claims, if only because I feel like innovation is constantly taking place, and these projections necessarily assume no new innovations (how could they predict innovation, after all?).
29
adventured 1 day ago 2 replies      
This premise is going to turn out to be entirely incorrect.

There's a wave of therapeutic approaches to antibiotic resistance coming in the next 10-15 years, courtesy of CRISPR (gene editing broadly). We're going to end up having hundreds of new experimental angles of attack at resistance and infection. In fact, by far the bigger problem, is going to be narrowing down the vast array of options of attack that CRISPR is going to unleash.

These articles repeating the same hyped up fear, are missing what's right around the corner (which usually happens with such statements of doom, ala the world running out of food claims from decades ago). And best of all it's going to be inexpensive, relatively speaking, and extremely fast paced, to make progress in that direction. Cas9 and its superior alternatives such as Cpf1 arrived just in time.

10
Stepping into math: Open-sourcing our step-by-step solver socratic.org
548 points by shreyans  4 days ago   174 comments top 17
1
analog31 3 days ago 8 replies      
This seems interesting because it addresses the issue of "show your work." Many years ago, I spent a semester teaching the freshman algebra course at the nearby Big 10 university. This is the course that you take if you don't get into calculus. My students were bright kids -- they were all admitted to the state flagship school -- but not mathematicians.

There was huge variation in the preparation that kids brought with them from high school. In particular, very few of them understood what "show your work" means. They were told "show your work," but nobody told them what it really entails. Is it just to provide evidence that you did some work, to deter cheating, or is it something else? Many of my students were taught "test taking skills" such as the guess-and-try method. So on one exam, a question was:

x^3 = 27

One student's work:

1^3 = 1

2^3 = 8

3^3 = 27

Answer = 3

I asked the professors to tell me what "show your work" means. None of them had a good answer! These were the top mathematicians in the world. I wanted to talk with my students about it, but I'm not even sure that my own answer was very good.

But if we did well in math, then we just know what it means. It's not just evidence that you did the work. It doesn't mean "turn in all of your chicken scratch along with the answers." It means something along the lines of supplying a step-by-step argument, identifying the premises and connecting them with the conclusion, in a language that is "accepted," i.e., that mimics the language of the textbook / teacher. In fact, the reason to read the textbook and attend lectures, is to learn that language. (It's not so different in the humanities courses).

At least, that's my take on it, as just one teacher with one semester's worth of experience.

In my view, a problem solving tool that actually addresses the process of building the argument and not just determining the answer, would be beneficial to students.

2
tgb 4 days ago 7 replies      
Has anyone done a study to see if this kind of aided solving actually helps students learn? I'm worried that "Eh, I'll just write this solution down today, I'm sure I'll learn it tomorrow" is what's happens.

Awesome software though.

3
jorgemf 4 days ago 1 reply      
Some years ago I tried to do something a bit more complex: http://telauges.appspot.com/mathsolver/

My idea was to use planning and A* search to solve any type of math problem, even create probes for things like the quadratic equation https://en.wikipedia.org/wiki/Quadratic_equation . I gave up after learnt the search space was so big for it that it was impossible to solve. If I had to do it today I will explore deep learning as heuristic, but I think it probably wont work.

I always like to see this type of projects, I hope they succeed where I failed.

4
yequalsx 3 days ago 2 replies      
It's a nice program and I can see it being both helpful and harmful. From my perspective, as a teacher of mathematics at a community college, students are unwilling to engage in thought about a problem. If they can't see the solution in a few minutes then they want to look at a complete solution. Mostly they are not willing to struggle through a problem.

I vacillate on whether, with the advent of computer algebra systems, it is necessary for students to master algebraic manipulations. I started to think that conceptual questions are better.

For instance, give me an example of an equation with no solution. Explain how a baseball player can have the highest batting average the first half of a season and in the second half of a season but not have the highest overall average. Draw the graph of a function defined on [0, 1] but has not maximum or minimum.

Students can't do those types of problems either. They are very frustrating problems for students because it requires you to really think about what the words mean and to think of extreme situations. So I've reverted back to the traditional style of teaching math. Manipulation of symbols.

5
stdbrouw 3 days ago 0 replies      
Worked on something like this as a hobby project a while ago, but to avoid the complexities associated with solving arbitrary exercises, instead I had it set up as an algebra exercise generator: you start with the solution, which you then (algorithmically) obfuscate by splitting terms and recombining things for a couple of rounds. Never got around to finishing it, but the neat thing is that you've already generated one possible way to solve the problem, it's just how you generated the exercise in reverse.

Another thing that's quite easy to do is to check intermediate steps in a solution for equivalence. You don't even really need CAS, just brute force the problem by probing the equations: set all variables to randomly chosen values, n times and if the sets of results are the same for both equations, you're good.

Anyhow, Socratic looks great and a great deal more advanced and useful than what I came up with, so kudos!

6
benbristow 4 days ago 4 replies      
I'm jealous of kids these days... homework would've been so much easier with this.

You could always use a calculator but the whole 'show your own working' catch meant you had to do it all manually. Not any more!

7
therealmarv 4 days ago 1 reply      
Does anyone know if there is a good open source library for making equations (Latex, MathML) out of pictures like in their demo?
8
equalunique 3 days ago 0 replies      
My academic math journey stopped at pre-calc, and I had been a C student for quite a long time. HS Algebra II would never have happened for me if I hadn't discovered XMaxima, an emacs-based CAS. Fortunately I took a Discrete Math course before dropping out of college, and it gave me a new admiration for math.

In spite of my weak math background, this has been the most enjoyable comments section on HN I've read so far.

9
MichaelBurge 3 days ago 1 reply      
People here keep saying this will change learning and be good for the students, but the only real difference is it's open-source. You can already get step-by-step solutions for more types of problems from Wolfram Alpha, and you can already get API access if you're a 3rd-party developer who needs it:

http://www.wolframalpha.com/input/?i=2*y+-+x+%3D+(8+*+x+%2B+...

I don't think it will have any real effect.

10
chriswarbo 3 days ago 0 replies      
Very interesting work, and well-explained in the post.

Like many others here, I suppose that in it's basic form this would mostly be used for cheating on homework; although it would certainly be useful for those (few?) students who are truly motivated to self-learn the material, rather than just pass the tests.

One thing which springs to mind is "Benny's Conception of Rules and Answers in IPI Mathematics" ( https://msu.edu/course/cep/953/readings/erlwanger.pdf ), which shows the problem of only focusing on answers, and on "general purpose" problem sets. Namely that incorrect rules or concepts might be learned, if they're reenforced by occasionally giving the right answer.

I think it would be interesting to have a system capable of some back-and-forth interactivity: the default mode would be the usual, going through some examples, have the student attempt some simple problems, then trickier ones, and so on.

At the same time, the system would be trying to guess what rules/strategies the student is following: looking for patterns, e.g. via something like inductive logic programming. We would treat the student as a "black box", which we can learn about by posing carefully crafted questions.

Each question can be treated as an experiment, where we want to learn the most information about the student's thinking: if strategies A and B could both lead to the answers given by the student, we construct a question which leads to different answers depending on whether A or B were used to solve it; that gives us information about which strategy is more likely to be used by the student, or maybe the answer we get is poorly explained by A and B, and we have to guess some other strategies they might be using.

Rather than viewing marking as a comparison between answer and a key, we can instead infer a model of the domain from those answers and compare that to an accurate model of the domain.

We can also use this approach the other way around, treating the domain as a black box (which it is, from the student's perspective) and choosing examples which give the student most information about it.

11
Steeeve 4 days ago 1 reply      
Now... the only thing remaining is to translate this to common core :).

I say that in jest, but doing so would make common core much easier for parents AND teachers to grasp. There's an enormous divide between those who get it and those who hate it, and providing parents/teachers with something that would help them understand the benefits of common core concepts would be a gigantic win.

12
aidos 4 days ago 0 replies      
That's so cool.

Reminds me of how different the learning experience is now. When we were at school (80s/90s), there was nowhere to turn if you didn't have the answer. My parents had an Encyclopedia Britannica set, so at least there was a paragraph to go on. It's amazing how good you became at fleshing out that paragraph into an essay :-)

13
gravypod 4 days ago 1 reply      
Now that this exists I think it's worth creating an opensource version of the TI-Nspire for engineers & mathamaticians. Something based on cheap hardware, runs linux, and can implement this + a theorum prover to basically make the most handy lab calculator.
14
poseid 3 days ago 0 replies      
that feels like a nice application of AI in a way. we often use a computer that can help in making a plan (e.g. a kind of map or "steps" as here). this might be nice to help understand problem solving in general. also, nice to see the project is in javascript, that means quite a few non-professional programmers could learn from it.
15
JotForm 4 days ago 0 replies      
This is such an inspiring software.
16
StefanKovachev 4 days ago 1 reply      
17
GrumpyNl 4 days ago 0 replies      
It looks like Sheldon came through.
11
The Awk Programming Language (1988) [pdf] archive.org
398 points by dang  1 day ago   95 comments top 31
1
nprescott 1 day ago 2 replies      
One of my favorite books - I initially bought a copy based on a review by Brandon Rhodes [0]:

> But the real reason to learn awk is to have an excuse to read the superb book The AWK Programming Language by its authors Aho, Kernighan, and Weinberger. You would think, from the name, that it simply teaches you awk. Actually, that is just the beginning. Launching into the vast array of problems that can be tackled once one is using a concise scripting language that makes string manipulation easy and awk was one of the first it proceeds to teach the reader how to implement a database, a parser, an interpreter, and (if memory serves me) a compiler for a small project-specific computer language! If only they had also programmed an example operating system using awk, the book would have been a fairly complete survey introduction to computer science!

[0]: http://stackoverflow.com/a/703174/2912179

2
david-given 1 day ago 5 replies      
I wrote a compiler in awk!

To bytecode; I wanted to use the awk-based compiler as the initial bootstrap stage for a self-hosted compiler. Disturbingly, it worked fine. Disappointingly, it was actually faster than the self-hosted version. But it's so not the right language to write compilers in. Not having actual datastructures was a problem. But it was a surprisingly clean 1.5kloc or so. awk's still my go-to language for tiny, one-shot programming and text processing tasks.

http://cowlark.com/mercat (near the bottom)

(...oh god, I wrote that in 1997?)

3
luckydude 13 hours ago 0 replies      
I've got the source code to both the book (in English and French) as well as awk. How? I sent email to bwk that we were trying extend awk to be sort of threaded (think awk scripts as first class so you have awk foo { } awk bar { } and you could do foo | bar). We called it bawk, BitMover's awk.

Anyhow, I asked Brian if we could base it off the one true awk and he tarred up ~bwk/awk and sent it to me.

I love that guy, the culture of the Bell Labs people and the people that worked with them is great.

I've stolen a bunch of awk ideas over the years. BitKeeper (first DSCM) has a programming "language" for digging info out of the repository. For example, this:

http://www.mcvoy.com/lm/bkdocs/dspec-changes-json-v.txt

prints out the repo history as a json stream. One of my guys said that it couldn't be done, heh, it could be :)

Everyone should learn some awk, it's so handy.

4
chubot 1 day ago 2 replies      
Last year I dug up Kernighan's 2012 release of awk, fixed up the test suite packaging and automated it, and wrote a makefile which adds clang ASAN support.

It found a couple bugs because the test suite is quite comprehensive. I think it's somewhat interesting that 5000 or so lines of C code polished over 20 years still has memory bugs.

I didn't fix the bugs, but anyone should feel free to clone it and maybe get some karma points from Kernighan. Maybe he will make a 2017 release. He is fairly responsive to email from what I can tell :)

https://github.com/andychu/bwk

5
patrickg_zill 1 day ago 0 replies      
I was working with a VOIP startup, and they needed to find some unique numbers in their CDR's (call detail records, basically a CSV list of calls made, duration, etc.) .

Loading the file into Excel took literally minutes as Excel tried to parse every field. It bogged down a 16GB RAM machine.

Using awk and uniq, the total run time of getting a solution , including reading the many MB of files and generating a summary into another file, was about 6 seconds.

6
vram22 1 day ago 2 replies      
One of my commonly used Unix one-liners, using awk, is to get the sum of the file sizes for the files listed by the ls command (with the -R for recursive option if wanted):

ls -lR /path/to/dir | awk ' { s += $5 } END { print s / 1024 " K" } '

$5 is the 5th field of the output, which is the file size field in the case of ls output. The code inside the first set of braces runs once for every line of input (which comes from standard input, so from the ls command, in this case), and the code inside the second set of braces runs at the end of the input, calculating and printing the desired result of the total of all file sizes for files found by ls, in kilobytes. It can easily be changed to output the total in bytes or megabytes by dropping the '/ 1024' or adding another one after the first. Variable s is initialized to 0 by default at the start.

You can get similar info with "du -hs /path/to/dir" but the ls plus awk pipeline lends itself to more customization, such as adding conditions for the type or owner of the file, etc.

7
cagey 1 day ago 3 replies      
PolyAWK (by Polytron), which included a copy of this book with each unit sold (see sticker on the cover of the linked PDF), was a _favorite_ tool of mine "back in the DOS (and early Windows) days". It was IIRC developed by Thompson Automation Software[0], who later sold the software package directly. The Thompson Automation Awk package included an awesome _awk compiler_, allowing creation of standalone .EXE files (using a 32-bit DOS extender, and later a Win32 version) from 1+ awk source files. The compiler presumably generated bytecode which was bundled into the .EXE file along with a 32-bit runtime which provided data capacity sufficient for a wide range of real-world projects. Anyway, TAWK gave me a huge productivity boost for a number of years during a time when such languages were only beginning to become available on the PC platform. And the ability to create single-file standalone EXE files greatly eased distribution of the tools I created. Good times.

[0] http://www.tasoft.com/

8
technofiend 1 day ago 0 replies      
Since this is Hacker News my plea may be answered: does anyone have the artwork or an actual example of the infamous AWK T-shirt? From memory it features a bird jumping (prachuting) out of a plane and is titled with AWK's most famous error message: "awk: bailing out near line 1."
9
lucidguppy 1 day ago 1 reply      
This book should be required reading for anyone looking to write their own tech books.

It's short, clear, and concise. It's useful and helps you solve real problems with AWK. Who could ask for anything more?

10
dang 1 day ago 0 replies      
Plain text version here, but the formatting is off in places: https://archive.org/stream/pdfy-MgN0H1joIoDVoIC7/The_AWK_Pro....
11
jph 1 day ago 0 replies      
Awk is the #1 language I learned this year for fun.

I wrote a simple command line statistics tool that uses awk to calculate sum, stddev, and more. https://github.com/numcommand

12
bglazer 1 day ago 1 reply      
I wish that certain simple tasks in awk were a little less verbose, especially for command line use.

The number one example for me is counting by string in a csv file:

>> awk -F',' '{a[$1] +=1} END {for(v in a) print v,a[v]}'

Not that this is particularly difficult stuff, it's just a bit exhausting to find myself typing that over and over again. I'd love a more concise alternative to this.

Also, 'sort | uniq -c' is not a viable alternative for very large files.

13
carlisle_ 1 day ago 1 reply      
People are often surprised when I mention that awk is Turing complete. It's quite a powerful tool, I can't imagine loving the command line as much as I do without it.
14
joepvd 1 day ago 0 replies      
I love awk for text processing purposes. When analyzing log files, I often drop down into awk-mode to check the exceptional constellation that is currently under investigation. Very powerful to be able to say after three minutes: This happens in 0.5% of the cases.

Bought this book 2nd hand online. This book on one day costs $150, and on the next $2. The first bit has been an awesome read, never got to read much more. Tend to read much more from $READER. Sure this PDF will get me going again!

15
iconara 21 hours ago 1 reply      
It triggers my OCD that the names of the authors are in alphabetical order on the cover and not in, you know, the logical order.
16
gallerdude 1 day ago 3 replies      
Man, I need to study some more weird languages. Just got done with the basics of Python and C for my first CS class. Over the summer I want to tackle LISP.
17
banku_brougham 21 hours ago 0 replies      
Are the /AA and /ObjStm items a concerning indicator? This is the limit of my familiarity with pdf-id:

 > python2.7 pdfid.py The_AWK_Programming_Language.pdf PDFiD 0.2.1 The_AWK_Programming_Language.pdfPDF Header: %PDF-1.6../Page 0/Encrypt 0/ObjStm 7/JS 0/JavaScript 0/AA 1/OpenAction 0/AcroForm 0/JBIG2Decode 222...
It has /AA which is an automatic load action, and it has a lot of objects which could contain javascript, would need closer scrutiny I think.

18
zoom6628 1 day ago 0 replies      
Used AWK a whole lot in early 90s for massaging source code. Mostly to analyse and refactor 1m+ LOC of COBOL. And Awk was brilliant for that. Have used it ever since when needed to text process. Around 2000 was using it a lot to get convert systems by running reports on old system and then getting the data from output text files. Clunky way to do it but faster than typing when there is no way to get the data directly. If a system can print to a text file then the data is available. Use awk still on Windows, OSX and Linux. Its an essential tool when faced with string/text processing tasks.
19
kazinator 1 day ago 0 replies      
Awk as Lisp macro in TXR:

http://www.nongnu.org/txr/txr-manpage.html#N-000264BC

It has direct counterparts to all POSIX features, plus a number of extensions similar to ones found in Gawk, as well as some of its own: for instance, range expressions which freely combine with other expressions (including other range expressions), and range expressions which exclude either or both endpoints.

20
getpost 19 hours ago 0 replies      
I used awk until I learned Python (long ago). For me, awk was yet another example of the "worse is better" approach to things so common in unix. For example, if you make a syntax error, you might get a message like "glob: exec error," rather than an informative message. "Worse is better" is probably a good strategy in business and for getting things done, but still, mediocrity and the sense of entitlement that so often goes with carelessness, sickens me.
21
contr-error 1 day ago 1 reply      
Is there anything that "explains" sed only half as well as this book? I know how to use basic sed, but haven't yet completely grokked the way pattern space and hold space really go together.
22
nat 23 hours ago 0 replies      
Awk meshes very well with a lot of my natural inclinations about text processing. I've sadly stopped using it lately as it seems that the majority of my use cases these days run up against a (to me) glaring deficiency in the language. Specifically, capture groups in pattern regexes. It's probably one of those "you're doing it wrong" kind of things, but if awk had that one feature, I probably wouldn't ever need to use perl.
23
sstanfie 1 day ago 0 replies      
Literally writing a small awk script, took a break to check Hacker News. Nice.
24
qwertyuiop924 1 day ago 0 replies      
AWK is still my go-to scripting language for quick tasks, like simple computation and basic data analysis. It is still the best thing in its problem space.

Given, AWK's problem space is very small, but still...

25
mcintyre1994 1 day ago 2 replies      
I've been learning this at work as part of a get-good-at-Linux regime :) One of the most surprising things for me is that as horrible to a beginner that some of the one liners in the command line can look, it's actually quite a forgiving scripting language. I don't think I've seen another language where you can increment a variable without declaring/initialising it, nor where you can set indices on an array without it being declared (except in a constructor fashion I guess).
26
gwu78 1 day ago 0 replies      
Not sure about GNU, but BSD build systems depend on AWK for building installation media.

crunchgen, a compiled C program, has to call AWK.

Anyone out there do AWK-less builds?

Why did I need to learn a little AWK?

Because I could work out how crunched binaries were built without knowing some AWK.

Best thing about AWK IMO is the C-like syntax.

For anyone learning C and AWK concurrently, this kills two birds with one stone.

27
elchief 1 day ago 1 reply      
28
kworker 16 hours ago 0 replies      
The printed book still expensive on amazon. I guess it's still important these days.
30
throwaway7645 1 day ago 3 replies      
I love Awk on Unix. I really wish Windows had something closer to this.
31
michaelsbradley 1 day ago 0 replies      
Robbins' open source book may be of interest as well:

GAWK: Effective AWK Programming

https://www.gnu.org/software/gawk/manual/gawk.pdf

12
United Arab Emirates goes from 10k Tor users to 250k in days torproject.org
354 points by temp  1 day ago   72 comments top 21
1
benjojo12 1 day ago 1 reply      
I believe that tor metrics counts when a connection starts, not when a connection is _established_

This is a important difference, because if there is active DPI that is shutting down a connection before handshake can happen, it will inflate the numbers massively.

I suspect what actually is happening is a ISP in UAE has deployed a DPI system that can detect the Tor TLS signature

2
indice 1 day ago 3 replies      
The same rise was seen in Turkey last month. The phenomenon is caused by failed reconnect due to DPI-based censorship. See Annex A: https://turkeyblocks.org/2016/12/18/tor-blocked-in-turkey-vp...
3
blunte 1 day ago 2 replies      
What this really indicates, bot or not, is that once you educate people, they will act in their self (and more importantly, self+others) interests. Oppressive and controlling (controlling or controling, I can never decide) regimes will try to prevent it. But it is like trying to prevent wind.

The wind will come. You must adapt and accept. And if you are against the wind, you must change.

4
omginternets 1 day ago 2 replies      
Any chance this could be a state-sponsored attack aimed at correlating traffic?
5
Raed667 1 day ago 1 reply      
This also happened in Tunisia in 2013 [0]. We believe that it was a bot. [1]

[0] : http://imgur.com/a/mjYsP

[1] : http://gizmodo.com/the-anonymous-internet-is-under-attack-12...

6
falloutx 1 day ago 1 reply      
The same graph with censorship events on: https://metrics.torproject.org/userstats-relay-country.html?...

Just on the start the spike, there are many events. Though I don't know how to find information on those events.

7
Asdfbla 1 day ago 4 replies      
Just out of curiosity: How is Tor looking these days, security-wise? Does someone have a recent analysis of the attacks Tor is facing from state-level attackers currently? Just wondering if any new threats to Tor have come up in the recent years that hadn't been considered before stuff like Snowden happened.
8
rmela 1 day ago 0 replies      
Looks like the UAE has outlawed use of torque, and is also using DPI to block it, resulting in inflated numbers due to dropped connections and ensuing attempted reconnects.

https://trac.torproject.org/projects/tor/ticket/6246

9
Jeaye 1 day ago 1 reply      
First thing that came to my mind was a botnet; it would be one of the easiest ways to get a huge spike in Tor usage, I'd think.
10
libeclipse 1 day ago 1 reply      
I can't find anything blatant in the news that would explain something like this, especially of this magnitude.

I think it might be a botnet or something similar, although that's just conjecture at this point.

11
elastic_church 1 day ago 1 reply      
The economic incentives over TOR have really improved TOR

I was pulling 800k/sec the other day, pretty surprised.

Some circuits are still slow. But I remember not that long ago (18 months?) it was a miserable expereince

12
nullrouten 1 day ago 0 replies      
It's possible that the geoIP records for a large IP block or set of IP blocks has been corrected (or broken) to reflect UAE.
13
ajaimk 1 day ago 0 replies      
Protonmail added support for TOR this week but that can't be it
14
k-mcgrady 1 day ago 0 replies      
The linked page doesn't seem to have any info other than the stats. Can someone explain the reason for the spike?
15
TheSageMage 1 day ago 0 replies      
If this is an attempt by the UAE to prevent TOR connections via DPI, who would the primary target(s) be? I recognize that's an awkward question to ask of an anonymized service like TOR, but who are the actors in the UAE who might use TOR and why target them now?
16
foota 1 day ago 2 replies      
Could this be a result of the articles about a "backdoor" in whatsapp?
17
aarontyree 1 day ago 0 replies      
Unless the massive uptick in Tor client connections can be correlated to a massive uptick in Tor client downloads its not a societal event and is more likely government sponsored.
18
farrokhi 1 day ago 0 replies      
Perhaps they accidentally lifted the blocking rules. Or it will drop as soon as they upgrade their censorship software.
19
baybal2 1 day ago 2 replies      
Did they block pr0n there?
20
anaccountwow 1 day ago 0 replies      
It must be something that was posted on hacker news lately!
21
SCAQTony 1 day ago 0 replies      
Not that is possible to detect gender but I suspect the bulk of those users are female since they are the most repressed. http://www.thenational.ae/business/telecoms/uae-top-for-fema...
13
How Discord Stores Billions of Messages Using Cassandra discordapp.com
430 points by jhgg  3 days ago   154 comments top 23
1
niftich 3 days ago 2 replies      
These kinds of write-ups offer valuable insight into a popular project's requirements and decision-making, and are some of the most instructive resources one can find: these show not only the kinds of challenges one has to face at scale, but also how architectural choices are made.

It's far more valuable to understand why Discord uses Cassandra than to merely be aware they do.

Out of curiosity, did you consider HBase and Riak? Did you entertain going fully hosted with Bigtable? If so, what criteria resulted in Cassandra winning out?

2
jakebasile 3 days ago 5 replies      
I use Discord a fair amount, and something that annoys me about it is that everyone has their own server.

I realize this is a key part of the product, but the way I tend to use it is split into two modes:

- I hang out on a primary server with a few friends. We use it when we play games together.

- I get invited to someone else's server when I join up with them in a game.

The former use case is fine but the latter annoys me. I end up having N extra servers on my Discord client that I'll likely never use again. I get pings from their silly bot channels (seemingly even if I turn notifications off for that server/channel), and I show up in their member lists until I remove myself.

I wish there was a way to accept an invite as "temporary", so that it automatically goes away when I leave or shut down Discord. Maybe keep a history somewhere if I want to go back (and the invite is still valid).

Aside from that, it's a great product and really cleaned up the gamer-focused voice chat landscape. It confuses me that people will still use things like TeamSpeak or (god help you) Ventrilo when you can get a server on Discord for free with far better features.

Now that I posted this, I realize this has little to do with TFA. Sorry.

edit: formatting, apology

3
ve55 3 days ago 3 replies      
Discord seems to me like it has a very polished user experience, and it's no surprise that users are trashing programs like Skype in favor of Discord when it is better in every area.

Discord seems to take security seriously, as they should, but I'm curious about their stance on privacy and openness.For example, I wonder if they would consider:

- Allowing end-to-end encryption to be used between users for private communications

- Allowing users to connect to Discord servers using IRC or other clients (or, at least having an API that easily allows this)[1]

- Allow users to have better control over their own data, such as providing local/downloadable logs so that they can search or otherwise use logs themselves

Discord is definitely succeeding within the gaming market, but I'm curious what other markets they would like to take a stab at.

[1] I'm aware Discord has an API, but if I understand it correctly, normal users cannot easily use Discord from anything other the official Discord apps, as this API is specifically for Discord 'bots'. I see there's a discord-irc bridge, but not much more than that. I may be incorrect on this.

4
pilif 3 days ago 3 replies      
> While Cassandra has schemas not unlike a relational database, they are cheap to alter and do not impose any temporary performance impact

in most relational databases, the schema is cheap to alter and does not impose a temporary performance impact.

In-fact, all of their requirements (aside of linear scalability) could also be met with a relational database. Doing so would gain you much more flexible access to querying for various reports and it would reduce the engineering effort required for retrieval of data as they add more features (relational databases are really good at being queried for arbitrary constraints).

I think people tend to dismiss relational databases a bit too quickly these days.

5
maktouch 3 days ago 2 replies      
It's really interesting to see that you're using Cassandra for this. IIRC, Cassandra was created by Facebook for their messaging, and realized that eventual consistency was a bad model for chat, so they moved to HBase instead. (source: http://highscalability.com/blog/2010/11/16/facebooks-new-rea...)

The tombstone issue was really interesting ! Thanks for sharing.

6
flyingramen 3 days ago 5 replies      
It is fascinating that more and more people are using Cassandra. DataStax believes they have fixed problems with prior guarantees claims that were exposed by Jepsen. But there has been no official Jepsen testing since.

On the topic of looking at Scylla next, I wonder why did the team not just start out with it to begin with. Also, are they people with experience running both. How is the performance? And what is the state of reliability?

7
alfg 3 days ago 1 reply      
Love Discord. Most of my friends and I have switched over from using Mumble and it's been great.

I run a small Mumble host [1] and I've always thought of the idea of wrapping the Mumble client and server APIs to function like Discord/Slack as an open source alternative. Mumble is great and all, but the UI/UX appeal of Discord is so much better.

Keep up the great work!

Also, is this is the same Stanislav of Guildwork? Ha, I remember when Guildwork was being formed back in the FFXI days.

[1] https://guildbit.com

8
jjirsa 3 days ago 0 replies      
Wildly biased Cassandra person, but I find this very well written and explained, and I'm especially happy that when you bumped into problems like wide partition and tombstone memory pressure, you didn't just throw up your hands, but you worked around it.

The wide partition memory problem should be fixed in 4.0, for what it's worth.

9
mahyarm 3 days ago 4 replies      
Discord missed an opportunity a year or two ago to become something like slack for large companies. Hipchat's perf is horrible and slack couldn't scale to +20k users a year ago. Managing a mattermost instance requires staff and is more outage prone.

It's really too bad that they didn't take advantage of it, since they were actually scalable compared to their competitors and had good voice chat. Slack has started becoming more scalable recently, so I don't know how much the opportunity is still there.

10
sparrish 3 days ago 2 replies      
If you're deleting often, I recommend running a full compact (after your repair) to free up space and rid yourself of those tombstones once and for all. Repairs without compacts make those SSTables grow and grow. It's amazing how much space a compact clears up.
11
joaodlf 3 days ago 1 reply      
Not surprised to see other companies facing issues with Cassandra and tombstones. Don't get me wrong, I understand the need for tombstones in a distributed system like Cassandra... It doesn't make it any less of a pain though :).
12
cookiecaper 3 days ago 2 replies      
I'm one of the people who nagged you on the redis post, and particularly expressed skepticism that such a transition would've been necessary. I haven't read this yet, but I just want to say thanks for actually following up to that thread and posting it. Looking forward to it!

---------

EDIT: Just read the post, and while it provides a good perspective on Discord's rationale to introduce Cassandra in the first place and does a great job pointing out some unexpected pitfalls, it doesn't specifically respond to replacing Redis with Cassandra due to clustering difficulty, per the prior thread. [0] Redis is only specifically called out as something they "didn't want to use", which I guess is probably the most honest answer.

The bucket logic applied to Cassandra seems like it could've been applied to redis + a traditional permanent storage backend nearly as easily. The biggest downside here would be crossing the boundary for cold data, but that's a pretty common thing that we know lots of ways to address, right? And Cassandra effectively has to do the same thing anyway, it just abstracts it away.

Again, I'm left wondering what specific value Cassandra brings to the table that couldn't have been brought by applying equal-or-lesser effort to the system they already had.

I also found it amusing that they're already contemplating the need to transition to a datastore that runs on a non-garbage-collected platform.

[0] https://news.ycombinator.com/item?id=13368754

13
beck5 3 days ago 2 replies      
Serious question, how do you backup a casandra database of that size. Do you even back it up or just rely on the sharing to prevent dataloss?
14
Globz 3 days ago 0 replies      
I love Discord and use it on a daily basis, one of our main concern with my gaming group is the voice latency compared to TS, Mumble or Ventrilo but this is mainly due to the inability to host your own server.

One of the big missing feature we would like to have in Discord is the ability to assign special permission to our groups leader so they can communicate over voice chat to other other group leaders in other channels (global voice chat).

When we play PVP MMO's and have 40+ users all in the same channel calling shots its impossible to coordinate properly.

What we normally do is split the group in 4 so 10 players in 4 different channels and each group leaders are calling shots independently BUT can also communicate via voice chat to other group leaders. Basically there's a global voice chat for group leaders that no one else can hear but them.

Other than that Discord is amazing!

15
mastax 2 days ago 0 replies      
For a bit more information about the tombstone issue from the perspective of the person who caused it: https://www.reddit.com/r/programming/comments/5oynbu/_/dcnxy...
16
glidek 1 day ago 1 reply      
> Having a large partition also means the data in it cannot be distributed around the cluster.

Why can't a large partition be distributed around the cluster?

17
jolux 3 days ago 3 replies      
Discord is great but I have intermittent performance issues with it that make it almost unusable in comparison to Slack which never has any noticeable latency.
18
smaili 3 days ago 1 reply      
Does anyone know what protocol/transport Discord uses? XMPP, web sockets, JSON, etc?
19
treenyc 3 days ago 3 replies      
I'm curious why would people use a closed source software, when you can use something like https://riot.im

Please let me know. I may be missing something.

20
simooooo 3 days ago 1 reply      
What's an upsert?
21
no_protocol 3 days ago 2 replies      
22
lightedman 3 days ago 1 reply      
You're storing messages, how are you guaranteeing safety of those messages when it looks like one can seemingly just blast through your API calls to find messages when one isn't even on that server?
23
marknadal 3 days ago 0 replies      
Wow! This is an incredible article. I do research and development for systems like this at GUN, and this article nails a lot of important pieces. Particularly there ability to jump to an old message quickly.

We built a prototype of a similar system that handled 100M+ messages a day for about $10, 2 minute screen cast here: https://www.youtube.com/watch?v=x_WqBuEA7s8 . However, this was without FTS or Mentions tagging, so I want to explore some thoughts here:

1. The bucketing approach is what we did as well, it is quite effective. However, warning to outsiders, this only effective for append-only data (like chat apps, twitter, etc.) and not good for data that gets a lot of recurring updates.

2. The more indices you add, the more expensive it gets. If you are getting a 100M+ messages a day, and you then want to update the FTS index and mentions index (user messages' index, hashtag index, etc.) you'll be doing significantly more writes. And you'll notice that those writes are updates to an index - this is the gotcha and will increase your cost.

3. Our system by default backs up / replicates to S3, which is something they mention they want to perhaps do in the future. This has huge perks to it, including price reductions, fault tolerance, and less DevOps - which is something they (and you) should value!

There backend team is amazingly small. These guys and gals seem exceptionally talented and making smart decisions. I'm looking forward to the future post on FTS!

14
Announcing Pipenv kennethreitz.org
462 points by imkevinxu  11 hours ago   100 comments top 26
1
cderwin 9 hours ago 6 replies      
This is great, but sometimes I think that python needs a new package manager from scratch instead of more tools trying to mix and mash a bunch of flawed tools together in a way that's palatable by most of us. Python packaging sucks, the whole lot of it. Maybe I'm just spoiled by rust and elixir, but setuptools, distutils, pip, ez_install, all of it is really subpar. But of course everything uses pypi and pip now, so it's not like any of it can actually be replaced. The state of package management in python makes me sad. I wish there was a good solution, but I just don't see it.

Edit: I don't mean to disparage projects like this and pipfile. Both are great efforts to bring the packaging interface in line with what's available in other languages, and might be the only way up and out of the current state of affairs.

2
olejorgenb 6 minutes ago 0 replies      
Seems to be a python specific nix-shell like tool?

With nix[OS] you just run `nix-shell -p python[2,3] python[2,3]Pacakges.numpy ...` to get an environment with the required packages.

Of course this requires that the python library is packaged in nix, but in my experience the coverage is quite good, and it's not very hard to write packages once you get the hang of it.

It also possible (but currently a bit clumsy in some ways) to set up named and persistent environments.

3
shakna 11 hours ago 0 replies      
> I wrote a new tool this weekend, called pipenv.

> It harnesses Pipfile, pip, and virtualenv into one single toolchain. It features very pretty terminal colors.

For a weekend project, this has some very nice things.

Which removes the need for me to run my own project that basically does these things... In more or less, a worse way.

Everything I've come to expect from Reitz, and hopefully it'll gain some decent ground like other projects of the same author.

4
therealmarv 10 hours ago 1 reply      
For people who want to do it right without using an additional tool read this: setup.py vs. requirements.txt by Donald Stufft https://caremad.io/posts/2013/07/setup-vs-requirement/
5
JeremyBanks 48 minutes ago 1 reply      
> Otherwise, whatever $ which python will be the default.

This is a bit strange because the python binary is always supposed to be Python 2. The Python 3 binary is supposed to be named python3. Some distributons don't follow this, but they're the weird non-conformant ones; it's not a behaviour that should really be relied on.

6
renesd 10 hours ago 1 reply      
Neat. Now for questions and comments.

Often people have a requirements.live.txt, or other packages depending on the environment. Is that handled somehow? Can we use different files or sections? [ED: yes, different sections]

Still wondering to myself if this is worth the fragmentation for most people using requirements.txt ? Perhaps the different sections could have a "-r requirements.txt" in there, like how requirements.dev.txt can have "-r requirements.txt". [ED: the pipfile idea seems to have quite some people behind it, and pip will support it eventually. Seems it will be worth it to standardise these things. requirements.txt is a less jargony name compared to Pipfile though, and has a windows/gui friendly extension.]

Other tools can set up an environment, download stuff, and run the script. Will pipenv --shell somescript.py do what I want? (run the script with the requirements it needs). ((I guess I could just try it.)) [ED: doesn't seem so]

Why Pipfile with Caps? Seems sort of odd for a modern python Thing. It looks like a .ini file? [ED: standard still in development it seems. TOML syntax.]

With a setup.py set up, all you need to do is `pip install -e .` to download all the required packages. Or `pip install somepackage`. Lots of people make the setup.py file read the requirements.txt. Do you have some command for handling this integration? Or is this needed to be done manually? [ED: seems no considering about this/out of scope.]

Is there a pep? [ED: too early it seems.]

7
helb 2 hours ago 2 replies      
> --three / --two Use Python 3/2 when creating virtualenv.

I use Python 2.7, 3.4, and 3.5 on various projects. Is there a way to choose between 3.4 and 3.5 using Pipenv? I'm using something like this with virtualenv:

 $ virtualenv -p `which python3.5` .venv

8
choxi 10 hours ago 2 replies      
Is this like Ruby's Bundler for Python? I've just been getting into Python and am really glad to see this, thanks for creating it!
9
wyldfire 2 hours ago 0 replies      
I haven't really used requirements.txt because I found that I could install 'extra' and 'test' specific content based on args to setup() in my setup.py. It seems more like the Right Thing than requirements.txt, from what I can tell.

At first glance, this doesn't seem to offer anything beyond what I already see from setup(). What am I missing?

It's unfortunate that CPython gave us distutils and took a very long time to converge on a built-in successor (setuptools?) that gives the right composability.

10
gourneau 10 hours ago 0 replies      
Hey other Reitz fans. Make sure to check out his newish podcast series: https://www.kennethreitz.org/import-this/
11
caconym_ 9 hours ago 0 replies      
I will definitely be trying this out. Python version and package management is a dumpster fire that wastes gobs of my time on the regular. I'll try anything that promises to end the pain.
12
command_tab 10 hours ago 0 replies      
See also: https://github.com/pypa/pipfile

I'm glad to see Python getting the same attention as other modern package managers. This is all great work!

13
jaybuff 10 hours ago 0 replies      
14
istoica 6 hours ago 0 replies      
Finally someones does it!I was using: pip -t .pipin my code, avoiding virtual-env completely, but that was not enough and incomplete.

As this is not cross platform and it would be nice to switch between Linux/Windows while coding to maintain platform compatibility, can the virtualenv envs be created with a os platform & subsystem prefix ?for example, having multiple envs at once:

 - env/posix/bin/activate - env/nt/Scripts/activate.bat

15
nejdetckenobi 9 hours ago 2 replies      
normally I use virtualenvwrapper and that makes a virtualenv directory for all virtualenvs you create with it. before that, I always create my projects' venvs inside my project hierarchy.

I had a dilemma about it. But after all, you can not move your venv directory unless you use `--relocatable` option. So, anyone have a strong argument about creating venvs inside your project directory?

16
sametmax 9 hours ago 0 replies      
I was really not a fan of the last "made in Reitz" project Maya. But this, I really can get along.

The whole things make it way easier to get started for a beginner. Now more activate. No more wondering about virtualenv. Automatic lock files are great since no project I know of use them since they are not well understood.

It's like node_packages (easy and obvious), but cleaner (no implicit magic).

Like.

17
noway421 10 hours ago 0 replies      
This is very interesting! I had exact same question how to do it in python just a while ago! http://stackoverflow.com/questions/41427500/creating-a-virtu...

Glad that someone thought about similar thing and made a tool to solve it!

18
mikhuang 10 hours ago 1 reply      
Uninstall by default removing everything seems a little scary. Otherwise looks really neat, looking forward to trying it
19
therealmarv 10 hours ago 1 reply      
Currently using fish and virtualfish. This seems incompatible to non bash shells. Did somebody tested?
20
zoul 9 hours ago 3 replies      
I always wonder if this could be done once and for all languages, instead of Ruby making bundler, Haskell Cabal sandboxes or stack, Perl Brew, etc. Is this where Nix is going?
21
btashton 10 hours ago 0 replies      
Interesting. I have been leveraging tox to provide a lot of what this seems to give you, but it certainly has been more of a hack than a solution.
22
zyxzkz 10 hours ago 2 replies      
What took Python so long to get a tool like this?
23
throw2016 2 hours ago 0 replies      
I think an app should not expose end users to its dependencies. That leaves the end user with a lot of pain figuring out versions of dependencies and god forbid you need to compile some dep then you need a build environment and its dependencies any of which can fail in this chain leaving a very unpleasant and even hostile end user experience.

Ruby and Node apps are particularly guilty of this pulling in sometimes hundreds of packages some of which need compilation. Compare that to a Go binary which is download and use. These things can get very complicated very fast even for developers or system folks let alone end users who may not be intimately familiar with that specific ecosystem.

24
zephyrfalcon 9 hours ago 1 reply      
A bit off-topic, but what font is used in the video/animated GIF?
25
auston 10 hours ago 0 replies      
I can only express my gratitude, thank you Kenneth!
26
korijn 10 hours ago 1 reply      
Windows support?
15
New Wyoming bill forbids utilities from using renewables csmonitor.com
341 points by xkcd-sucks  22 hours ago   320 comments top 27
1
ghouse 22 hours ago 18 replies      
The coal industry is one of, if not the largest industry in Wyoming. As wind and solar become the least-cost source of new generation, the economic viability of coal generation is threatened. Rather than allow the free market to select the least-cost solution, so-called Republicans who think government shouldn't pick winers, are picking winners.
2
bediger4000 22 hours ago 11 replies      
How is this not the kind of regulations/laws/red tape oft decried by conservative, free market politicians? How is this not picking a winner, another practice oft decried by conservative, free market politicians?

Is this just a case of "free market for me, but merchantilism for thee"?

3
diafygi 21 hours ago 7 replies      
I work in renewables, and I encourage this community to not to ridicule people in Wyoming for their very clear majority choice. In their point of view, they are making the best choice for their interests, so calling them stupid only deepens their resolve.

If you want renewables in Wyoming, there are four options (in order from most effective to least):

(a) Move to Wyoming and outvote the existing population (it's not crowded, so it wouldn't take much).

(b) Donate money and time on marketing, education, and advocacy campaigns to try and convince people in Wyoming to your point of view (somewhat difficult to do as an outsider, but not impossible).

(c) Boycott fossil fuels from Wyoming (very difficult since they self consume a lot).

(d) Wait for the existing generation to die and hope the next generation makes decisions in your favor.

Currently, it sounds like HN has settled on option (d), but if your really want change, (a) and (b) are the things that work the best.

4
bwb 20 hours ago 2 replies      
WTF, I used to be closer to a lot of the republican ideology, but their move over the last 10 years to be so anti-science, anti-fact, anti-thinking it is repulsive. They used to stand for letting people live and not be bothered by the government, now they try to control women's vaginas and so much other shit that bugs me.
5
dmichulke 20 hours ago 0 replies      
When the wind of change blows, some men build walls and some build windmills

- Chinese proverb

6
padseeker 14 hours ago 0 replies      
Remember when the state legislatures in "Conservative" "Republican" states like Texas voted to ban Tesla sales in their state?

Lots of people in Wyoming and Texas vote "conservative" and conservatives are supposed to advocate for the free market.

And those people claim to be "conservative" want capitalism and the free market UNTIL the free market threatens to put their jobs at risk. Maybe they were never really conservative in first place?

7
anon1253 19 hours ago 1 reply      
Oh downvote me into oblivion. But whatever the hell you're doing US, it's not good. It's not good for you, it's not good for the planet. It's not good for your citizens, it's not good for humans all around the planet. With all your overseas missions to "protect freedom" or some other utterly irrelevant excuse to satiate your military-industrial complex, you fail to do the one thing that might actually help. Progress. Progress beyond your dependence on fossil fuels, progress beyond the need to entice war, slavery, destruction for those who hold those mineral resources you value so dearly. Just think for a second: however much is left, however destructive it might be to the planet to extract and burn it(losses beyond imagination included), there is still a finite amount of it. You could be the front-runners in a revolution never seen before. The front-runners in an economy liberated from the need to see energy as "scarce". You're stuck in a mindset. "Energy is scarce". It's not, it's abundant. It's /everywhere/. Solar, Geothermal, Wind and Hydro-electric are not some hippy post-fact climate change conspiracy: they can and will provide you with unlimited and free energy. Just /think/ for just a second what that would do to your infrastructure. Grow food in deserts? Done. Free transport across the world? Done. Virtually free drinking water through ocean desalination? Done. Massive reductions in prices for food and other necessities? Done.

But no, you want to live in a world where energy is scarce and you're the sole "protector" of its use and freedom. Cite "jobs lost" or whatever you can think of to protect your bubble; but this path you're on is not sustainable.

8
Hondor 14 hours ago 1 reply      
It's not forbidden, it just has a ~10% tariff (fine). $10 per MWh compared to the current price of $120 per MWh. If solar or wind ends up actually cheaper than coal, it'll probably be by more than that 10% so it'll still be economical to use them.

Furthermore, the fine doesn't apply to exported electricity, which is most of it:"Wyoming sends two-thirds of the electricity it generates to nearby states" [1]

[1] https://www.eia.gov/state/analysis.cfm?sid=WY

9
gumby 16 hours ago 0 replies      
This is not as bad as it sounds. In fact the answer is in the article: > Wyoming already generates more electricity than it consumes.> The state already has wind farms, and a 3,000-megawatt installation is under construction in Carbon County.

People will still build wind farms, they'll just export the electricity. This is no different from Germany going "nuclear free" (they still import energy from nuclear plants in France and the Czech Republic). California is a leader in green energy, of which quite a bit comes from hydro of which CA has almost none...but Washington does. Etc etc.

Still, grandstanding does send a message and this one is a stupid one.

10
Touche 22 hours ago 7 replies      
Missing from this article is the justification being given. They must have one, what is it?
11
startDaemons 12 hours ago 0 replies      
The bill was (according to it's creators) designed to make the utilities reserve the cheapest power for Wyoming residents rather than exporting it all to states where they could get higher retail prices and bigger profits, leaving Wyomings the less reliable and more expensive renewables.

In case you missed it the bill only affects energy sold to Wyoming residents, that's only 584,153 people (!). The utilities can sell renewable energy outside the state with no penalty.

'When asked about the motivation for the bill and concerns about it driving away future wind generation, bill sponsor Republican Rep. David Miller from Fremont County said, "Wyoming is a great wind state and we produce a lot of wind energy. We also produce a lot of conventional energy, many times our needs. The electricity generated by coal is amongst the least expensive in the country. We want Wyoming residences to benefit from this inexpensive electrical generation."'

12
CalChris 21 hours ago 1 reply      
Meanwhile China is shutting down coal plants.

https://www.nytimes.com/2016/04/26/business/energy-environme...

I know that Utah coal is exported to China since it gets shipped by rail to Port of Stockton. A local developer was trying to build a coal terminal in Oakland (Oakland Army Base which is basically attached to the Port of Oakland) but the city (mayor+sups) wisely shut that down. It is insanely stupid to have a coal terminal upwind of a populated area.

13
rosser 21 hours ago 1 reply      
As terrible as this is, at least they can continue selling their renewables out of state. The winds in Wyoming are incredible (google image search: "Wyoming wind sock"; it's less an exaggeration than you'd think).

The Fine Article also points out that ~90% of WY's electricity already comes from non-renewables (though, for whatever reason, they count hydro amongst those).

Net, this isn't exactly changing the status quo for WY.

14
deftnerd 21 hours ago 1 reply      
Many states and utility companies are making it difficult for consumers to sell locally generated power to the grid.

There is pain in the short term, but in the long term it'll just cause the renewable manufacturing industry to throw more resources at improving power storage as much as the power generation has improved over the last few decades.

Tesla's Powerwall is a good step, but there are many other opportunities for companies to live in this field. Once storage technology improves, then more and more users can generate locally and store their own power and just use the grid as a backup power source.

Interestingly enough, some jurisdictions don't allow people to go "off grid" entirely because of laws passed to ensure that all citizens have a source of power and potable water. Many of those laws don't take self-generation into account.

15
ridgeguy 14 hours ago 1 reply      
Given that the US grid is highly interconnected, would this legislation require Wyoming utilities to avoid using renewable-generated electricity from sources outside Wyoming?

This would seem to require that Wyoming's utilities isolate the state's grid to prevent using "eligible resources", according to the bill.

16
woodandsteel 18 hours ago 0 replies      
The goal of the bill is to protect Wyoming's fossil fuel industries. But even if passed, it would largely fail to do that. That's because the great majority of the state's fossil fuel production is sold to other states and countries that are charging ahead with renewable energy.

Instead of trying to stop the unstopable, the state government should be working on how to adapt to the new world where no one wants to buy their fossil fuel exports.

17
gersh 18 hours ago 0 replies      
I'd take this as a sign the fossil fuel industry is dying. Trying to ban your competition is usually a last resort for dying industries.
18
ransom1538 18 hours ago 0 replies      
This is in the same bucket of stupidity as Oregon preventing you from pumping your own gas.[1] I do enjoy the comedy however.

[1] http://mentalfloss.com/article/18812/why-cant-you-pump-your-...

19
caf 16 hours ago 0 replies      
So... Wyoming exports most of the electricity it generates, this doesn't apply to exported electricity, and electricity is fungible: in other words, this is just meaningless culture-war posturing, then?
21
coldcode 19 hours ago 0 replies      
Introduced. Not passed yet.
22
crb002 19 hours ago 0 replies      
It might make sense. If coal demand plunges they will have a glut. As long as they don't have to pay the costs of the pollution it makes sense for Wyoming taxpayers.
23
sandworm101 20 hours ago 1 reply      
Solar and wind both need land. Wyoming is a land of farmers, people who extract thier livings from thier land. Where is the farm lobby? They should be defending against any law attacking a potential "crop". Are they all putting ideology ahead of profits? Are we that far down the rabbit hole?
24
codecamper 21 hours ago 3 replies      
25
xg15 20 hours ago 1 reply      
So nice of the republicans that they suddenly discovered their love for coal workers.

I'm curious if they would keep that love if coal companies invested in automation and laid off workers, or if then, they'd suddenly rediscover the faith in a free market.

26
gragas 17 hours ago 6 replies      
This is dumb. If you look at the bill itself [1], it explicitly states

 11 (a) In compliance year 2018, each electric utility 12 shall procure a minimum of ninety-five percent (95%) of its 13 sales of electricity in Wyoming from eligible generating 14 resources. 15 16 (b) In compliance year 2019, each electric utility 17 shall procure a minimum of one hundred percent (100%) of 18 its sales of electricity in Wyoming from eligible 19 generating resources.
The key part here is that by 2019, each electric utility shall procure all of its sales of electricity in Wyoming from "eligible generating resources."

Now, let's see how "eligible generating resources" is defined:

 6 (v) "Eligible generating resource" means an 7 electricity generating resource either located within 8 Wyoming or delivering electricity into Wyoming from another 9 state that produces electricity from one (1) or more of the 10 following sources or system: 11 12 (A) Coal; 13 14 (B) Hydroelectric; 15 16 (C) Natural gas; 17 18 (D) Net metering system, as defined by W.S. 19 37-16-101(a)(viii); 20 21 (E) Nuclear; 22 23 (F) Oil.
Oh no! It looks like "solar" and "wind" aren't on the list of eligible energy resources! Aye, take a closer look: both solar and wind energy fall under "net metering system," so in reality, climate activists are bashing their heads against the wall. They have the right sentiment, but they're only fighting against the bill because they didn't read it. :-(

1. http://legisweb.state.wy.us/2017/Introduced/SF0071.pdf

27
lightedman 16 hours ago 1 reply      
Drop the sensationalist BS headline please. "Individual net metering" is explicitly mentioned. Net metering = individual solar panels owned by property owners. AKA The utility can't build solar but they can use the solar from any household that connects their own solar to the grid with net-metering tie-ins.

This means that renewables of some sort are in fact allowed. All it took was reading to the 9th tiny paragraph.

16
Uber Hires Former Google Search Chief Amit Singhal as SVP of Engineering techcrunch.com
323 points by leothekim  2 days ago   186 comments top 15
1
oculusthrift 2 days ago 17 replies      
I'm really confused on what to think about Uber. My personal thinking/logic is really bearish on them, similar to the post on the front page yesterday [1]. However, I keep seeing extremely smart/accomplished people joining it which makes me second guess my intuition.

[1] https://news.ycombinator.com/item?id=13437414

2
tyingq 2 days ago 8 replies      
There's a number of things that have happened in the organic search area of Google that seem to suggest a declining interest in quality organic results.

There's Matt Cutts' long leave of absence, his departure, and the announcement that he's not really being replaced. A much lower volume of communication from Google on initiatives in the space (they used to talk endlessly about Panda, Penguin, etc). Amit's original reason for departure was "his next journey will involve philanthropy"..that seems to have changed.

My guess is that two things are driving the declining interest...

a) The marketshare battle is done. Google won. No competition.

b) Their various initiatives to push organic results down the fold (more ads, knowledge graph, various widgets, and so forth) has made the quality of the organic results not as important. Good enough is the target.

3
ChuckMcM 2 days ago 1 reply      
This statement -- Those computer science challenges for a computer science geek are just intriguing you give a geek a puzzle, they cant drop it; they need to solve the puzzle. Thats how it felt to me.

When I've been asked what keeps me going this is it, I really like interesting puzzles and I'm sitting there stuck trying to solve it.

It also says a bit about what Uber thinks their big problems are (or where their value add will be). I was expecting them to go with someone more operations focused like Urs Hoezle.

4
r_sreeram 2 days ago 2 replies      
Amit joining Uber after a year's break coincides with the common "1 year no-solicitation" clause in employment contracts. I wonder if we are about to see some top people in Google get poached. Not that there's anything wrong with that.
5
inverse_pi 2 days ago 0 replies      
Kevin Thompson, another VP from Google also joined Uber very recentlyhttps://www.linkedin.com/in/kevinthompsontech
6
1qaz2wsx 2 days ago 1 reply      
have people considered this may eventually lead to Google acquiring Uber? There is the advising CEO Travis Kalanick bit in there.
7
faragon 2 days ago 0 replies      
The no-driver Uber arguments as profitable future it is crazy stuff, in my opinion. Could be as simple as Uber being a bubble? How much will take until that bubble bursts?
8
carussell 2 days ago 3 replies      
I'd like to see Uber get into mapping. Besides Uber's core business that everyone focuses on, they've got a self-driving cars program that's halfway off the ground and they do food deliver through UberEATS. In either case, they've got a vested interest in making sure high-quality mapping data is availablehigher quality than what Google provides.

Given their deals with tons of local businesses through UberEATS, they've got operating hours and location data that's fresher than what anyone else can provide on the scale that they're operating on. Would be nice to see them improving the OSM dataset and partner with e.g. Maps.me.

9
LaFolle 2 days ago 0 replies      
Here is Singhal declaring this on his personal website: http://singhal.info/home/
10
eva1984 2 days ago 1 reply      
I don't think this is going to change anything though. He is just one person.
11
general_ai 2 days ago 1 reply      
That's a pattern among very senior googlers. When they get bored of working, they go to other companies to semi-retire. I'm almost certain this is not going to be any different.
12
sAbakumoff 2 days ago 1 reply      
Btw - What are the responsibilities of SVP of engineering?
13
oh_sigh 2 days ago 2 replies      
Why would a person worth billions work for someone else?
14
cornchips 2 days ago 0 replies      
"The destiny of search is to become that 'Star Trek' computer and that's what we are building..." -Amit Singhal

Laugh.

15
killbrad 2 days ago 0 replies      
Uber is a middle man that takes money from existing and potential cab drivers' pockets, puts it into their own, and artificially reduces consumer costs.

Cab companies aren't innocent bystanders, but the drivers generally are. But All Hail Uber anyways, I guess.

17
How Do You Measure Leadership? ycombinator.com
370 points by craigcannon  3 days ago   146 comments top 41
1
freddyc 3 days ago 5 replies      
Over the years a test I've often used is asking "how does this person respond to being challenged/questioned?" A great leader tends to embrace the fact that someone is asking "why" and uses it as an opportunity to learn and potentially convert the questioning party (if they're questioning something in the first place, then you haven't nailed it 100%). A weak leader who doesn't have confidence in their abilities sees the challenge as a personal attack and reacts in a knee-jerk fashion (often, though not always, resulting in a termination). If you can't reconcile differing opinions and convert those with opposing views to you then you're doomed as a leader and odds are your company/team will experience high turnover.

Obviously there's a whole range of other traits that make great leaders, but I've found people that fail this test are almost always terrible leaders who others don't want to work for.

2
swombat 3 days ago 2 replies      
How do you measure a great car? There are three factors I've observed: great cars can accelerate, great cars are fun to drive, and great cars have steering wheels.

Sorry, but leadership cannot be reduced to these three factors. There are many excellent leadership frameworks out there which provide great insight into how different leaders operate, and what the great ones have in common. Look up topics like Spiral Dynamics, the Action Logic leadership framework, Kegan & Lahey, and the Integral framework, and you'll have some good starting points on models of adult development that correlate to effective leadership.

3
edw519 3 days ago 6 replies      
I've had 80 bosses. 77 of them sucked. I would march through hell to help the other 3 get something done. For me that pretty much sums it up. All the rest is fluff.

FWIW, OP's 3 metrics:

 1. Clarity of Thought and Communication 2. Judgment about People 3. Personal Integrity and Commitment
Those should be necessary but not sufficient characteristics of every person in your organization.

EDIT, response to walterbell & el_benharneen about what made the 3 different (in no particular order):

 - They always told the truth (to everybody). - They knew their stuff (tech, system, user domain). - They figured out the right thing to do. - They communicated often and flawlessly. - They did whatever it took to get the right thing done. - They smiled almost all the time. - They made each other person feel special. - They made work fun. - They were always teaching something. - They called bullshit instantly. - They protected their team. - They inspired us by showing how good things could be.

4
claar 3 days ago 2 replies      
Also a great read along these lines is "The 21 Irrefutable Laws of Leadership" by John Maxwell, which I'm close to finishing currently.

Maxwell claims that leadership is influence, not authority. When I became a co-founder, I thought that made me a leader. But as PG's excellent post and Maxwell affirm, leadership is quite distinct from positional authority -- and is much more difficult to attain.

Speaking directly to this post, I found that rating myself against Maxwell's "21 laws" was a sobering and likely accurate gauge of my leadership ability.

5
ChuckMcM 3 days ago 2 replies      
It is always interesting when someone who believes themselves to be a great leader, discovers that they are not. And since many of the traits that make great leaders, self awareness, humility, honesty, Etc. are missing in these folks, the world around them sort of explodes when that realization hits. In my experience it is a time when they are most likely to embrace 'leadership through politics.' It is always a strong signal that it is time to distance oneself from the faux leader's area of influence.
6
js8 25 minutes ago 0 replies      
With a ruler.
7
prewett 3 days ago 0 replies      
Leadership is people development. So, how many people have you developed? How many times have you reproduced yourself?

If you want grow your company, you are going to have to reproduce yourself so that the new you is doing the old role so that you can step into the new one, or perhaps relieve yourself of excess roles. That role may or may not have the title you had when you were doing it, however. You might be titled "CEO" when you are leading a team of 5 people, but you will reproduce yourself as "Team Leader" as you start adding teams.

Merely having clarity of thought and integrity does not make you a leader, it makes you a great team member. Merely having good people judgement makes you a good manager, not necessarily a good leader. Developing people makes you a good leader. It's hard to do that without the other three, though.

8
remarkEon 3 days ago 0 replies      
This is a hard question to answer, and my personal opinion is based on my experience in the Army over a while. The best leaders I encountered managed to somehow turn out the best in the people they led. That can manifest in a lot of ways. Improvements on subordinate performance, increases in technical proficiencies, a more disciplined approach to their work. Those are all good metrics, but the best leaders managed to get their subordinates to actually want to improve on their own, without sufficient goading from their leaders. Most of that, therefore, lands in the realm of understanding group dynamics, behavioral economics, and leadership psychology.
9
arca_vorago 3 days ago 0 replies      
Leadership is intangible, hard to measure, and difficult to describe. It's quality would seem to stem from many factors. But certainly they must include a measure of inherent ability to control and direct, self-confidence based on expert knowledge, initiative, loyalty, pride and sense of responsibility. Inherent ability cannot be instilled, but that which is latent or dormant can be developed. Other ingredients can be acquired. They are not easily learned. But leaders can be and are made. General C. B. Cates,19th Commandant of the Marine Corps

Ingrained to my brain from my Marine Corps days is the acronym JJDIDTIEBUCKLE as the list of leadership traits, and it has served me well since, although in the civilian world I have had to lower my expectations of others around me in having even a fraction of such traits.

Relevant reading for those curious about how the Corps approaches leadership: http://www.tecom.marines.mil/Portals/120/Docs/Student%20Mate...

10
jonathanstrange 3 days ago 0 replies      
What about this study mentioned in Kahneman's Thinking fast and thinking slow according to which there was only a very slight correlation between the success of a company and the qualifications of a CEO?

Don't get me wrong, CEOs have my uttermost respect and I don't claim that it's an easy job. I just wanted to point out that there are reasons for believing (at least the possibility) that from the point of view of a realistic assessment the choice of a leader and his or her personality, qualifications and ambitions do not have much to do with the performance of a company and that the many apparent examples to the contrary are mostly based on selection bias and some biases towards oneself such as regarding one's own success more as an achievement rather than chance as those of others, estimating your own social status higher than those of others, believing your less biased than others, etc.

Maybe the best qualification for leadership is being at the right place at the right time, and nobody else really wants to do it?

If that sounds too negative, let me stress again that I think CEOs and people in certain kinds of leadership positions often (though not always) do some difficult work that I generally respect. I just don't buy the claim that the successful ones are little geniuses. A decent amount of intelligence (smartness), some generic business knowledge and being good with social relations seem to suffice.

11
eruditely 3 days ago 0 replies      
You should probably follow Nassim's idea of not trying to measure x (leadership) vs output of leadership f(x) and try to measure the exposure and how it impacts it. Since probably the most significant effort has been pulled into probability theory and trying to get a measure of x that's probably the place to look.

And you would NOT try to measure it as a point estimate as many have reminded us, you would try to set bounds lower&upper.

12
unabst 3 days ago 0 replies      
Two words that weren't repeated enough in this essay, especially one with a focus on trust.

1. EMPATHY

Great leaders have empathy towards their customers, their employees, and above all, to their cause, which is what is contagious.

This is an emotional connection that garners an emotional response. The person that initiates the connection is leading. The person responding is following. When this pattern repeats itself, it strengthens the form and function of the relationship.

2. RESPONSIBILITY

Taking responsibility is not to be confused with taking blame, because they are opposites.

Responsibility is taken before the mistake, and doesn't go away after the mistake. When the mistake happens, you apologize, then fix it, because you're still responsible. Blame is only taken after the mistake. It ends with an apology or a legal defense, possibly an acceptance of punishment, and afterwards we forget it all happened. One is progressive. The other is regressive.

There was also one word that wasn't even mentioned.

3. PROMISES

A leader makes promises, and delivers on them, until everyone succeeds. They make promises to clients, to customers, to partners, to investors, and to employees.

You cannot be a liar and keep promises. You cannot be incompetent and keep promises. You cannot make excuses and keep promises. You have to be aware, proactive, and capable to even know which promises to make.

And with every promise you keep, you've just given everyone another excuse to trust you, depend on you, and follow you.

In a nutshell, if they can promise to be responsible for delivering on a cause they deeply believe in, they're a leader.

13
Cyranix 3 days ago 1 reply      
RE: "Clarity of Thought and Communication" I have worked at a couple of places that put a lot of effort into internal communications, selling employees on upcoming product changes they'll be working on, but failed to acknowledge the existing significant problems that everyone saw and that were repeatedly punted on. Being able to give a slick pitch is not sufficient for this leadership criterion; the narrative must be "credible" (as mentioned rather briefly in the article). Is it just me, or do other people find themselves frustrated at internal messaging that is self-consistent but not grounded in reality?
14
Bahamut 3 days ago 0 replies      
For leadership principles & qualities, I am biased towards the list that the Marine Corps has put out: http://www.tcsnc.org/cms/lib010/NC01910389/Centricity/Domain... .

The Marine Corps may not be a paragon in efficiency in some ways, but I have found that these qualities hold strikingly well for good leaders in the civilian world as well.

15
curiouslurker 3 days ago 2 replies      
Great read but did Steve Jobs really have personal integrity? He was famously double faced, manipulative and as petulant and petty as a child, often settling personal scores with business decisions.
16
pjmorris 3 days ago 1 reply      
A leader is best when people barely know he exists, when his work is done, his aim fulfilled, they will say: we did it ourselves.

- Lao Tzu quote opening 'Becoming a Technical Leader' by Jerry Weinberg

17
Macsenour 3 days ago 1 reply      
Being a boss and being a leader are two very different things.

That may seem obvious to those that understand it, those that don't will think I'm nuts. As a scrum master I have been a leader at every company where I have worked. I have never had anyone report to me in those same companies, aka not a boss.

18
treenyc 3 days ago 0 replies      
Before we can measure leadership. Maybe we ought to first figure out what we mean by leadership.

Often there has being a mix up between leadership, management, and a bunch of other stuff that has nothing to do with leadership.

If people are interested in how and leadership is effectively exercised and what it is. Take a look at this paper: https://ssrn.com/abstract=1392406

19
fuzzfactor 1 day ago 0 replies      
Good article by someone who is obviously well-informed on concentrated startups, and whose efforts I can easily respect.

I agree that these are some of the universal features that have always functioned best when combined they yield the trust that is so essential.

Plus some of the best leaders will actually earn enough respect to exceed that which would be expected by their position alone, earning every bit of it, rather than imposing it from above (from a naturally lower ceiling).

In most cultures, it does seem, that situations will always arise where better leadership is needed in ways that can not be measured.

Sometimes only a type of natural leadership will do, the kind that can not be acquired.Interestingly, this can also be the kind that does not fade even during periods without a team to lead.

And, some of the time when it really counts most, the need to recognize the optimum or required type of leadership will not be met without defying metrics completely.

So I then ask the question "Why would I want to measure leadership?"qualitatively or quantitatively - in some way other than based on my own abilities and intuition developed over a long lifetime of influence by those who have gained the most respect for their superior leadership

And I get the expert answer; "In a startup culture that is obsessed with management by metrics".

20
ktRolster 3 days ago 1 reply      
There's kind of a difference between a manager and a leader.

Manager - Makes sure things get done. If someone quits, finds a replacement, etc. We should all be managers of ourselves.

Leader - A person that employees are willing to follow. Makes the group into a team, working together. Actually cares about the members in his team, protects and defends them. Fights to get them raises, etc.

21
StreamBright 7 hours ago 0 replies      
This is insanely valuable post, helped me a lot with understanding leadership.
22
visarga 3 days ago 1 reply      
Optimize for more than immediate profits. Don't consider themselves detached from common population, and act in the interest of the greater good. It's a case of game theory - we need to cooperate even at the cost of a personal loss for the greater good, otherwise we all lose.
23
zzalpha 3 days ago 1 reply      
All the qualities they identify, here, are, in my mind, absolutely necessary (though not sufficient) for someone to be a good leader.

But, despite the title of the article, none of them are objectively quantifiable.

24
6stringmerc 3 days ago 0 replies      
Leadership can be measured by simply stripping away all external factors that could distort the ability to quantify the Individual Leadership Quotient. A few such elements would include, but are not limited to: A) Talent and Aptitude of Followers, B) Macro Economic Conditions, C) Luck, D) The Weather...basically I think the notion of Leadership is very elastic and, more often than not, highly circumstantial.

What is good Leadership for a bunch of grunts storming a beach in combat isn't objectively comparable to good Leadership for a bunch of teenagers in a classroom environment. There are some "Characteristics" I think that can be described and discussed as a useful musing on the concept, but it has to be qualitative not quantitative from my perspective.

25
rebootthesystem 3 days ago 0 replies      
Well, there are others factors at play today. Here are a couple of videos that discuss the general topic:

https://www.youtube.com/watch?v=hER0Qp6QJNU

https://www.youtube.com/watch?v=R0xYCy2eft8

I have seen and continue to see some of the behaviors described in these two videos and it is deeply disturbing.

Attempting to lead people with deep social challenges is an exercise in frustration and futility. Leadership, in this context, is a very different thing than in what I'll call more traditional settings. It almost has to be reduced to appeasement and coddling. Latte's and ice cream.

We have a generation of adults who behave as petulant children half their age did in prior generations. Except they are in a 25 year old body. Some of these 25 year olds today would be slapped out of the building by 25 year olds a generation or two ago. They are weak, oversensitive, self-serving, entitled, delicate and disconnected from reality.

This is how you end-up with some of the crazy stuff coming out of outfits like Facebook and Google. They are completely devoid of real world social and business skills yet interact and affect the personal and business lives of millions.

One example that comes to mind are account suspensions and cancellations without even a shadow of customer care or service offered. If you can't swipe or click a problem away the option to actually engage with a real human being and exercise the ability to resolve problems simply isn't there.

How do you lead these people? Well, first they have to grow up. I suspect that will happen once they get to 35 or 40 years of age and finally understand reality. What will the consequences of such dysfunction be a few decades from now? Not sure.

26
ImTalking 3 days ago 0 replies      
I think you can overcomplicate this question but a leader is someone that, over time, people follow. Why they follow is up to the individual.

Gandhi was a leader, but then conversely, Hitler was also a leader. There is no morality in leadership, but I would say a common trait would be charisma.

27
laurex 3 days ago 0 replies      
These all seem like good things in a leader, but it does miss a vital quality, which is being able to guide those around the leader to perform at an elevated level, usually because the leader has the ability to both convey the importance of the mission and to be a good "whisperer," i.e. someone who listens to their team, understands what makes them tick, and supports them in performing at their best.
29
perseusprime11 3 days ago 0 replies      
Here's my short list:

1. Listen to your people and look after them.2. Delegate work because your people can do it better than you.3. Make sure your team is working on the right things

30
mempko 3 days ago 2 replies      
A leader is not a position, but a role anyone can play at any given time.
31
kogus 3 days ago 0 replies      
Proof is in the pudding. Measure leaders by how many follow them, giving greater weight to leaders who are followed by other leaders.
32
calinet6 3 days ago 0 replies      
Having integrity, being able to judge people, and being a smart thinker? That's how you measure the leaders of an organization which was heavily influenced by the management principles of W. Edwards Deming? Have you even read Creativity Inc? Ed Catmull himself said, "As we struggled to get Pixar off the ground, Demings work was like a beacon that lit my way."

Get a used copy of this book and read it cover to cover: https://www.amazon.com/Leaders-Handbook-Making-Things-Gettin...

Chapter 2: The New Leadership Competencies:

- Competency 1: The Ability to Think in Terms of Systems and Knowing How to Lead Systems

- Competency 2: The Ability to Understand the Variability of Work in Planning and Problem Solving

- Competency 3. Understanding How We Learn, Develop, and Improve; Leading True Learning and Improvement

- Competency 4. Understanding People and Why They Behave as They Do

Those sound a tad more concrete and believable, don't they? That's an understanding of reality that might help you be a better leader to an organization that actually works. Dismiss the surface-level personality games and get yourself into the scientific reality of organizations, and you have a hope of leading one well. There are no missing partsthe whole system is important. That's the leadership secret.

My bet is that the leaders described in this post are better described by the above characteristics, and they more reliably predict leadership success, than any of their individual traits or abilities. Certainly Ed Catmull, who was himself a big believer in Deming's way of managing companies, fits that model, and Steve Jobs was heavily influenced by Deming and Juran in creating a system able to produce extraordinary quality. In fact, the whole Pixar team this post is about was more heavily influenced by Deming's concepts than any trite personality fluke, yet that influence is entirely ignored here.

This is forgivable: it's attribution bias. We instinctually want to attribute to the greatness of the individual that which was actually more nuanced, the outside factor in this case being a great body of knowledge about management and leadership that led them to be extraordinary.

Now you know. Read Peter Scholtes' Leader's Handbook, read Creativity Inc., and keep thinking about it. There's way more to it than just having integrity, being able to judge people, and being a smart thinker. If excelling at those were all it took, we'd be up to our necks in extraordinary leaders. Must be something else, then.

33
treenyc 3 days ago 0 replies      
Hmm, do we distinguish leadership from management?
34
ThomPete 3 days ago 0 replies      
You don't. You experience it.
35
alfonsodev 3 days ago 0 replies      
Two things:

By the profesional/personal growth of each team member and by the harmony of the group.

36
benkitzelman 3 days ago 0 replies      
Look behind them and see who is following (following.... not just obeying)
37
losteverything 3 days ago 3 replies      
Getting people to do things they don't want to do.

I believe from Jack Welch

38
z3t4 3 days ago 0 replies      
How many people that follow him/her literally
39
ajmarsh 3 days ago 0 replies      
By the output of the employees that are lead/managed?
40
imh 3 days ago 5 replies      
I'm sad not to see an emphasis on giving a shit about the lives of those people you're leading. Personal development, career development, family, fun, etc. These are all hugely important to people outside of whatever widgets they are contributing to. A good leader should care about helping the people they lead achieve their goals, and not just in the sense of finding people who are willing to pretend their goals align with the widgets.
41
sbierwagen 3 days ago 2 replies      

 It is based on observations I made when working closely with four leaders that I consider extraordinary: Ed Catmull (Pixars founder), Steve Jobs (Pixars CEO), John Lasseter (Pixars Chief Creative Officer), and Bob Iger (Disneys CEO).
All four of these guys were involved in wage-fixing, which cost their companies $415 million. https://en.wikipedia.org/wiki/High-Tech_Employee_Antitrust_L...

So, "extraordinary" in the sense of being extraordinarily unprofitable.

18
Japanese toilet industry agrees to standardize complex bidet controls theverge.com
287 points by prostoalex  4 days ago   228 comments top 19
1
bjackman 4 days ago 16 replies      
> The government has recommended removing the Buddhist manji symbol from maps aimed at foreigners, for example, for fear of unintended associations with the Nazi swastika.

Does anyone else think this is a shame? When I've been to Asia and seen "swastikas" everywhere I've found it in a way joyful. The hate symbol has no power here, I thought; it's a positive thing. Why should one culture change their own iconography just because it was perverted in a different culture? It seems like a mild example of self-inflicted cultural imperialism.

Maybe I'm just being very philosophically nave.

2
patio11 4 days ago 11 replies      
If you want an illustration of the problem, here's one sourced by the very scientific method of finding the closest men's restroom:

https://www.dropbox.com/s/0iripy6m7fu5f3o/2017-01-19%2016.57...

There are over 20 individual controls on that unit (which is, FWIW, common and reasonably expensive). If you do not read Japanese, good luck at finding flush... and finding it will not help you finding it on the next machine you use.

3
piyush_soni 4 days ago 6 replies      
Oh, how badly I wished Americans had a clean way of cleaning their ... bottoms during my 8 years of stay there. Thankfully, Amazon had nice portable bidets which you could fit in your home toilets, but everywhere else it was still the same.
4
tunesmith 3 days ago 2 replies      
A year ago I bought a Luxe bidet attachment for my wife as half-joke. We basically fell in love with it. Bought it for friends of ours to be funny. They thought it was hilarious, waited months to install it, then installed it and fell in love with it. Now they want to buy it for friends. We bought it for family members for this Christmas. Uproarious laughter, and then... you guessed it, they fell in love with it, and thinking of who they might buy it for. For Americans, this is truly one of those things where you come away thinking "Holy moly, why did we not do this sooner?"

It's $60 for the deluxe version, cheaper if you don't want hot water (and you actually don't need the hot water). It's a subtle-but-massive improvement in quality of life.

5
dguest 4 days ago 1 reply      
I was hoping that along with the standardization we'd get an explanation (in english) as to what these symbols mean.

I mean it's nice to know that we can standardize to:

- a pair of line wobblers

- two tornadoes of different intensity

- two different fountain rides with different camera angles and zooms

- getting mauled by a three-toed sloth, and

- the all important black box

but maybe for people with less inductive skill some words would help too.

6
wapz 4 days ago 3 replies      
For those who don't know, you don't actually have to use the bidet when you go to Japan. The most complicated thing about toilets in Japan is that the flush button is on the wall sometimes (and the emergency button is often on the wall, too).
7
ericdykstra 4 days ago 2 replies      
It's nice that these companies have come together to standardize their icons.

I don't see the intrigue in this story, though. Can someone explain it to me? Is it the iconography design? The "wow Japanese toilets are complicated!" reaction? People just upvoting anything that has to do with Japan?

8
dcow 4 days ago 0 replies      
I think the real problem is lack exposure to such miraculous devices outside of Asia-Pacific.
9
IE6 4 days ago 0 replies      
As an American who has used Japanese toilets I can testify that they are solving a very real problem.
10
agumonkey 4 days ago 1 reply      
My favorite thing is not the complexity, but that you can operate the toilets without touching them. I wish we had foot controls for cover, flush, water cleaning faucet and soap mandatory everywhere.
11
allengeorge 4 days ago 2 replies      
At the risk of sounding completely uncultured, what exactly is a rear spray, and how does it differ from the bidet functionality? Is the bidet targeted and the rear spray a delightful misting?
12
jzl 4 days ago 5 replies      
Serious question: WHY can't the computer industry do this for USB-C cables and ports? It's desperately needed and shameful that they haven't done this.
13
hmoghnie 2 days ago 0 replies      
All those different symbols were a real shitshow and people just couldn't deal with that crap, now that they are going to sort this out it won't be a pain in the ass anymore with them flushing their old way of doing things away.
14
cm2187 4 days ago 8 replies      
Are these used outside of Japan?
15
dronedronedrone 4 days ago 0 replies      
this is tantamount to cultural genocide /s seriously though, one of the peculiar joys of being in japan and having a very poor grasp of the language is the inescapable urge to play with the bidet buttons. you will inevitably start spraying water all over the bathroom, get yelled at by a nice robot voice, and panic a great deal.
16
codeddesign 3 days ago 0 replies      
To me... for a an industry standardization, those icons are pretty elegant. Just thinking of standard restroom signs and then looking at these.
17
binarynate 3 days ago 0 replies      
This standardization makes it clearer that there's a button that blasts you into the air.
18
petepete 4 days ago 0 replies      
What are the chances that they'll standardise on the Three Seashells system?
19
homakov 4 days ago 0 replies      
So they watched Why him?
19
Dont Tell Your Friends Theyre Lucky nautil.us
321 points by dnetesn  3 days ago   287 comments top 39
1
colanderman 3 days ago 8 replies      
The article (and some comments here) seem to conflate luck and what I will call lot. "Luck" I define as random happenstance during one's life. You can manage luck. Doing so is the central theme of many board games. You can increase your luck "surface area" by taking more chances. Entire industries (e.g. insurance) exist to manage luck.

Your "lot", on the other hand, I define as what you were born with. How you were raised, where you grew up, what kind of education you got -- everything you can't control that does have a significant impact on your life's outcomes. You can work to improve your lot, or minimize its impact on your life, but it's very difficult.

Of course there's some correlation: those with a good lot often learn early how to manage luck, and those who manage luck well can negate a poor lot.

Hence I begrudge no-one with seemingly good "luck": often (possibly more than not), their fortune is simply a byproduct of how they managed their luck. Good for them!

But those born into a good lot? They're the true "lucky" ones.

2
ergothus 3 days ago 15 replies      
My father and I have somewhat productive political conversations: He's fiscally conservative, I tend towards the liberal side of the scale.

Drilling in to find what we really disagree about, it seems to boil down to two concepts: (1) I view success as a matter of luck that your effort can make better or worse. He views effort as the single most important deciding factor in success in life (2) I'm willing to tolerate an amount of "unfairness" in people getting help they "don't deserve", while he finds this very offensive.

I honestly feel that if considered luck to be a larger factor and effort to be a lesser factor, his political stances would change pretty dramatically. (same applies to me in reverse). I wonder how much the social willingness to accept luck as a factor impacts popular political positions. (Perhaps not much, as the author in the article promotes a consumption tax, which is generally seen as more regressive)

3
dv_dt 3 days ago 3 replies      
I think this touches upon one of the biggest weaknesses of the current economic system. We systematically waste the human capability of millions of people because the system essentially randomly gives much better opportunity to some over others. Meritocracy somewhat exists but mostly to the extent that people can maximize the opportunity they've drawn as their lot in life.

I like the idea of Basic Income, but it's a somewhat limited solution to capping how far down someone can fall in society - what would really supercharge a future economy is opening up avenues to truly distributing equal opportunity. Wealth inequality suppresses this strongly, when people receive better margin of income over the absolute minimum economic allocation of their wages, they can then allocate their own wealth from their personal outlook in multiple ways - including starting businesses which may change the world.

4
kyleschiller 3 days ago 5 replies      
Debating the actual importance of luck seems a lot less important than developing the proper attitude towards luck.

Pretending luck doesn't exist can lead to arrogance and a lack of empty for people who haven't succeeded. On the other hand, believing that luck controls everything can lead to fatalism.

It might seem best to find a happy medium, but being wishy washy about this whole thing just gives you opportunities to blame your own failure on circumstances outside your control, while continuing to take credit for success. In the general case, looking for balance between opposing ideologies makes no guarantee that you'll walk away with the best parts of both instead of the worst.

In practice, it's probably best to drop the determinism/indeterminism dichotomy completely and just focus directly on the desired end attitudes.

On a side note, the reason American society is obsessed with meritocracy has nothing to do with a belief about the nature of luck. Denying luck as the path to success is just a way to make people work harder.

5
downandout 3 days ago 3 replies      
It's certainly true that you need to be very lucky to become a billionaire - generating wealth at that level usually involves tremendous numbers of other people loving whatever business you have decided to create. But if you're reasonably intelligent, at least in the US, it's quite possible to become a millionaire without much luck, through decades of hard work and discipline.

Examples: software engineers at large companies that stick around for decades (usually through options), doctors (at least specialists, such as cardiologists and anesthesiologists), and lawyers that go to the best schools and are able to land jobs at top-flight firms. Even tradesmen that stick to their craft, such as master electricians or plumbers, can quite reasonably expect to achieve millionaire status over the course of their lifetime assuming that they manage their money well.

So yes, luck plays a huge role in the creation of enormous sums of wealth. But if you live in a country with abundant economic opportunity such as the US, there's no reason to be poor unless you have been extremely unlucky (health problems, accidents, etc have befallen you), you are unwilling to work, or you've made extremely poor life/financial decisions.

6
phkahler 3 days ago 5 replies      
Progressive consumption tax is ridiculous. It requires your tax rate at the point of sale to be dependent on all your purchases to that point in time. That's just not practical. Or it may require every purchase you make to be recorded for tax-time when you then pay the taxes. Either way it requires the government to know every purchase you make, or at least the price. This is not something anyone should want.
7
cmurf 3 days ago 0 replies      
Veil of ignorance. There's a significant part of the upper end (wealth wise) of the population that like our classist society just the way it is, or maybe that it should be more classist. Everything should be a rent, there should be no public lands, everything is to be exploited, and if you're on the short end of the stick it's merely unfair, not a wrong or a failure of society. Or the more extreme versions of this, higher class folk have better money, better ideas, better genes, make and sell better things. They are better than others. Democracy and socialism are threats to these notions.
8
chrishacken 3 days ago 1 reply      
Maybe I'm naive, but I don't think any one denies the role luck plays in one's success or not. However, to completely discard effort and determination is selling everyone short. I'm running a successful company partially because of "luck", I happened to start it at the perfect time, but also because I pour every ounce of money and time I have into it. My nights and weekends don't exist. Some people aren't willing to put in the time to turn luck into success.

Telling people that success is just a matter of luck will only reinforce the thoughts of unsuccessful people to believe they're "unlucky". You are able to make your own luck to an extent.

9
real-v 3 days ago 0 replies      
This reminds me a little bit about one of my favorite philosophers, Alain de Botton. Sometimes, he discusses meritocracy and meritocratic societies.

Basically, in a meritocratic society, such as the US, people tend to believe everyone's lot in life is deserved; luck is not considered a big factor. This creates a problem where the poor believe the rich made it through their hard work, while the rich believe that poor people deserve to be poor because they are lazy or stupid. People are where they are because they deserved to be there.

I used to place a high value in the concept of a meritocratic society, but experience is convincing me that the lack of compassion that such societies experience is not worth trade off.

10
jeffdavis 3 days ago 3 replies      
Just like when people are trying to sell you something, they call it an "investment"; people trying to implement government spending programs call it "spreading opportunity".

Some government programs really do spread opportunity, but that requires close examination and criticism; I don't just buy into it because a politician calls it opportunity. Is college an opportunity? It can be a huge opportunity to get ahead in life; but it can also just subsidize a partying lifestyle and a phony major for four years. It depends on the college, the student, and the structure of the opportunity.

It's hard to tell the difference between spreading opportunity and spreading results. It often requires looking at the details, measuring along the way, and it is often different for different people.

11
jartelt 3 days ago 0 replies      
I think a lot of people do not realize that you are lucky if you are born into a middle class or upper class family. Having parents with some savings allows you to take extra career risks because you know that you can likely get help from your parents if none of the risks pay off. It is more difficult to make the decision to work at a startup or buy a house if you are totally on your own when things go south.
12
minikites 3 days ago 3 replies      
I think a lot of people are emotionally unable to deal with a world that is as dramatically unfair as ours is, so they fall back to the childish notion that people who have fallen on hard times deserve it and successful people controlled their own destiny to get there, because the alternative is too uncomfortable to think about.
13
ChuckMcM 3 days ago 0 replies      
If you get a chance to experience an "exit", where a number of people suddenly have much more wealth than others around them who are essentially doing the same things but joined the company at a different time, you will get to see all the different ways that people internalize that event (both positively and negatively).

Luck is very much a part of success and a big part of the way that Vikings talked of sailing with successful leaders ('they have a lot of luck'). And most importantly luck has no bearing character. But internalizing that can be hard when someone you despise gets rich, or someone you really care about fails to get the rewards that others in the same place have.

14
luckystartup 3 days ago 0 replies      
> Then at the end students got a bonus for their participation experiment and they were told that they could donate some or all, any fraction of their bonus, to one of three charities, their pick, just by saying so to the experimenter. What she found was that people who had listed external causes of the good thing happening donated about 25 percent more of their bonus to a charity than the people who had listed things they had done to cause the good things to happen. The control group was somewhere roughly in the middle of those two.

> There have been many experiments that have shown if you prime people to feel the emotion of gratitude, they become much more generous toward others, much more willing to pay forward to the common good.

> If you want people to think about the fact that theyve been lucky, dont tell them that theyve been lucky. Ask them if they can think of any examples of times when they might have been lucky along their path to the top.

That's the gist of the article. People get defensive when you say "you're lucky", because they interpret this as "you don't deserve your success". By reframing the message and asking people questions about times where they were lucky, then this can make them feel more generous.

Very practical advice for anyone who is delivering a speech at a fundraiser.

15
charles-salvia 3 days ago 1 reply      
In the United States, at least, poverty tends to be concentrated geographically in inner-cities and rural areas instead of being evenly spread out. This would seem to indicate fairly conclusively that location and environment affect opportunity and wealth more so than an individual willingness to work hard. In fact, being born into an environment of concentrated poverty like this molds your mental state and perception of the world, to the extent that the idea of breaking out of poverty may not always even appear as a possibility, thus discouraging you from even believing that hard work might pay off.
16
emodendroket 3 days ago 0 replies      
This seems to take a sudden leap from a relatively uncontroversial (I'd think) proposition into a political program. I wonder about this bit:

> The price of the average American wedding in 1980 was $10,000. In 2014, the most recent figure I had, was $31,000.

According to a random inflation calculator I checked online $10k in 1980 would be worth almost $30k today. https://data.bls.gov/cgi-bin/cpicalc.pl?cost1=10000&year1=19...

17
nisse72 3 days ago 1 reply      
Tangentally related, I find it interesting that we often call people lucky when something very bad happened to them, but they somehow managed to survive the situation or land on their feet. We aren't as keen to describe people as lucky who avoided danger entirely.

Lone survivor in a plane crash? Lucky. Took a cruise instead? Meh.

Personally I think it's preferable to not be in the crash, than to have survived it.

18
slitaz 3 days ago 1 reply      
"luck" is not a good choice as a word here.They mean something like a chaotic event that ended up being positive to them.

Also, just waiting for such a positive chaotic event to happen to you, is probably not the best strategy.

If you make good social interactions that you maintain, then those positive chaotic events are more likely to come your way.

19
baldfat 3 days ago 0 replies      
I am anti-determinist and Soren Kierkegaard (founder of existentialist thought) so inspired me that I named my son Soren. The fight between the two parties of thought is huge and bigger then Windows vs OS X.

> Jean-Paul Sartre:

"What is meant here by saying that existence precedes essence? It means that, first of all, man turns up, appears on the scene, and, only afterwards, defines himself. If man, as the existentialist conceives him, is indefinable, it is because at first he is nothing. Only afterward will he be something, and he himself will have made what he will be."

Society sees luck in terms of fairness. This article used the word fair or fairness zero times. Fairness is a HUGE issue in deterministic thought especially dealing with how we perceive others around us.

20
tabeth 3 days ago 4 replies      
I'm a strong determinist. Effort, hard work and skill is irrelevant (any relevance comes from the fact that you're already in your statistical band for expected success and are trying to maximize within that). I believe most of your success is determined before you even take one step on this planet. Step one is acknowledging the truth: your initial circumstances dictate your future. Once this is acknowledged, we as a species can begin focusing on making the initial conditions ideal for everyone.

Note: I am not saying you shouldn't work hard. I am just saying that it's not doing as much as you think. Individual examples of success (I've done decently despite two parents who didn't finish elementary school, live in inner city, etc) are not of relevance for planning the future of the human race. The world is chaotic, so there will be outliers in spite of the "determinist property" of the world.

Parents' own desperation to "set their children up" for success is anecdotal confirmation of this fact.

---

Some examples:

Socioeconomic status v. Educationhttp://www.apa.org/pi/ses/resources/publications/education.a...

Health v. Educationhttp://www.nber.org/digest/mar07/w12352.html

Health v. Socioeconomic Statushttp://www.apa.org/pi/ses/resources/publications/work-stress...

Parent education v. child long term successhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC2853053/

Skin color v. attractivenesshttp://journals.sagepub.com/doi/abs/10.1177/0095798405278341

Height v. successhttp://www.independent.co.uk/life-style/health-and-families/...

Weight (at birth) v. successhttp://ns.umich.edu/new/releases/5882

Attractiveness v. successhttps://www.psychologytoday.com/blog/games-primates-play/201...

Gender v. successhttps://www.historians.org/publications-and-directories/pers...

Eye color v. alcoholismhttp://www.sciencedirect.com/science/article/pii/S0191886900...

Geography v. socioeconomic successhttp://www.cid.harvard.edu/archive/andes/documents/bgpapers/...

21
kartan 3 days ago 0 replies      
> The whole process of constructing life narratives is biased in ways that almost guarantee that people wont recognize the role of chance events adequately.

This is also a cultural thing. Here in Sweden it is easier that people dismisses their achievements to not look like they are bragging and to accept that chance is part of life. I do that myself. And I feel better and less stressed recognizing that luck is part of why I have what I have.

So I work not too hard - that can be bad for my health and a bad long term investment - not too little - work is needed to achieve anything and you have to do your part to not let other down. So you work lagom.

22
dang 3 days ago 1 reply      
This topic always reminds me of a line of pg's from years ago: https://news.ycombinator.com/item?id=1621768.
23
philosopheer 16 hours ago 0 replies      
huge success may have luck, but it is always in combination with greed.

The average worker and entrepreneur wannabe in Silicon Valley has nowhere near the requisite and absolute focus on pure self interest that Larry Ellison, Bill Gates, Steve Jobs, etc. have/had. Look to how Woz or Paul Allen was treated by them.

24
Karlozkiller 3 days ago 1 reply      
So if I believe in luck I will be more inclined to pay high taxes? I don't think that's how it works.

I mean of course people with money who realise not everyone who is poor is poor because they're lazy bums will be more inclined to help a "poor person" than they would if they believed all poor are lazy bums. But does that mean they will accept high taxation? Say I am rich and narcissistic, I believe I'm better than everyone and that my skill put me on top. I then realise that other who are skilled are poor and I want to help them become richer. Do I believe that paying the government to use my money for welfare to be the most effective use of my money to fulfil this end? Probably not in that case.

Furthermore some people are lucky, some unlucky. This does not mean that no effort but mere luck goes into building an empire. If luck was the only factor then sure, this argument or taxation might hold. But there's a lot more than luck to it, which is much more in the control of the individual.

25
thret 3 days ago 0 replies      
It's hard to detect good luck - it looks so much like something you've earned. - Frank A. Clark
26
jrs235 3 days ago 0 replies      
I believe https://news.ycombinator.com/item?id=13437977 ties in with this in that many "lucky" people prepared so that when luck struck things were aligned to take off.
27
aresant 3 days ago 1 reply      
Ben Franklin has a great line on this topic - "Diligence is the mother of good luck."

The author illustrates this major point with an example of the "TOP" cellist in the world:

"One [cellist] earns eight or nine figures a year while the cellist who is almost as good is teaching music lessons to third graders in New Jersey somewhere. . . The person who is eventually successful got there by defeating thousands, maybe tens of thousands, of rivals in competitions that started at an early age. . . [but] the luckiest one . . [is] that person who is going to win the contest most of the time."

EG - you need to put in the hours of preparation & subject yourself to competition of the highest order to even have a chance at being the "luckiest" in your field.

28
jrs235 3 days ago 0 replies      
"It takes 10 years to achieve overnight success."

http://www.inc.com/empact/why-successful-people-take-10-year...

29
coka 3 days ago 0 replies      
A few of the commenters here mention meritocracy, and it seems to me that they value it, or think that it is something we should strive for. I would just like to point our that the term "meritocracy" originally carried a negative connotation, with a very elitist endgame[1].

[1] https://en.wikipedia.org/wiki/Meritocracy#Early_definitions

30
swolchok 3 days ago 0 replies      
Related reading: Fooled By Randomness, by Nassim Nicholas Taleb.
31
MikeTLive 3 days ago 0 replies      
the first thing that impacts your future success is the luck of the conditions of your birth. you have no control over this. hard work MAY make up for this, however having a "better" birth condition plus this same hard work does not negate the value of that first starting position.

this is lost on many successful people who wrongly attribute the entirety of their success to their own efforts and presume that anyone who is not successful has simply not worked hard.

32
djyaz1200 3 days ago 1 reply      
There is a pretty good book that addresses some of the business aspects of this... "Competing Against Luck" by David Duncan and Clayton Christiansen (the same guy who wrote "The Innovator's Dilemma"). I'm not done with it yet but so far it goes into some interesting detail about how to reframe everything people pay for as jobs... and that building a successful business is about understanding the job to be done and mastering it.
33
andrewclunn 3 days ago 2 replies      
A lot of this "luck" can be traced back very easily to causes like "had two parents who gave a damn" or "had enough to eat eat growing up." The people pushing this narrative that you're not really responsible for your failure / success want it both ways. They want to make you admit that you benefit from living in a peaceful stable society with infrastructure, while also not wanting to hold parents accountable for having too many kids too early, or admit that impact that divorce has on young children. It always comes down to pushing some narrative that is meant to justify further state intrusion into our lives and the dismantling of the family unit, all with pseudo-scientific (see "the gray sciences") justifications and emotional appeals. Spare me the bullshit, I aint buying it.

EDIT -

Looking for another example of this obvious propaganda? Try the latest episode of RadioLab:

http://www.radiolab.org/story/radiolab-presents-media-busted...

34
jgalt212 3 days ago 0 replies      
here's my formula (and I've been around the block a few times).

decent level of success = 0 units of bad luck + 3 units of skill and 2 units of hard work

yuge level of success = 5 units of good luck + 3 units of skill and 2 units of hard work

35
pier25 3 days ago 1 reply      
This reminded me of the film "Match Point" directed by Woody Allen. IMO his best film.
36
jagtodeath 3 days ago 3 replies      
Not super related but I cant resist. The guy in the article looks almost EXACTLY like Steve Jobs.
37
Mendenhall 3 days ago 0 replies      
Dont tell your friends there is no such thing as luck. There are only factors that are too numerous for you to account for them all.
38
rbanffy 3 days ago 0 replies      
Why would I? They live in this timeline.
39
chrismealy 3 days ago 1 reply      
Frank is a terrific writer and his books are excellent (rare for an economist).
20
The Infrastructure Behind Twitter: Scale blog.twitter.com
335 points by kungfudoi  3 days ago   68 comments top 13
1
niftich 2 days ago 0 replies      
The day before, Discord did high-quality a write-up on why they chose Cassandra [1], and now this post hits explaining how one of the world's most popular and trafficked service has engineered their infrastructure; it's like a dream.

I'll echo the praise I wrote earlier, that insights like this aren't only some of the best content to hit HN, but become some of the most valuable resources for designers who have yet to face a scaling issue, but know they will soon.

Since you have developed custom layers on top of open-source software to fit your particular usecase and load profile, and host all this in-house, have you considered monetizing your infrastructure for outsiders who may have similar needs?

Today, one has limited, unpleasant choices: either pay out the nose for something like AWS or Google Cloud to get elastic scaling and the captive storage systems that can be made to handle these kinds of workloads and still have to write a fair bit of custom glue to get all pieces to play nice, or you can build out the servers yourself, but have to employ dedicated talent with the requisite expertise. Either way, the barriers are fairly steep; you could tap into an under-served market should you choose to sell IaaS (edit: or, more accurately, PaaS). Has this conversation come up in the past?

[1] https://news.ycombinator.com/item?id=13439725

2
burgreblast 3 days ago 6 replies      
100,000's of servers for 100,000,000 of messages/day ?

I understand that half the servers aren't even doing messages, but, isn't WhatsApp doing 2 orders of magnitude more messages with 3 orders of magnitude (?) fewer servers?

Is that right? I'm curious how one would justify 10,000X worse?

So for each message, 10,000X more equipment is needed?

3
atcole 3 days ago 5 replies      
This is a fairly technical analysis, and the terminology used in many cases is above what I know about networking. But the one quote that will stick is this.

"There is no such a thing as a temporary change or workaround: In most cases, workarounds are tech debt."

4
rollulus 3 days ago 4 replies      
It strikes me that so much of the components they use (e.g. under "Storage") are in-house built (several dbs, blob store, caches, etc). Is that because at that time equivalent solutions didn't exist? Is that because Twitter suffers from NIH?
5
tabeth 3 days ago 3 replies      
I know nothing about storage, so I'm a bit confused about why Twitter needed:

1. Hadoop

2. Graph

3. Redis/Memcache

4. Blobstore

5. SQL variants

(and a few others).

I do see that the post has a short snippet briefly describing what they're storing, but I'd be curious to know why (speed, cost, latency, space tradeoffs/constraints).

Also, if any more experienced folks want to chime in: Elixir/Erlang is "built for concurrency" as they say. I'd love to hear people's opinions on their one sentence simplifications of what kind of situation Hadoop/SQL/Redis/etc should be used for (similar to how Erlang is best used for situations where concurrency and fault tolerance is desired). In particular, is there a "Code Complete" type book for storage?

6
jzl 3 days ago 0 replies      
Lots of CIO-type buzzwords and acronyms in this, including many coined by Twitter themselves. More worthy of a skim than a rigorous reading. But interesting for a bird's eye view of how complex Twitter is behind the scenes.
7
mnutt 3 days ago 2 replies      
They mention that they used to base their geo routing on the location of the client's DNS server but have moved to BGP Anycast. I've heard that there can potentially be routing issues for long-running connections using anycast to end users, is anybody else doing something like this and do these issues happen in practice?
8
seanmccann 3 days ago 2 replies      
> Fast forward a few years and we were running a network with POPs on five continents and data centers with hundreds of thousands of servers.

That seems high given Twitter's size and the hardware distribution pie chart they showed. Does anybody have an idea how this compares?

9
ashayh 2 days ago 0 replies      
| We have over 100 committers per month, over 500 modules, and over 1,000 roles.

| we were able to reduce our average Puppet runtimes on our Mesos clusters from well over 30 minutes to under 5 minutes.

This isn't just tech debt .. it's poorly designed, poorly thought, poorly architect-ed and poorly managed in the first place.

Is it because Twitter cannot find good talent because of its falling stock?

10
hueving 2 days ago 0 replies      
They switched to a BGP anycast model for twitter.com, which implies TCP. I'm curious how they deal with situations where route preferences change in intermediary ISPs mid-TCP stream. Does the new server reject the TCP connection or are they synchronizing TCP sessions across clusters?
11
therealmarv 3 days ago 1 reply      
No Ansible? In an older (2014) Ansible video they claim Twitter is using Ansible but I only see Puppet mentioned.
12
turbohz 2 days ago 0 replies      
If Twitter can manage to operate at profit, then I might be interested in Twitter's infrastructure.
13
marknadal 3 days ago 1 reply      
I've talked/pitched (full disclosure: got a "no") some of the bigger names behind the product at Twitter. I was a little disappointed because they seem to be proud of their at least $15M+/month server costs which is partly driving their company into the ground (the user facing product hasn't improved despite "re-engineering" of their backend, at non-substantial price differences, and lack of innovation for consumers have made them lose everyone to Snapchat or anti-censorship sites).

Of tweets alone (about 200bytes per tweet), over the last decade, they probably have about 3 petabytes. Unknown to them (because of the aforementioned pride) they have 1.5 petabytes a month of free storage/caching they aren't even touching. If they switched to a P2P model like IPFS or (full disclosure: I work here) http://gunDB.io/ , but Twitter seems determined to stay as a centralized monolith. Which is too bad, because that has now become their own death - the regime change is happening and decentralized services will win instead.

Edit: Compare against 100M+ messages (1000bytes each) for $10 a day. 2 min screencast here: https://www.youtube.com/watch?v=x_WqBuEA7s8 . Even if you multiplied the feature set by 10K times, you would still be saving $12M+ a month. At this rate the Discord guys (pushing 120M+ messages a day) are doing way better - there post is on top of HN right now too, see my comment there as well. And they only have 4 backend engineers.

21
Introducing ProtonMail's Tor hidden service protonmail.com
319 points by vabmit  3 days ago   100 comments top 14
1
ergot 3 days ago 4 replies      
For those wondering how to create your own custom Tor onion adress, look no further than: https://timtaubert.de/blog/2014/11/using-the-webcrypto-api-t...

And for those who think Protonmail are the only service with a custom address, think again, because Facebook has one too: https://facebookcorewwwi.onion/

You can find a tonne more at this list:

https://github.com/chris-barry/darkweb-everywhere/tree/maste...

And staying on topic, Mailpile has their own .onion

https://raw.githubusercontent.com/chris-barry/darkweb-everyw...

2
mike-cardwell 3 days ago 4 replies      
This is not quite as good as riseup.net's onion support as it doesn't include SMTP services. See:

https://riseup.net/en/security/network-security/tor#riseups-...

 mike@snake:~$ torsocks telnet wy6zk3pmcwiyhiao.onion 25 Trying 127.42.42.0 Connected to wy6zk3pmcwiyhiao.onion. Escape character is ^]. 220 mx1.riseup.net ESMTP (spam is not appreciated)
So if your mail service supports onion addresses, then you can just replace "@riseup.net" in a users email address with "@wy6zk3pmcwiyhiao.onion".

Alternatively, your mail service could have explicit configuration in place to identify @riseup.net addresses and route them to wy6zk3pmcwiyhiao.onion instead of the normal MX records. I do this with Exim by utilising Tors TransPort+DNSPort functionality and then adding the following Exim router:

 riseup: driver = manualroute domains = riseup.net transport = remote_smtp route_data = ${lookup dnsdb{a=wy6zk3pmcwiyhiao.onion}}
Obviously this would be better if there was a way to dynamically advertise the onion address in the DNS instead of having to hardcode it in Exim.

[edit] - If they co-ordinated, Riseup and Protonmail, and potentially other similar privacy respecting mail services could send all their traffic over each other via Tor. If you work for either of these companies, please consider the possibility of looking into this sort of relationship.

3
tptacek 3 days ago 2 replies      
If you are so threatened that you feel the need to use a Tor hidden service to reach your email provider, you should know that email --- "encrypted or not" --- provides the worst protection of all possible encryption messaging options. Don't use email for sensitive communication, and certainly don't rely on the security features of any email provider for your own safety.
4
a3n 3 days ago 3 replies      
From ignorance, why would I (a non-interesting person in a nominally free country, with non-interesting interests that could nevertheless become interesting depending on political shifts and shit) want to use this hidden service, rather than plain old ProtonMail?
5
jron 3 days ago 2 replies      
Last I checked, ProtonMail required SMS verification for account creation.

Edit: When using Tor

6
dogma1138 3 days ago 0 replies      
I wouldn't recommend accessing email over TOR, especially not a paid account.

Infact I would not recommend accessing any public service that requires a unique account authentication over TOR.

This at least is somewhat more useful than facebook over TOR but unless you are accessing only free throwaway accounts (and never use those to communicate with anyone you know) using this somewhat defeats the purpose of TOR.

7
dgiagio 3 days ago 4 replies      
Could someone expand how an email service over Tor helps when the messages you sent to others still go through SMTP protocol (even with TLS) and is stored/relayed in/to unprotected severs?
8
_eht 3 days ago 3 replies      
Can anyone speak to their like/dislike of ProtonMail vs Fastmail. I currently use Fastmail and I'm happy, but always looking for something better.
9
ortekk 3 days ago 1 reply      
I wish ProtonMail would offer more email aliases with its paid plans - credentials reuse is what often allows to snoop on someone's online identity. That would really boost its value in terms of privacy.
10
benwilber0 3 days ago 1 reply      
I always get the feeling that these kinds of services are NSA honeypots. Whether intentionally or unintentionally.
11
akerro 3 days ago 1 reply      
12
tghw 3 days ago 2 replies      
If only ProtonMail could import old mail, I would be giving them money.
13
gjjrfcbugxbhf 3 days ago 1 reply      
Has anyone thought of DNS for onion addresses?
14
lazyeye 3 days ago 2 replies      
Why the funny domain name? Is there any technical reason why they cant use protonmail.onion?
22
Little Snitch 3 Protect your privacy obdev.at
373 points by ergot  3 days ago   213 comments top 37
1
tedmiston 3 days ago 4 replies      
> Research Assistant

> Have you ever wondered why a process youve never heard of before suddenly wants to connect to some server on the Internet? The Research Assistant helps you to find the answer. It only takes one click on the research button to anonymously request additional information for the current connection from the Research Assistant Database.

I'm so glad they built this feature.

The hardest part about using Little Snitch is trying to figure out whether processes that look like system or daemons are making legitimate connections.

2
diggan 3 days ago 13 replies      
Why are OSX applications in general so bad at telling website users which platforms they support? Like always, I have to keep digging around in the website, just to find out that it only runs on OSX...

Does anyone know a similar utility for Ubuntu/Linux systems? Paid or free, doesn't matter.

3
zitterbewegung 3 days ago 3 replies      
This is a prime example on how to make a landing page for a product. I understand what you are selling and why I would want it. The product looks great and I think I'll try it out after work.
4
noja 3 days ago 4 replies      
Excellent product, but needs some kind of rule sharing feature. There are so many network requests from different components that it can be overwhelming knowing what to allow.
5
lazyjones 3 days ago 5 replies      
I tried an earlier version of this and was a bit disappointed by the (apparent?) lack of information regarding these connections from applications, since there's so much going on on OS X and it's hard to tell what's legitimate and what isn't. It would be great if we could record traffic on a per-application/process basis and display it comfortably, or even have some built-in heuristics to identify common tasks like "Firefox update check" or "iCloud authentication".

It's very similar to the venerable "Spybot S&D" on Windows (the "TeaTimer" functionality, now apparently called "Live Protection": https://www.safer-networking.org).

6
Hernanpm 2 days ago 3 replies      
I noticed no one mentioned https://www.tripmode.ch/ I used to use Little Snitch before but it was to complex for what I wanted to do, allow disallow internet access to certain apps, tripmode does the trick in the simplest way I've even seen.
7
vijucat 3 days ago 4 replies      
Please steal this idea and make a product; I'll be your first paying customer:

Data Loss Protection (DLP) for retail consumers.

DLP (see http://whatis.techtarget.com/definition/data-loss-prevention... for a definition) goes beyond what Little Snitch does and does packet inspection to ensure that credit card numbers (for example) are never sent out from your network / box. Ideally, you can add regular expressions to define other PII that shouldn't be allowed to be sent out (your name, address, etc;).

DLP products exist for corporate use, but I don't know of any lightweight + inexpensive one for personal use.

WireShark, Fiddler or Charles can incorporate this functionality, if I am not wrong. Not sure how one would MITM SSL with WireShark, though.

8
rbritton 3 days ago 2 replies      
Not related in any way, Little Flocker[0] is a similar program but for file access. It's a little rough around the edges but has been improving steadily.

[0]: https://www.littleflocker.com

9
bsmartt 3 days ago 1 reply      
why was this posted today? I bought Little Snitch 3 in January 2013. I was thinking maybe this was a new major version but it's not.
10
djsumdog 2 days ago 1 reply      
There's a great Defcon talk about someone breaking Little Snitch:

https://www.youtube.com/watch?v=sRcHt-sxcPI

11
jstoja 3 days ago 1 reply      
> A firewall protects your computer against unwanted guests from the Internet.> But who protects your private data from being sent out?

A firewall? No kidding, a firewall is not supposed to only block incoming traffic...

12
mostafah 3 days ago 1 reply      
Ive been using this happily for a long time. For those taken back by the endless prompts on the first run: thats only for the start. Select forever for connections you trust and youll soon have much less prompts.

On a side note: the developers also have Micro Snitch, an app that warns when the camera or the microphone on your mac is in use.

13
mellamoyo 3 days ago 8 replies      
Any similar software recommendations for Windows?
14
koolba 3 days ago 2 replies      
How does this work? Does it override the networking DLLs to proxy the socket creation calls?
15
iends 3 days ago 13 replies      
Those of you who own Little Snitch...do you regularly block outgoing connections from applications you regularly use?
16
alphonsegaston 3 days ago 0 replies      
Little Snitch is at once both great and horrifying. If you watch the day to day stuff that happens on MacOS, you'll see that Apple's reputation for security and user privacy is a pretty low bar. Aside from the constantly pinging Apple defaults, so many third party apps are just all the time phoning home to corporate servers when they're not even in use. Chrome can really just look for updates when I open it, not check in with Google about god knows what every thirty minutes.
17
therealmarv 3 days ago 1 reply      
Serious question: Can I use only profiles (e.g. no connection until VPN is connected) and the rest of the time Little Snitch should behave like it's not installed? I'm not a big fan of watching every connection... have done this in the distant past with Zone Alarm and Windows and it was more bothering than anything else. I also doubt it increases my personal security a lot.... especially when I think about my normal Android phone which is sitting beside my PC.
18
rwinn 2 days ago 0 replies      
First thing I install on any new system, couldn't recommend it more!

And the ability to do per-application captures and open them in wireshark is excellent for debugging.

19
jedisct1 3 days ago 5 replies      
Little Snitch is a fantastic way for people to shoot themselves in the foot.

Most people using it have no clue what they are doing, block random things, and prevent software from working as expected. Not only this can make things less secure by breaking features such as automatic updates, it also makes developer's life miserable by having to provide support to people running their software in a half broken environment.

20
problems 2 days ago 1 reply      
Does Little Snitch catch process injections (ie: I am currently running in EvilMalware, I open up Chrome, create a new page, write my code into it and create a new thread in it), or is it vulnerable to the same problems of Windows firewall applications before LeakTest and the like. The good Windows firewalls now are able to catch this kind of thing.
21
andrenotgiant 3 days ago 1 reply      
I wish something like this could run at the router level. I am certain my low-end IoT devices are sending out data I don't know about.
22
mkj 3 days ago 0 replies      
Objective Development (the developers) are a nice company, also providing V-USB - a bitbanging USB implementation for AVR microcontrollers without USB support. https://www.obdev.at/products/vusb/index.html
23
bisby 2 days ago 0 replies      
4-5 years ago when I last used a mac for work, there was a program that had an unlimited evaluation period and was just setup to nag on launch (like winzip). using little snitch just blocked the nag (literally the license did was remove the nag, so it didnt affect functionality). In the end, I wound up not using the program anyway - I really was just trying to evaluate it without the nag. For some reason sublime text comes to mind? I think I wound up just going back to vim

Installing little snitch, I got overwhelmed by how much stuff was trying to make calls in and out. It really does serve its purpose, but you also have to have an idea of what you should be letting out, you can easily break things and if you just "allow all" it somewhat ruins the point of having it.

24
Sykox 3 days ago 3 replies      
Is there one absolutely similar to windows? Closest i found was GlassWire
25
twsted 2 days ago 0 replies      
I think these features should be included in every OS nowadays, like we have firewalls.

Anyway, I will probably buy this app, even if I share some concern others have about its own network calls.

26
markneub 2 days ago 0 replies      
Has anyone figured out how to stop Google's autoupdate process (ksfetch) from tripping LS nonstop? It spawns multiple new temporary processes when checking for updates, and LS requires a path to a specific process file to block it. This has made LS unusable for me since uninstalling all Google products isn't an option for me.
27
libeclipse 3 days ago 4 replies      
Something like this would be brilliant on Android. Anyone know anything related?

It'd be great if it was for non-root too, but I'm not sure if it's possible.

28
Khaine 2 days ago 0 replies      
Little Snitch is great. You need to have a strong understanding of networking and the apps that you use, to use it successfully. It is great at opening your eyes to what apps are trying to connect where, and by catching a cap you can investigate what they are sending.
29
icanhackit 3 days ago 0 replies      
Long time LS user and love it - yes the constant notifications will tax your Qi but once you've set up the bulk of your rules it'll give you a lot of peace of mind. Also grab Lingon X if you're serious about control.
30
watersb 1 day ago 0 replies      
FWIW, I love Little Snitch and have used it for at least ten years.
31
mattcoles 3 days ago 3 replies      
Is it open source? Couldn't find anything on their site which is disappointing.
32
lwfitzgerald 3 days ago 1 reply      
I'm currently using LS, but one of the problems I have is that it doesn't support wildcard domain rules. This means ephemeral hosts quickly build up a large number of rules which soon become redundant.
33
benologist 2 days ago 0 replies      
One day consumer rights protection agencies are going to scrutinize what we are doing in the background just like they're starting to do to ads.
34
FullMtlAlcoholc 2 days ago 0 replies      
If anyone is looking for a summer application that won't inundate you with so much information, try radio silence
35
thehashrocket 2 days ago 0 replies      
Little Snitch reminds me of Zone Alarm from back in the day.
36
admax88q 2 days ago 3 replies      
Protect your privacy by running this proprietary application!
37
teaearlgraycold 2 days ago 2 replies      
This seems like a joke given that it's not open source.
23
Philadelphia Poised to Ban Employers from Asking Hires About Salary History wsj.com
234 points by prostoalex  1 day ago   234 comments top 25
1
hackuser 1 day ago 8 replies      
An important detail: Comcast, backed by the local Chamber of Commerce, has threatened to sue the city over this law. On what basis? The corporation's Constitutional First Amendment free speech right to ask the question:

http://www.investopedia.com/news/comcast-threatens-sue-city-...

Are these the same free speech rights the Supreme Court recently bestowed on the corporations? I remember that at the time legal analysts said it would make it hard to regulate corporations. How can you regulate, for example, what goes in a prospectus issued to shareholders if the corporate has free speech rights?

2
jimnotgym 1 day ago 2 replies      
Interesting that various people here are horrified by restrictions on what you can ask in an interview. In the UK there are various restrictions on what you are allowed to ask, such as a candidates age. Actually the main risk to a business from this is a careless interviewer asking by mistake.

An enlightened employer should not find any of this restricting at all. All of the worst interviewers I have faced have always asked about my salary, and recruitment consultants always do (they also tend to lie about your expectations to potential employers in order to get interviews). It is a deliberate attempt to screw you down and is totally unnecessary.

3
hlieberman 1 day ago 0 replies      
Massachusetts already passed a law that bars companies from asking potential hires about their salary history until after they've made an offer.

(The law is signed, but doesn't go into effect until June.)

4
snarf21 1 day ago 4 replies      
I understand the goal of this law but don't see how it solves it. I usually ask people what their current salary is now. I want to offer them (at least) a modest raise to come here. I want them to be happy and excited. I also want to not waste a ton of time negotiating salary if they want 50K more then I have available in my budget.

You are worth whatever you can get someone to pay you. Some people may come in with a lower ask but then you have yourself, are they a bargain or are they just less talented? If they come in high, are they overpriced or are they just that good?

I also think this only works for the employee if there is a shortage in this field and employers are desperate (but the market will force those prices up) or if you are trying to just get more than your experience might warrant. Let's say you are a knowledge worker making $80K and hoping to go to $90K. So, in the new rules, the employer asks for your expected salary and you say "give me your best offer". The employer will either get annoyed and work with the candidate that said $85K in response or they low ball you at $60K because they can play games too. You say "no" and they ask what you are looking for, rinse and repeat. It seems like you still end up at the same place with the same result except the negotiation process sucked. As a candidate, I don't see the downside of saying I want X+ dollars. I'd rather know about the job and culture and other things than $5K more or less.

5
aswanson 1 day ago 1 reply      
I answer what I deem unethical/ unfair lines of questioning with any answer I feel necessary to advantage myself in negotiating. Others should do so as well.
6
hackuser 1 day ago 3 replies      
Most innovation and progress now is at the city level (I'm not the first to say it). Name the last thing government achieved at a national level in the U.S. - there's health care, but that's now looking shaky.

The priority of one major party in the U.S. is to shut down government as much as possible; that party can block progress everywhere but in cities, where they have no power for now. However, they are trying to pass 'pre-emption laws' in many state governments which block cities from doing anything the Republicans in the state capitals don't like. So much for the belief in liberty and local control.

7
gigatexal 1 day ago 0 replies      
Good. Put the negotiating power back into the hands of the employees. Get an offer. See that it's too low based on previous compensation and then say so. And then proceed from there. I've burned my own ability to negotiate by sharing current salaries before but that was with recruiters.
8
otakucode 1 day ago 1 reply      
It is bizarre that so many people tolerate this behavior by employers yet also cling to ideas about compensation being linked to the value of the work being performed (things like 'work hard and you will make more'). If compensation is determined based primarily upon the value of the work being done, prior salary could not be more irrelevant.

But of course, what people are paid has nothing to do with the value of their work any longer. Computers and software unfortunately make that socially untenable. They make employees so abundantly productive, and increase that productivity so quickly, that it would have to translate into very high salaries and high annual raises for many, if not most, employees. It doesn't take a genius to look at the average profitability of corporations overall since 1980 to see the meteoric rise coupled with total wage stagnation for non-executives.

Get 2% more productive, then a 2% raise is acceptable. Get 100% more productive, then a comparable raise is unacceptable. Especially if "the computer is doing all the work."

9
tabeth 1 day ago 1 reply      
A society truly run by the people, ala individuals, would have banned this long ago. Just think about it: for an average person, what benefit do they receive by allowing employers to ask about past or current salary information? None.

Wouldn't be surprised if there have been measures to ban this that were ultimately struck down by business people.

The less information that's shared with employers, the more employers will have to refine their actual process for vetting people. Whether this means more accurate metrics, that correlate to actual employee performance, or something else, who knows. Either way, I believe it's good for the people. However, we will have to be careful, if we go this route, to not slip into nepotism (well, more than we already have).

10
jimmywanger 1 day ago 0 replies      
There are two things that spring out instantly.

First, the motivation is to close gender pay inequality. Assuming such a thing exists (women get paid less for the same job), where is the evidence that this policy would reduce the supposed gap?

Second, they're prohibiting you from asking, not prohibiting employers from not choosing you for failing to answer with the truth.

It's part of negotiation. Would it be legal to ask "what's the least amount of money we can give you for you to accept the job?" (Which, btw, is the question they're indirectly asking you.)

Why are they trying to outlaw negotiation skills for a somewhat unproven premise and an unclear cause and effect relationship?

11
jacquesc 1 day ago 0 replies      
I support this, but I also think employers should be banned from requiring a credit check for employment. There are regulations on discriminating on race, gender, etc. The one missing piece is discriminating against the poor (which indirectly tie back to race, gender). It's still amazing to me that this is a practice that's still allowed for a society that purports to value social mobility.
12
re_todd 1 day ago 2 replies      
It always amazes me when I hear about people asking this question. Don't these hiring managers consider for a second how intrusive and rude it is? Aren't they concerned they may come across as douchebags for asking it? And yet I've been asked this several times by people that act like it's no big deal, like asking how was the traffic on the way in. Are they under pressure from the top? Or are hiring managers in general lacking in scruples?
13
jackbrian 1 day ago 1 reply      
> Councilman William Greenlee, a Democrat, who sponsored the bill, says the measure wouldnt prevent employers from setting a fair salary, bar salary negotiations with prospective hires, or keep job candidates from voluntarily sharing their pay history. But if passed, violators who quizzed would-be hires about salary could face fines up to $2,000.

Even if passed, the penalty hardly seems disincentivizing.

14
gesman 1 day ago 4 replies      
Asking doesn't hurt anyone.However giving certain answers could.

What prevents employees from claiming that this information is "confidential" (polite version of "it's not your business").

Or outright lying about it?

15
criveros 1 day ago 2 replies      
I added 10k to my salary last time I was asked.
16
tossaway1 1 day ago 1 reply      
Massachusetts did this last summer. Seems like a strange trend. Not necessarily bad (though it makes things harder for me as a hiring manager) but I wonder what group is pushing for this. Laws don't tend to get passed unless they're backed by special interests.
17
general_ai 1 day ago 1 reply      
Can we go a little further than this? Can we require employers to post the details of the compensation package that comes with each job? That would save a lot of time for all involved and obviate the need to even ask this question.
18
davej 1 day ago 1 reply      
I'll either politely refuse to answer or if I have thought about it beforehand then I'll give a number that guides them towards the salary I am looking for this time around.

I don't really mind the question. It's a little rude and it's only asked to get a gauge of salary expectations so why not ask what the current salary expectations are directly.

19
gdulli 1 day ago 2 replies      
That's too bad. My salary history tells a good story about me, that employers have found me more valuable than average. It's a pretty tangible and direct signal. In an interview it's convenient for them to ask and for me not to have to awkwardly bring it up.
20
anon31415927 1 day ago 1 reply      
Most people don't realize that companies can get salary history from a number of sources. ADP actually provides this as a service to larger customers.
21
tyingq 1 day ago 1 reply      
Paywalled, so I can't read it. Curious if this would ban asking about even current salary. That will make things interesting in Philly.
22
danieltillett 1 day ago 0 replies      
Every regulatory hurdle to hiring decreases the propensity of businesses to hire marginal candidates. The easier you make it to hire and fire people the more marginal candidates will be given the chance. There is no free-lunch here.
23
iamthepieman 1 day ago 0 replies      
All of my employers are already banned.
24
logicallee 1 day ago 2 replies      
[Edit: if a potential employer asked my previous salary] I would have zero qualms (zero) saying any number I wanted. If I was asked for paystubs I would say I'll get them later and add that the number may not have been exact, it could be off substantially I want to be upfront about that. If I have to put a number in a form and sign that I swear it is accurate, I would put an asterisk and explain that it is indicative of what I want them to think.

I don't see ethical problems with this behavior. It is practically what I actually would do.

If HR absolutely cannot start without my notarized sworn copy of pay stubs then I would not work there, and their loss. For anyone else, HR's job is to find qualified people, they are not going to kick you out once they've started onboarding you.

You don't have to lie, but you don't have to answer their questions any more than they have to ask yours about their company - HR doesn't work far from payroll, if as part of your interviews you asked to know the CEO's salary, would they tell you? Or the ratio between then lowest and highest salary at the company? (2.5x, 3x, 5x, 10x, 50x, 100x)

Things don't have to be hard and fast. If they don't like how I roll they can hire someone else. If I don't like their level of transparency I can refuse their offer.

Plus they're free to fire me if they don't like me.

You should answer illegal questions the same way. e.g. in an interview, how old are you. Say whatever you want, just add "I don't like to give am exact age just use that". If they're surprised that a 55 year old shows up too bad for them. Fire you if they don't like it. You're not paid to be 25 and you're not paid to have a certain salary history. You're paid to do a certain job - everything else is just friction.

Say whatever you want, as long as you can do the job. Don't lie about degrees or credentials.

Don't practice law or perform surgery without a license.

25
FrancoDiaz 1 day ago 1 reply      
24
Neocities: Free, modern Geocities reboot neocities.org
411 points by matthberg  2 days ago   99 comments top 17
1
kyledrake 2 days ago 15 replies      
Hi! I started Neocities. I read HN regularly.

Neocities was actually launched and bootstrapped on HN about 3 years ago, and donations from HN users bankrolled the first year of operating it. Things have been going really well, the site is growing and still sustains it's own existence through donations and supporter accounts.

I still work on the site heavily. We're launching some big features soon (more space, Github webhook deploys, etc.). We just finished migrating to SSL (for everything, including hosted sites).

I've had to do some pretty crazy stuff to make the site work, some of which is not documented well and I think the HN crowd would find pretty interesting. For example, we figured out how to run our own global anycast network for "cheap". I would love to share how to do that with people, there is approximately zero information online or in books on operating anycast networks.

My new years resolution was to get better at writing about some of the crazy tech I've had to do for the site. If you're interested, there's an RSS feed you can subscribe to for our blog where it will be posted in the future: https://blog.neocities.org/feed.xml

Again, thanks go to HN. Site literally wouldn't exist without it, you were basically our seed investor. Feel free to ask me questions, I'll try to answer. That's something you're supposed to do with your investors, right?

2
jmduke 2 days ago 4 replies      
Original discussion from Neocities' launch: https://news.ycombinator.com/item?id=5918724

Personally, I don't have any reason to use Neocities but I am enormously happy that it exists. Geocities and Angelfire and all of those weird web sandboxes are where I first wrote code, and what led me to pursue software engineering as a career.

3
tbirdz 2 days ago 1 reply      
Kyle, thanks so much for all your hard work setting up Neocities! I think it's a wonderful thing you are doing.

Also, since we're talking about neocities, here, does anyone have any cool neocities sites they know of and want to share? Just post a reply to this comment, so folks who want to have other neocities discussions can easily collapse all the links at once.

4
steve918 2 days ago 0 replies      
I really love Neocities. I have used it to host https://www.makersatwork.com/ for some time and have donated to Neocities and participated in their Kickstarter campaign.
5
avenoir 2 days ago 0 replies      
Ah, the name brings back some memories. A long time ago, about 15 years in total, when hosting a PHP site was too expensive for a 15-year-old and free hosting usually came with ads there was a community on neopages.net which offered no cost ad-free PHP hosting. The only thing you had to do to get it was submit an entry on neopages forums containing a link to your current site along with a description of why you deserve a subdomain on neopages. You'd then get judged by hosted members and either get an account or try again. Neopages was the community of teenage geeks that got me hooked on PHP after I outgrew GeoCities. Curious if any of you guys know what i'm talking about or had a site hosted there? :) Some of you are likely in your early to mid 30s now.
6
partycoder 2 days ago 0 replies      
I used Geocities. I think it still exists in Japan.Back then, HTML templates/generators were very mainstream.

Nowadays that use case was largely replaced by what we now call blogs. Many of those websites were in fact proto-blogs.

Later, as bandwidth increased, digital cameras became mainstream and processing power increased, photo sharing was added as common use case. Try downloading a 2 megapixel+ PNG with a dial-up connection... very slow. Now we take it was granted.

Now in addition to blogs, you also have Google Sites, wikia, and countless others... including Wikipedia.

7
lawpoop 2 days ago 2 replies      
let's create a crowd-sourced effort to migrate archived geocities sites to neocities.
8
Kapura 2 days ago 0 replies      
I've used Neocities for something like two years now to host my game development portfolio and other miscellaneous docs. I've been super happy with the ease of creating and uploading content, and I recommend it whenever somebody technical is looking for a cheap static hosting option.
9
bredren 2 days ago 0 replies      
The welcome page, including the UX on how card data entry works is really great. Taking notes.
10
Apocryphon 2 days ago 1 reply      
I wonder what the Yahoo! dismantling will do to all of the Geocities content. Hopefully they lay dormant in archive somewhere, clearly labelled.
11
jimnotgym 1 day ago 0 replies      
I showed my 9yo this and now he has a site about minecraft. I showed my 14yo and now she is building a gallery for her manga. My other kid is away but he is the one who is really going to love it. I really like this site.
12
cJ0th 2 days ago 0 replies      
Thanks for this!I remember the first time Neocities was introduced to HN. Very happy to see it is still going strong!
13
dicroce 2 days ago 1 reply      
I've always thought geocities could have been facebook. People just wanted a page to post pictures and share with their friends. Geocities could have automated something like that....
14
ausjke 2 days ago 3 replies      
what is the key difference comparing to wordpress.com
15
RUG3Y 2 days ago 0 replies      
I've been on Neocities for a while and I really love it! It's refreshing.
16
rdiddly 1 day ago 0 replies      
Neocities is a brilliant name on 3 levels:

Obvious: rhymes with Geocities, and is one letter different from Geocities

Less obvious: the prefix neo makes it a "new" Geocities

Subtle, and my favorite: seemingly everyone on the web today is jacked into a Matrix of "walled gardens" where they are tended and farmed, and Neo is the name of a guy who undoes that shit.

17
hoodoof 2 days ago 0 replies      
What is the underlying architecture?
25
Investing Is More Luck Than Talent nautil.us
256 points by dnetesn  1 day ago   199 comments top 31
1
scottmsul 1 day ago 11 replies      
I've read a bit into econophysics. In stat mech, the exchange of energy between atoms is assumed to be random. When the exchange of energy is random, the resulting distribution of energies is a Gibbs distribution, which means that the probability of atoms with higher energies falls off exponentially.

The application is that if you replace energy with money, and atoms with people, then the same equations hold for a model of people who randomly exchange money. Therefore if investments were truly random, you would expect the distribution of investors' wealth to follow an exponential distribution, not a power law! This directly contradicts the article.

Interestingly enough, the bottom 99% follows an exponential distribution, while the top 1% follows a power-law, and the transition is very sharp (eg, wealth plots have a "kink" in them).

Brief introduction to econophysics for the mathematically inclined:https://arxiv.org/abs/0709.3662

2
valuearb 1 day ago 12 replies      
This article basically repeats old economic saws of dubious validity. The counter argument is Warren Buffett. Last I looked at it (going back to his Buffett Partnership days), he beat the market every year but one for his first 30 or 40 years. Even better, he was beating the market by an average of 20% a year during his partnership days, and something like 10% a year during his first 20 years running Berkshire Hathaway.

It's not only statistically impossible for Buffett to be a fluke, it's statistically impossible for him not to possess a a skill providing a substantial edge in market investing. Not a "1% a year" type skill.

Nowadays and for the last 20 years or so, Buffett has been managing hundreds of billions of dollars. The immense size of his portfolio restricts his opportunities to a far smaller pool of potential investments and his edge over the market has clearly declined because of that restriction. But he's still beating the market the vast majority of the time.

3
Applejinx 1 day ago 2 replies      
"How big can this talent differential be and still stay statistically consistent with the power law wealth distribution we see in the real world?

It turns out that it cant be more than about 1 percent.2 A larger talent differential would produce a wealth distribution that is even more extreme than the real one, and that would not follow a power law."

I thought this article looked like it was going to be very obvious, but that passage got my attention. I'm all for experiments of this nature: one day AI will be doing it as a matter of course, on a massive scale.

Lines up with my observation, too: talent makes a difference, and it's about 1% over time. Kind of like compound interest. You can win out if you're very persistent and very determined and you fail a lot, because you're shooting for that 'luck' moment. There's no telling where it will strike, because it's luck! You have to stay in there and not go broke because it's luck, there's very little correlation between merit and success.

I suspect if you went by tenacity rather than 'talent', the number might be a lot more than 1%. But bozos can be tenacious too, which is a daunting thing to consider.

4
ImTalking 1 day ago 1 reply      
> We all know stories of ambitious and talented people like Steve Jobs or Bill Gates, who grew companies and created great wealth.

True. Yet both were extremely lucky at the beginning. We all know the story but Gates' mother knew IBM President John Opel, his father was a prominent contract lawyer, and IBM originally wanted CP/M but Gary Kindall missed the meeting to fly his plane. And Microsoft retaining the rights to the OS was, in hindsight, a very grave mistake.

Jobs had the charisma and drive, but without Woz I'm sure we wouldn't be talking now about Apple.

But given the opportunity that this initial luck gave them, they absolutely maximised it, so good on them.

5
ikeboy 1 day ago 1 reply      
>If the ratio of 50,000 were to hold for other traits, it would imply individuals who are 53 miles tall, have IQs of 5 million points, and live to be 4 million years old.

That's just not how to do statistics, sorry. First of all, one of those is not like the others: IQ is normalized, the others are not. There are so many assumptions about linearity and distributions here that would need to be a lot more rigorous before anything like this could be said.

See http://www.thedailybeast.com/articles/2013/02/06/department-... for an actual interesting point regarding income and bell curves.

6
sgustard 1 day ago 4 replies      
The richest eight people now have the same wealth as the poorest 50 percent. Most of that wealth comes from investing (rather than labor). There is no probability model that would lead to this result; eventually someone on a "hot streak" is bound to lose.

What we're seeing is that luck may give some random people a boost at first, but that boost is sufficient to let them take all the winnings over time. The rich get richer. Investing success and wealth building follow a Darwinian rather than a probabilistic model.

7
atemerev 1 day ago 0 replies      
This article deliberately confuses normal distribution, which applies to such things as height or IQ, and power law distribution, which applies to wealth, social connections, city sizes etc.

I think nobody is surprised about the fact that the largest cities are more than million times larger than the smallest settlements. Or that that top 1% of the largest cities includes the sizable proportion of human population.

8
jacques_chester 1 day ago 3 replies      
Even the Steve Jobs example of a super-talented individual relies on luck.

In the universe where he didn't meet Steve Wozniak, Steve Jobs was probably consistently in the top 3 Mercedes salesmen in California.

Not to mention: merely being born in a wealthy country during a period of relative peace is so much luckier than the lot of humans throughout history that it is almost worthy of bad science fiction.

9
acd 1 day ago 0 replies      
There is a book where monkey throws darts at a dartboard beats professional investors. There is also a similar wherecats pick stocks with their whiskers and beats the pros.

A Random Walk down Wall Streethttp://www.forbes.com/sites/rickferri/2012/12/20/any-monkey-...http://www.marketriders.com/investing/why-googles-investment...

10
erikb 1 day ago 1 reply      
Do the rich really keep getting richer? I would assume that as you have more money you don't just have more opportunity but also more competition. There are lots of people who try to take your money one way or another without any return.
11
snarf21 1 day ago 0 replies      
Although I completely agree that luck is a major factor, the author loses me when he takes his hypothetical "lucky people get 30% gain, unlucky people get 10% loss" and builds on top of that as if it was a provable fact. It is okay to use this as a mechanism to illustrate that this matches the power law distribution that we see in reality. This part makes sense as an intellectual exercise.

Applying 1% on top of that as talent and rerunning it, stayed within power law distribution but making talent larger would break it. The faulty logic to me is what makes the "lucky 30% gain, unlucky 10% loss" fact? It seems much more likely that there is an 8% base (index funds, e.g.) with luck playing a +-10% and talent being another ~7% (as a hypothesis). Also, remember his "game" to prove luck > talent only ran for 20 years. I'm curious if I plugged this into a spreadsheet and ran for 50 years if I couldn't get the same power law distribution with greatly different factors. Mainly, it requires a lot of luck AND talent.

12
mack1001 1 day ago 1 reply      
Among the 1% there exists multiple deep and consistent insider networks where stock tips and wealth creation ideas are exchanged. So the 1% crowd thrives from being in that position.
13
leed25d 1 day ago 2 replies      
People have been bullshitting us for centuries that market investing is a game of skill. If it really comes down to luck then it is gambling, and it should be treated (legally) as such.
14
DeBraid 1 day ago 0 replies      
One of the best 'Talks at Google' via Michael Mauboussin "The Success Equation: Untangling Skill and Luck"

Thesis: where absolute talent/skill is high, variance in relative talent is low, thus luck > skill.

https://www.youtube.com/watch?v=1JLfqBsX5Lc

15
amelius 1 day ago 0 replies      
At least with investing you can spread your risk. In contrast, if you are a programmer at a startup, you typically have all your eggs in one basket. In other words, more talent than luck.
16
cies 1 day ago 0 replies      
Maybe "insider trading" is often rebranded as "luck".
17
paulus_magnus2 1 day ago 0 replies      
Dubious article relying on Gell-Mann Amnesia.

We'd easily spot the fallacy if it read "engineering correct software Is More Luck Than Talent"

18
ariwilson 1 day ago 0 replies      
If you can't optimize for talent, you should optimize for costs. Investing in low cost passive index funds is best for most people's low risk plans (retirement, house, etc).
19
myf01d 1 day ago 1 reply      
It certainly needs considerable amount of luck to earn the first big $ on your investments, but you should be ruthless, sociopath, cold blooded, patient and hard-working to continue growing & become like Icahn or Buffet.
20
edblarney 4 hours ago 0 replies      
This article represents a disingenuous premise.

It's also grossly mis-titiled.

'Wealth' is mostly not generated by speculative investing.

And yes - most 'investing' is luck, but investors already know that.

But declare that some kid who worked his pants off through school, got into a half-decent Uni, and went on to get a high paying job - to his brother, who didn't do much in school, and took a job at their fathers auto-shop earning a respectable but relatively small salary - in terms of 'luck' is just unfair.

21
readhn 1 day ago 6 replies      
There is no investment. There is only speculation. Speculation that the asset price will increase. Speculation that your purchase will return higher value in the future. In order to be successful at speculation you do have to have certain talents and character traits. Study most successful speculators of or times and you will see certain patterns and common traits.
22
cs702 22 hours ago 0 replies      
Here's Warren Buffett's response, published on May 17, 1984[1]:

"I would like you to imagine a national coin-flipping contest. Lets assume we get 225 million Americans up tomorrow morning and we ask them all to wager a dollar. They go out in the morning at sunrise, and they all call the flip of a coin. If they call correctly, they win a dollar from those who called wrong. Each day the losers drop out, and on the subsequent day the stakes build as all previous winnings are put on the line. After ten flips on ten mornings, there will be approximately 220,000 people in the United States who have correctly called ten flips in a row. They each will have won a little over $1,000.

Now this group will probably start getting a little puffed up about this, human nature being what it is. They may try to be modest, but at cocktail parties they will occasionally admit to attractive members of the opposite sex what their technique is, and what marvelous insights they bring to the field of flipping.

Assuming that the winners are getting the appropriate rewards from the losers, in another ten days we will have 215 people who have successfully called their coin flips 20 times in a row and who, by this exercise, each have turned one dollar into a little over $1 million. $225 million would have been lost, $225 million would have been won.

By then, this group will really lose their heads. They will probably write books on How I turned a Dollar into a Million in Twenty Days Working Thirty Seconds a Morning. Worse yet, theyll probably start jetting around the country attending seminars on efficient coin-flipping and tackling skeptical professors with, If it cant be done, why are there 215 of us?

By then some business school professor will probably be rude enough to bring up the fact that if 225 million orangutans had engaged in a similar exercise, the results would be much the same 215 egotistical orangutans with 20 straight winning flips.

I would argue, however, that there are some important differences in the examples I am going to present. For one thing, if (a) you had taken 225 million orangutans distributed roughly as the U.S. population is; if (b) 215 winners were left after 20 days; and if (c) you found that 40 came from a particular zoo in Omaha, you would be pretty sure you were on to something. So you would probably go out and ask the zookeeper about what hes feeding them, whether they had special exercises, what books they read, and who knows what else. That is, if you found any really extraordinary concentrations of success, you might want to see if you could identify concentrations of unusual characteristics that might be causal factors.

Scientific inquiry naturally follows such a pattern. If you were trying to analyze possible causes of a rare type of cancer with, say, 1,500 cases a year in the United States and you found that 400 of them occurred in some little mining town in Montana, you would get very interested in the water there, or the occupation of those afflicted, or other variables. You know its not random chance that 400 come from a small area. You would not necessarily know the causal factors, but you would know where to search.

I submit to you that there are ways of defining an origin other than geography. In addition to geographical origins, there can be what I call an intellectual origin. I think you will find that a disproportionate number of successful coin-flippers in the investment world came from a very small intellectual village that could be called Graham-and-Doddsville. A concentration of winners that simply cannot be explained by chance can be traced to this particular intellectual village."

[1] https://www8.gsb.columbia.edu/articles/columbia-business/sup...

23
deepnotderp 1 day ago 2 replies      
Don't firms like Renaissance and Virtu destroy this notion?
24
m3kw9 1 day ago 0 replies      
Luck is when preparation meets opportunity.
25
hvd 1 day ago 0 replies      
great way to tell the masses and countless business school grads that its not even worth trying. Any astute investor will support articles like this since it reduces competition. Thinking is bad. Long live index funds.
26
known 1 day ago 0 replies      
"I'm a great believer in luck, and I find the harder I work the more I have of it" --Thomas Jefferson
27
imaginenore 1 day ago 0 replies      
That's demonstrably false. How would you explain quants? All the financial firms that exist solely because of how good they are at buying and selling securities?
28
Bud 1 day ago 0 replies      
Pretty funny that this apparently got flagged. Perhaps a couple entitled investors didn't like this particular message.
29
ry4n413 1 day ago 0 replies      
Says the professor who has never worked in the investment industry.
30
FullMtlAlcoholc 1 day ago 0 replies      
If your takeaway from this is most, if not all, investment is pure speculation, you are either very naive or willfully ignorant.

I'm nit saying talent per se is a signifixant factor, but this view suffers frim the sane nakady that a lot if wconomic theories do... an idealized situation/world where everyone plays by the rules. To some individuals, life is not a game, but a winner take all conflict and they will Kobayashi Maru the situation whille others are still playing a game of chance

31
downandout 1 day ago 1 reply      
In the short term, investing may indeed be more luck than talent, however this article strives to imply that market-beating results are essentially pure luck, which simply isn't the case over time. Over time, the best investors will always wind up with market-beating returns, and the worst investors will always wind up with market-trailing returns. Like any game with significant short-term variance, those with the greatest skill might not be readily apparent from a small sample size of results - I might beat Phil Ivey for a day or even a month at a poker table. But if we play long enough, I literally have no chance of being ahead of him. The same holds true for investors.

Articles like this seem like a cop-out for failed investors, and perhaps an argument for using index funds. If it's all luck, then your own failings aren't your fault. But that view doesn't apply to most areas of life, including and perhaps especially when it comes to investing decisions.

26
How to Get into Natural Language Processing ycombinator.com
331 points by craigcannon  2 days ago   73 comments top 20
1
d_burfoot 2 days ago 5 replies      
> Why is NLP Hard? ... Language is highly ambiguous - it relies on subtle cues and contexts to convey meaning.

This is true, but it is only part of the answer.

Another part of the answer is what I call the Long Tail of Grammar. It turns out that if you try to write down all the rules of grammar, you will not get 40 or 60 rules, but something more like 100s or maybe even 1000s of rules. Most of those rules are obscure, rare, archaic, or useable only in specific contexts or with specific words. However, they are part of the language, a native speaker will be able to use them and comprehend them without difficulty, and an NLP system must be able to "understand" them in order to extract the correct meaning from a sentence.

As just a minor example off the top of my head, compare the phrase "peeled peach" with "hairy-peeled peach". The former phrase means a peach without a peel, while the latter means a peach with a hairy peel. So a good NLP system must not only recognize the existence of the two grammatical rules, but also be able to disambiguate them correctly.

2
blcArmadillo 2 days ago 1 reply      
I think this is a good idea for a series. Although I think more detail needs to be given on the actual path, that is after all the purpose of the series. Most of this article seemed to be describing what NLP is and why it's hard. This isn't bad and some attention should be given to it but people looking to find the path into NLP will already be familiar with most of this information. I was expecting a bit more of a syllabus type format. There was mention of needing some college level algebra and statics, I would have liked more detail in this area with links to more resources (classes, articles, datasets, etc). Keep up the good work!
3
demonshalo 2 days ago 3 replies      
How? just get started working on a fun problem. A good place to start is keyword extraction. You don't need a PhD or expensive tools. All you need is some free time and willingness to read some cool stuff.

Copy a few articles into text files and get working on implementing some of these methods until you have enough of an understanding to construct your own methods for the fun of it.

Here's some good reading material:

https://www.facebook.com/notes/facebook-engineering/under-th...

https://www.researchgate.net/profile/Stuart_Rose/publication...

http://cdn.intechopen.com/pdfs/5338.pdf

https://arxiv.org/pdf/1603.03827v1.pdf

https://www.quora.com/Sentiment-Analysis-What-are-the-good-w...

http://hrcak.srce.hr/file/207669

http://nlp.stanford.edu/fsnlp/promo/colloc.pdf

https://arxiv.org/ftp/cs/papers/0410/0410062.pdf

http://delivery.acm.org/10.1145/1120000/1119383/p216-hulth.p...

Edit: Don't get deterred by the math formulas in these papers. They look far more complicated than they actually are.

4
andrewtbham 2 days ago 1 reply      
I researched deep learning for nlp for a year and compiled this list of papers and articles about some of the most interesting topics.

https://github.com/andrewt3000/DL4NLP/blob/master/README.md

5
hobofan 2 days ago 1 reply      
I like the idea of the Paths series, though some of the points in this first article read like they could be written about most "emerging technologies". Anyway, I'm looking forward to the next one!

The two questions about the PhD's do feel a little bit misplaced for a startup audience. Who here stops and thinks "Am I supposed to have a PhD to do that?", when setting out to start something new? (<insert theranos reference here>)

6
deegles 2 days ago 3 replies      
There are a ton of libraries and tools available for NLP, so I feel that side is relatively mature.

What I want are more tools for Natural Language Generation. Can anyone recommend some good ones? (beyond what's on Wikipedia)

7
visarga 2 days ago 1 reply      
If you want to play with NLP, then just try Gensim, sklearn and Keras. If you're serious about NLP, it's hard stuff, you need a PHD in the field.

In a way, vision is easier. Instead of discrete symbols (words) it's continuous signal which are much easier to interpret and generate from neural networks. By comparison, best language models are behind best image generation models (2-3 years behind, in my estimation).

For example, there are few applications of GANs to text, and many applications to images, GANs being the hottest thing in deep learning now. So you have to keep in mind that NLP is by and large still not solved. There is no decent conversational chat bot yet. We can reason over small pieces of text but that is far from full understanding. NLP at this level is hard.

What you can easily do now is to classify text, detect sentiment, entities, word vectors, grammatical parsing and summarization. All are low level stuff.

8
posterboy 1 day ago 1 reply      
> Take this simple example: I love flying planes.

> Do I enjoy participating in the act of piloting an aircraft? Or am I expressing an appreciation for man-made vehicles engaged in movement through the air on wings

Clearly the latter, as the former begets the infinitive, "I love to fly ...".

Maybe I am wrong, going by the American usage of the gerund I clearly am, but then "I want going flying" sounds ridiculous in any case. Maybe I am missing the difference, so as a second language speaker, I'd love to be corrected.

9
JoeDaDude 2 days ago 0 replies      
I'm a little surprised GATE [1], the General Architecture for Text Engineering tool is not mentioned. It is incredibly flexible, open source and has a very long track record as a research and prototyping tool.

[1] https://gate.ac.uk/

10
ktRolster 2 days ago 0 replies      
This book was really helpful for me when I was getting started with natural language processing: http://www.nltk.org/book/

It's practical, readable, and it's free.

11
p1esk 2 days ago 1 reply      
NLP right now looks like the computer vision 5 years ago: DL methods are starting to work really well, so a lot of "traditional" methods to process text might soon become obsolete.

The goal is to just feed gigabytes of raw text to a huge, complex neural network, and hope it will extract relevant features.

12
danso 2 days ago 0 replies      
Love the concept of this "How to" series. Seems like it'd be a good opportunity to spotlight the interesting HN threads on any given topic.

e.g. for NLP:

- https://news.ycombinator.com/item?id=11686029

- https://news.ycombinator.com/item?id=11690212

- https://news.ycombinator.com/item?id=1839611

13
rstuart 2 days ago 0 replies      
If you are interested in working in NLP, feel free to reach out to Kapiche. The website, Twitter or hello at kapiche dot com are all good options.
14
edblarney 2 days ago 0 replies      
I strongly recommend Stanford's youtube min-course by Dan Jurafsky & Chris Manning

https://www.youtube.com/watch?v=nfoudtpBV68

15
_spoonman 2 days ago 1 reply      
I think "Paths" is a terrific idea. There have been times where I've wanted to do a "first principles" look at a topic but don't want to go back through my HN upvotes. "Paths" allows for a curated and practical advice-driven jumping off point. Looking forward to more content. Best of luck with it!
16
elchief 2 days ago 0 replies      
http://web.stanford.edu/class/cs124/kwc-unix-for-poets.pdf is fun and easy. Text analysis with bash
17
beders 2 days ago 1 reply      
Fun problem: Write a parser for the English language.See it fail at tweets :)
18
fnl 2 days ago 1 reply      
> text summarization are examples of NLP in real-world products

Can someone point me to a satisfying demo of a professional text summarization software?

19
mankash666 2 days ago 0 replies      
Just noting that the article is better titled "why to get into ..."
20
earthly10x 2 days ago 1 reply      
One of the best places to start is reading this patent from Berkeley Lab/DOE which word2vec was based on https://www.google.com/patents/US7987191
27
The lost art of 3D rendering without shaders machinethink.net
313 points by mmphosis  2 days ago   72 comments top 24
1
antirez 2 days ago 4 replies      
The year I learned to write C code I was 19, second year of university and already willing to drop out, so I started spending time with C and 3D graphics. I was just fresh of the math exam so the 3D matrix transformations to do rotations was trivial to implement. I just wrote a function to draw triangles, used a simple z-sorting technique, and the basic shading calculating the cosine of the angle between the observer and the surface. With just these basic things I ended up with 3D "worlds" similar to the ones I saw in DOS games when I was a child. All the effort was maybe 500 or 1000 lines of code, but to build things from scratch, only starting from the ability to draw an RGB pixel, gave me a sense of accomplishment that later shaped everything else I did. I basically continued for the next 20 years to create things from scratch.
2
pcwalton 2 days ago 3 replies      
This is a good tutorial, but it's important to note that scanline rasterizers are not how GPUs (or even high-performance SIMD software implementations) work. Instead, they use barycentric coordinate sign tests for better parallelism and "free" interpolation.

A good explanation on this is Fabian Giesen's: https://fgiesen.wordpress.com/2013/02/06/the-barycentric-con...

3
ChuckMcM 2 days ago 1 reply      
Takes me back. A long time ago I wrote a simple rendering library for the 3DFx "Glide" library. It didn't do shaders but it would do mipmapped texture rendering which allowed you to have an image (texture) on your triangle. For a while I was stuck on the projection matrix and understanding screen clipping until my Dad gave me his copy of the Kodak Reference Handbook[1] third edition, copyright 1945. And they describe focal length, field of view, fstops, and lens effects very clearly.

[1] https://books.google.com/books?id=6DgYAQAAMAAJ&dq=Kodak%20Re...

4
alkonaut 2 days ago 0 replies      
Nitpick: this is software rendering. This is how we did before any kind of 3D api existed. Both GL/D3D/etc were without shaders to begin with. I still maintain a fixed pipeline (no explicit vertex or fragment shaders) 3D app with DirectX.

One can argue that the fixed pipeline of D3D is using a kind of implicit shader, but it's not the kind of shader we usually mean when we talk about vertex and fragment shaders today.

5
dahart 2 days ago 2 replies      
> Back in the day way before we had hardware accelerated 3D graphics cards, let alone programmable GPUs if you wanted to draw a 3D scene you had to do all that work yourself. In assembly. On a computer with a 7 MHz processor.

7 MHz? That's so fast and modern. Back in the day we were writing 3d fill routines on the 6502 going 1 MHz. With no floating point and no diagonal line support. And in bare machine language, going uphill both directions in the snow! ;)

6
rl3 2 days ago 1 reply      
>The framework then takes these shaders and your 3D data, performs some magic, ...

If we juxtapose that statement with the following in an unrelated introduction[0]:

'WebGL is often thought of as a 3D API. People think "I'll use WebGL and magic I'll get cool 3d". In reality WebGL is just a rasterization engine.'

I suppose when you're writing a software renderer from scratch without the luxury of any API or hardware acceleration, such things are indeed magic.

[0] http://webglfundamentals.org/webgl/lessons/webgl-fundamental...

7
air 2 days ago 0 replies      
Minor nitpick

"The green and blue colors, z-position, and normal vector are all interpolated in the same manner. (Texture coordinates behave slightly differently because there youd also need to take the perspective into account.)"

Colors (c), z, and texture coordinates (t) should all be interpolated differently because of perspective. You need to interpolate 1/z, c/z, t/z and for every pixel then do division eg. c/z / 1/z = c

8
fizixer 2 days ago 0 replies      
It may be a lost art for game developers. Far from it for CG grad students and researchers. Quite the contrary, it's actually part of the rite of passage, heck an undergrad level prerequisite to know these things like the back of your hand, plus a whole lot more, to do graduate level CG work.

Even if you're not a researcher, but wish to write your own path tracing code for example, you would end up learning this.

So no, not a lost art at all in my opinion.

9
vvanders 2 days ago 2 replies      
Kinda of a shame they omitted matrices. They're one of the foundational bit of any 3D api and one of the few things that translates well from fixed function/sw raster to modern pipelines.

Still great to know the fundamentals, texture formats, tiling and other things are also really useful pieces to understand when working with 3D pipelines.

10
ykl 2 days ago 2 replies      
This is a great article!

I strongly believe that an understanding of how old school 3D rendering worked is an excellent thing for modern graphics programmers to have, to appreciate and understand where all of our fancy modern graphics APIs and whatnot come from. Back when I helped teach a GPU programming course, one of the assignments I gave was a full-blown software rasterizer implemented entirely in CUDA. Not so much "program in OpenGL" as "program an OpenGL". :)

11
linuxhansl 2 days ago 0 replies      
Ahh. The days. I remember before I had learned about linear algebra, I saw somebody rendering molecules as 3D wire frames. I had an Amiga back then with it's "Blitter" (could draw lines in hardware, a long as you tell it which if eight octants the line's angle falls into).

Then, being the geek I was, I sat down every day until I had figured out perspective transformation and rotation (later I found I had just done matrix multiplication). Of course I never thought of homogeneous coordinates, so translation was an extra step to be done for each point.

Even worked out "real" red-green 3D. Oh the days when I had time for this stuff. Fond memories.

12
bhouston 2 days ago 1 reply      
I used to write my own triangle fill algorithms with their own shaders back in the 1990s. Fun times: https://github.com/bhouston/3DMaskDemo1997

Here is the optimized triangle fill code with embedded asm pixel shaders: https://github.com/bhouston/3DMaskDemo1997/blob/master/src/N...

13
leeoniya 2 days ago 0 replies      
Related: "JavaScript library for simple 3D graphics and visualisation on a HTML5 canvas 2D renderer. It does not use WebGL. Works on all HTML5 browsers, including desktop, iOS and Android."

http://www.kevs3d.co.uk/dev/phoria/

https://github.com/kevinroast/phoria.js

14
paulddraper 1 day ago 0 replies      
Great, great stuff. Terrific article.

---

It does seem to perpetuate -- or at least not make clear -- a misconception.

> 3D rendering without shaders

> We wont use any 3D APIs at all

Those are two independent statements.

Metal, OpenGL, WebGL, and Vulcan are not a 3D APIs. They are (2D) rasterization APIs using shaders. Any 3D-ness of the math is external to them. In contrast, OGRE, Java 3D, and three.js are 3D rendering APIs.

Two independent choices yield four types of ways to do 3D rendering. E.g., in browser they could be

 | 3D API | no 3D API | ---------------|------------------------|------------------------| GPU shaders | three.js, using WebGL | WebGL | ---------------|------------------------|------------------------| no GPU shaders | three.js, using canvas | canvas | 
This article fits in bottom-right corner.

I take notice when I hear the oft-repeated fact that OpenGL/WebGL are 3D rendering APIs. At www.lucidchart.com, in 2015 we chose to use WebGL when available to improve rendering performance for (2D) diagramming. Were WebGL made for 3D stuff, it'd be a weird choice, but WebGL is for high-performance rasterization of all kinds.

http://webglfundamentals.org/webgl/lessons/webgl-2d-vs-3d-li...

15
buzzier 2 days ago 0 replies      
16
jlarocco 2 days ago 0 replies      
A while back I created a small project for drawing 3D wireframe graphics using the Common Lisp LTK interface to Tk.

It's slow (uses inefficient matrix algorithms, uses Tk, etc.) but it's "fast enough" for some simple 3D scenes. Not very practical for real-life use, but it was fun.

https://github.com/jl2/ltk3d

FWIW, it's not doing hidden line removal, IIRC I was careful to pick a viewing location that made it look good.

17
a_c 1 day ago 0 replies      
This is the kind of article that I enjoy reading a lot. Most tools available today mask away fundamental concepts, and many aspiring young engineers learn to use "tools". While the ability to use various tools is of paramount importance, the most valuable skills an engineer can possible possess, in my opinion, is the ability to create new tools/concepts/whatever from 1st-ish principle
18
c0ffe 2 days ago 0 replies      
Great article!Reading the title, I thought it was about the "tricks" that games used when the best thing available was the fixed pipeline.

I still remember how amazed I was when learned the good balance between performance cost and the resulting image when using textures for static lighting (lightmaps).

19
Waterluvian 1 day ago 0 replies      
Any suggestions on a good primer for what shaders are and how they work? For years I've always thought "shaders" are just effects you can layer onto a rendered scene. Say, to get an 80s effect, or bloom, or a cel shading effect, etc. I never really thought of it as a way to actually do the base scene rendering.
20
hellofunk 2 days ago 0 replies      
I have a question about the rasterization step. When creating the scanlines, would this be a possible entry point for anti-aliasing, by giving the lines a subtle gradient that goes to near 0 alpha at the right and left edges? (and maybe also the top and bottom edges for the lines at the top and bottom of the stack). There are many ways to do anti-aliasing and this seems like one possibility to me.
21
kgabis 2 days ago 0 replies      
scratchpixel.com has excellent tutorials on computer graphics and how to do rasterization [1].

[1] https://www.scratchapixel.com/lessons/3d-basic-rendering/ras...

22
foota 2 days ago 0 replies      
I've thought before that it would be cool to implement a rasterizer in something like OpenCl.
23
BatFastard 2 days ago 1 reply      
Thank god it is almost lost, interesting, but highly specialized, and ORDERS of magnitudes slower.
24
quickben 2 days ago 2 replies      
All these 'for' loops and sequential trigonometric calls...

See, I wouldn't say that art was lost. It was obsoleted to dust by a more modern and scalable approach.

28
Galaxy Note7: What We Discovered [infographic] samsung.com
381 points by richardboegli  11 hours ago   114 comments top 19
1
owenversteeg 8 hours ago 12 replies      
Since I have a bit of experience in the world of batteries I thought I'd post.

First of all, deflected electrodes, insufficient insulation tape, and high welding burrs are absolutely fine explanations for what happened here. As someone who works with batteries, that would tell me everything I wanted to know. If I was running a battery fab, I'd be able to check for these problems in 20 minutes.

Further in the favor of Samsung is that battery fabs are some of the most opaque places in the world. They never release any information of any value whatsoever to the outside, and there are only a handful in the world. For Samsung to release something like this so candidly is absolutely unprecedented in the world of batteries; I was shocked when I saw the post.

Say what you will about Samsung in general, or about their treatment of the whole process, but posting this is a first among any of the major battery manufacturers and shocked me with its level of openness and detail.

-------------------

If anyone doubts this, try getting information on the closest battery fab to you. You can try to find what batteries they produce, but chances are that's not public. You can try to find their name, or if they still exist, but that's probably also not public. You might eventually find an address, and you might go there, but when you arrive you'll be forcibly removed from the area by armed security and/or police. To give an example, there are a couple debates in the industry about what company acquired another company. It's years after the supposed acquisition, and nobody knows anything.

The battery industry is a whole secretive world of its own, and this is because batteries are simultaneously a low-profit commodity and a gateway to unlimited riches. If Apple is successful with its dream technology, it might capture 50% of the world smartphone market instead of the 14% it has now. If a battery company is successful, it will become successful beyond your wildest dreams. Lots of things are just on the edge of being possible with today's battery tech. The first one to 'win' gets to go from selling a couple billion of batteries (to small fish, like EV and power tool makers) to selling trillions of dollars to everyone that will buy, including every electric utility in the world that will be ecstatic to have the perfect demand-smoothing device.

2
JohnJamesRambo 9 hours ago 2 replies      
I'm pretty proud of them for making an easy to understand infographic about the problem instead of a bunch of text jargon.
3
richardboegli 11 hours ago 0 replies      
Samsung Announces New and Enhanced Quality Assurance Measures to Improve Product Safetyhttps://news.samsung.com/global/samsung-announces-new-and-en...

Also a YouTube video with findings and new Quality Assurance Measureshttps://www.youtube.com/watch?v=OeKdcIOAEL8

4
asafira 10 hours ago 2 replies      
I'm glad my question was finally answered! During the paranoia in September/October, I remember reading everything and wondering why nothing seemed to explain why the issue happened in the first place. Note7's aren't exactly the only tech with batteries...

Otherwise, sorry for being negative, but this doesn't seem like a very complicated reason for the batteries to have been defective, right? Why didn't they just release this a couple of weeks after the incidents started happening? (Did they want to confirm with a large number of the recalled ones they got back you think?)

6
ChuckMcM 10 hours ago 1 reply      
I realize they had to root cause this problem but it was interesting to see and I wonder if they fired the Battery manufacturer.

The other thing I find interesting is that given the root cause they could essentially do 100% battery screening and have a lot of stock of G7's. They would be better than a cheap android phone in a magazine advertisement[1] :-). Also the G7 has a gorgeous screen, I wonder if those are being recycled into new gear or if there is some booth selling off screens somewhere.

[1] http://mashable.com/2012/10/02/ew-has-smartphone-inside/#Nq6...

7
iscrewyou 9 hours ago 1 reply      
Missing insulation tape. That one made me say, "of course they caught on fire."
8
richardboegli 11 hours ago 2 replies      
TL;DR Both batteries had a design / fabrication error which caused issues
9
heisenbit 7 hours ago 0 replies      
The MSN reports this as battery defects. Both "defects" were however made more critical by the tight containment within the phone. It is not simply a component but a system engineering issue - from objective, design, tooling, manufacturing to QC. Like with the Challenger disaster there is a cultural problem.

The press release calls into question that Samsung fully owns up to it.

10
c3534l 8 hours ago 7 replies      
I think the Galaxy Note 7 is going to become a classic business case study. They did everything right following the crisis and in the follow-up investigation. I wonder how will it will actually pan out for them, though.
12
pasta 5 hours ago 2 replies      
Maybe all those Note7s are already recycled, but couldn't they just release it again with good batteries?

Or would replacing the battery and re-distributing the phone cost more than just release a new phone?

13
sschueller 8 hours ago 3 replies      
Now if the battery would have been removable this would probably have worked out a lot different.

Why do we not have standardized removable batteries for mobile phones?

14
chinhodado 10 hours ago 5 replies      
Props to Samsung. They reacted fast to the incidents, quickly announced recalls and worked hard to convince people to return their devices, launched in depth investigation, identified the problem and made changes to their processes.

If the result of the investigation is correct then they have different flaws in both the original and new batteries, which is quite unfortunate. The fiasco costed them something like $17B so they can't afford to have it happen again.

15
yeukhon 9 hours ago 0 replies      
16
SaaSAddict 7 hours ago 0 replies      
At least they admit to theit mistakes...
17
partycoder 10 hours ago 0 replies      
It will be hard for them to recover from this.They will have to aggressively lower the prices to attract customers.
18
hetfeld 9 hours ago 1 reply      
What about independent researches and tests that all state obvious facts - battery is not that bad, but the overall internal construction was flawed the battery was positioned close to high-heating elements which caused the overheating the battery and blasts. Pathetic false statements from Samsung, but nothing new. Who can trust this crooked company after such bs explanations...
19
hisham_hm 3 hours ago 0 replies      
Is it me or the zoomed-out images for both sides in the Main Cause section of Battery B are the same? They clearly shouldn't be when they expand to such different images, and the difference is precisely what the infographic is trying to explain.

It seems to me it's a classic copy-and-paste error caused by haste/laziness. How ironic that the infographic made to explain the buggy batteries is buggy as well.

29
US Announces Withdrawal from TPP nikkei.com
270 points by jaboutboul  1 day ago   224 comments top 20
1
niftich 1 day ago 6 replies      
TPP was a communications and messaging failure on parts of governments. Other than being a sprawling agreement touching on multiple unrelated topics largely developed in secret, the US government in particular did little to convincingly persuade the populace about TPP's advantages. The anti-globalization folks predictably seized on those aspects while a different debate about the expansion of copyrights (i.e. the harmonization of copyright protections with those of the US) was raging in the tech sphere.

A gulf began to widen between the administration and those opposing TPP, and quality independent analysis was much more difficult to come by than fearmongering. A few corporations spoke out in favor of TPP [1], but given their vested interest, they made a poor case of swaying average people. Meanwhile, even mainstream news coverage of TPP tended negative. Hillary Clinton notably had modify her messaging on TPP [2] to be seen as a viable candidate, despite endorsing it before.

[1] https://news.ycombinator.com/item?id=11893512#11894446 [2] http://www.snopes.com/hillary-clinton-called-trans-pacific-p...

2
Maarten88 1 day ago 3 replies      
I so much hope that one of these countries will quickly change local laws to reduce copyright length to old levels (like 25 years after the life of the author) and start legally distributing (or even producing) movies featuring Mickey Mouse and other Disney "properties".
3
lancewiggs 1 day ago 0 replies      
TPP was a trade agreement ruined by a long series of provisions sharply biased towards the USA and certain US industries. So as a New Zealander I'm really happy to see the back of it. Meanwhile we already have an FTA with China (and many other countries) and our trade both ways is booming.

Perhaps we can start again with a much cleaner, people friendly TPP that excludes the US and includes China.

4
ggame 1 day ago 2 replies      
Not a fan of Trump but this is a good thing. See this exert from John Oliver on the effect of such trade deals; https://youtu.be/6UsHHOCH4q8

And that's the existing trade deals, TPP would have made it much worse. There is a reason why democratically elected governments need to protect their sovereignty. The scope of the legal provisions along with the requirement to use easily corruptible mediation is superfluous to free trade.

In addition, I don't understand those that think boxing out Russia and China from the rest of Europe and Asia is a good thing.

5
tormeh 1 day ago 0 replies      
This is essentially the resignation of the US in the Pacific. The TPP was intended to bind the smaller countries in the Pacific closer to the US and away from China. Generally, the election of Trump is the resignation of the US in the battle to remain the global hegemon. We live in interesting times. I just hope it won't become too interesting.
6
Bluestrike2 1 day ago 4 replies      
It will take decades to undo the damage from this latest bout of isolationism and protectionism. There was a wealth of FUD surrounding TPP, obscuring the benefits and over-inflating the effects of the more controversial issues like intellectual property rules. A number of TPP criticisms would lead one to believe that intellectual property rules were the main element of the agreement.

Instead of pushing to deal with those criticisms, the entire deal gets scrapped. And along with it, we now have an isolationist push against NAFTA and international trade in general. It'll be interesting to see the damage that's wrought in a vain attempt to somehow stack the deck in favor of "America First."

Sadly, it'll take much longer to fix than it did to screw up in the first place.

7
acconrad 1 day ago 2 replies      
I'm very confused. When the TPP first leaked as a series of secret talks, everyone on HN was seemingly up in arms about the trade deal. Then it went through, and HN shifted to being very pro TPP. Now it's withdrawn, and now the HN mentality is back to voting for comments that are against the TPP. What is the prevailing opinion here?
8
orangecat 1 day ago 0 replies      
The intellectual property provisions were terrible, so this may be the right action for the wrong reasons.
9
Animats 1 day ago 1 reply      
TPP was dead anyway. The next question is whether the new administration will withdraw from NAFTA.
10
breatheoften 1 day ago 0 replies      
I'm curious to see if the markets react to this. In the grand scheme of things, one trade deal shouldn't move the markets that much -- but the reasoning behind this withdrawal from the TPP is nothing at all approaching any of the real reasons one might be opposed to it -- the cause for this withdrawal is pure unfiltered sabre-rattling protectionism. "Let the trade wars commence and America intends to 'win'." The level of stupid behind that notion is nigh on incomprehensible -- and yet -- here we are.
11
candiodari 1 day ago 0 replies      
Maybe it's just me, but wasn't everyone in Silicon Valley that wasn't senior management of a huge company vehemently anti-TPP ?

Why the change ?

12
paulus_magnus2 22 hours ago 0 replies      
TPP is an attempt by multinational giants to solidify their position above governments / democracy and secure their monopolist rents.
13
leke 1 day ago 1 reply      
As a European, what does this mean for the TTIP and CETA?
14
philliphaydon 22 hours ago 0 replies      
Ah I really wish NZ would withdrawal too. I feel embarrassed to be a Kiwi when they signed it.
15
shmerl 1 day ago 0 replies      
Good! Let's see what new monsters the DRM lobby will pull out of their twisted minds.
16
chris_wot 1 day ago 2 replies      
What is the best summary of the TPP? It was done in such secrecy that it occurs to me that this may be the only good thing that comes from a Trump presidency.
17
sandworm101 1 day ago 3 replies      
Many here love to hate on any and all international trade deals. The problem with is that withdrawing from such negotiations does not stop them. Canada, china, europe, japan, the uk and everyone else will still be meeting and inking deals. Not every deal is great for everyone, but those who sit on the sidelines never win at anything.
18
meesterdude 1 day ago 4 replies      
Probably the only silver lining to be found in the new administration.
19
williamle8300 1 day ago 2 replies      
20
notpc 1 day ago 2 replies      
TPP was a threat to the sovereignty of the United States. It would have encoded in a multilateral trade agreement, representing huge amounts of economic activity, massive regulatory requirements, enforcement courts, and processes for further multilateral regulation. It would have made the cost of changing those regulations unbearable for any future administration.

We almost had the regulatory state imposed at an international level. Good riddance.

30
Microsoft Azure in Plain English expeditedssl.com
277 points by handpickednames  3 days ago   46 comments top 15
1
Swinx43 3 days ago 2 replies      
This and the AWS in Plain English are both awesome. Is there an equivalent for Google Cloud Platform?
2
chucknelson 3 days ago 0 replies      
This is cool - one item I think is wrong/misunderstood is Big Data > Data Lake Store.

It has nothing to do with ETL, it's basically just "HDFS in the cloud" [1] and a successor to using blob storage/regular old storage accounts for distributed/Hadoop-ish workloads.

[1] https://azure.microsoft.com/en-us/services/data-lake-store/

3
jsingleton 3 days ago 0 replies      
Re-post from the AWS thread (https://news.ycombinator.com/item?id=13442022).

That's a good high-level list, although the comparisons don't always match up. For example, I'd say Traffic Manager is more like Route 53 than ELB (which only works within a region).

If you're after something a bit more in-depth (but covering less services) then I wrote a three part series last year. It may be a little out-of-date, but most of it still applies. Azure now supports MySQL, for example.

1: https://unop.uk/on-aws-vs-azure-vendor-lock-in-and-pricing-c...

2: https://unop.uk/on-aws-vs-azure-vendor-lock-in-and-pricing-c...

3: https://unop.uk/on-aws-vs-azure-vendor-lock-in-and-pricing-c...

Edit: Should that "puts da" be on that page?

4
yread 3 days ago 0 replies      
It seems that Azure naming is a lot better than Amazon, perhaps so much so that this guide is not even needed
5
sumitgt 2 days ago 1 reply      
I don't think Service Fabric is like AWS Lambda. Azure Functions is AWS Lambda.
6
davidmichael 3 days ago 0 replies      
Microsoft themselves publish a comparison document of services with AWS: https://docs.microsoft.com/en-us/azure/guidance/guidance-azu...
7
expertentipp 3 days ago 5 replies      
If it only was easy to starting playing around with Azure. Only to activate the account they require proper, bank issued, credit or debit card. They explicitly refuse to accept prepaid cards even though they are VISA/MasterCard (BTW the same problem with Google Compute)... or am I doing something wrong?
8
klausjensen 3 days ago 0 replies      
This. This is absolutely brilliant. I have worked with Azure for years, and mostly love it - but I learned about a few services, that I never knew what were.

Great work, ExpeditedSSL

9
youdontknowtho 3 days ago 2 replies      
"Cloud services" should not be named "Azure IaaS" because Azure IaaS is named Azure IaaS.
10
viach 3 days ago 0 replies      
I love how the slogan on the main page written "Bam! ..." It takes attention and you actually read further. Nice small trick.
11
andysinclair 3 days ago 1 reply      
Very good overview.One point that I disagree with, Cloud Services:"Run stuff but worry a fair amount about configuration and patching." We run a bunch of cloud services and MS are responsible for patching, I would describe it more as PaaS that IaaS.

We built this in our product to help visualise how the services fit together:https://my.sharpcloud.com/html/#/story/f7522de0-98ff-4d02-8e...

12
m0d0nne11 3 days ago 0 replies      
Very useful, as is the one for AWS. Yay! though if these things are being touted as "plain English" they should probably steer rigorously clear of smart-ass insider references (no matter how full of cheer the writer may be feeling at the moment) because that's probably how these titles and terms came to be so opaque in the first place. But, again: yay!
13
k__ 3 days ago 2 replies      
so service fabric is API Gateway and Lambda in one product?

Sounds good and removes a bunch of complexity, I guess.

14
kevingibbon 2 days ago 0 replies      
Azure in REAL plain English: Microsoft AWS
15
itaysk 2 days ago 3 replies      
There are so many fundamental mistakes here that I don't even know where to start.. Nice idea though.(I am a cloud solution architect with Microsoft)
       cached 23 January 2017 16:11:01 GMT