hacker news with inline top comments    .. more ..    9 Aug 2017 Best
home   ask   best   2 weeks ago   
Google Fires Employee Behind Controversial Diversity Memo bloomberg.com
1674 points by QUFB  1 day ago   2319 comments top 2
sctb 1 day ago 0 replies      
We've closed this thread to new accounts because of trolling. If you're going to comment here, please take extra care to make it civil and substantive.

If you have a new account and want to comment (thoughtfully, as opposed to flaming) please email us at hn@ycombinator.com and we'll whitelist you.

tunesmith 1 day ago  replies      
People can't seem to summarize his argument without getting much of it grossly wrong, because his manifesto was a haphazard collection of good points, bad points, good arguments, lousy arguments, misrepresentations of others' views, and unstated implications. A perfect recipe for people to argue past each other about it.

It's worth remembering that one of his conclusions was to end or replace gender-based diversity programs at Google. Given that, it's easy to understand why people would be upset. If gender-based diversity programs are responsible for qualified women getting jobs that they otherwise wouldn't have gotten due to bias, then the lack of that program means those women wouldn't have gotten those jobs.

Disney acquires own streaming facilities, will pull Netflix content thewaltdisneycompany.com
645 points by anigbrowl  18 hours ago   657 comments top 2
geff82 16 hours ago 18 replies      
Maybe when I am 60, 70 or 80 the film industry will get their shi* together and finally agree on a solution that has long been found in the music business.

For a truly complete platform, I would FOR SURE pay more than the 10$ a month for Netflix. 20, maybe 30! But then I want it ALL. All films they have in storage.

I mean, it is 2017 and there are a lot of films I can't find on Netflix, Amazon Prime or, when I am in spending mood, on Apple TV. Why? I mean how silly would you want to be as studios? There is no big DVD business anymore, BlueRay never totally took off. People have a net connection and multiple streaming devices at home, thats it. Thats the big asset they could build on! Instead they let their libraries die the death of the unseen film.

Still, many keep shuffeling around harddrives with terabytes of pirated films. And why shouldn't they, as long as there is no substantial offer?

So I decided for me (and the cloud guy I am), that with my 3 services I have, I am ok. If a film is not there, I don't care. I surely won't order a DVD of some old film somewhere and I surely will not subscribe to another service. If Disneys pulls their films from Netflix: thanks Netflix for their growing self produced content that often has a quality not seen before.

Anatidae 17 hours ago 46 replies      
If every studio thinks I'm going to pay them $10+ a month to stream their content, they are going to be very mistaken.

I can't imagine that a lot of people want to spend the collective hundreds of dollars to sign up for all the streaming services. It's almost asking to drive people to torrents.

Now, if Disney does something like $30/year or something really affordable - sure. I might do that on a whim. I guess it's all about volume vs. price.

Netflix, however, I'll keep paying for gladly because of the library size. For the streaming price, it is well worth the value.

The Internet Archive has digitized 25,000 78rpm Gramophone records archive.org
633 points by yurisagalov  21 hours ago   90 comments top 33
indescions_2017 18 hours ago 5 replies      
House of the Rising Sun. As interpreted by Josh White, advisor and confidant to F.D.R. Priceless ;)


I find myself on Internet Archive a lot during these dog days of summer. Delving into classic texts like Edgar Rice Burroughs A Princess of Mars or Aldous Huxley's The Perennial Philosophy. Discovering a forgotten H. P. Lovecraft story in the Weird Tales archive. Mining old time radio shows like Suspense for story inspiration. And using the Internet Arcade for screen grabs that can be used in retro-style game texture art. It makes me think I should do a better job of preserving my own output. You never know what future generations may find useful!

komali2 20 hours ago 2 replies      
Fun to read some of these reviews, apparentl from random internet folk, like on jungle boogie - https://archive.org/details/78_jungle-boogie_the-bobby-true-...

Some guy just wanted to tell everyone some neat little facts about this thing he apparently knows a lot about. I find it fascinated how much people care to know about things like this.

EDIT: whoever this "arc-alison" character is, they're prolific - I'm finding their informational reviews all over this archive.

guyfawkes303 20 hours ago 10 replies      
The records I clicked on have this notice

Digitized from a shellac record, at 78 revolutions per minute. Four stylii were used to transfer this record. They are 3.8mm truncated conical, 2.3mm truncated conical, 2.8mm truncated conical, 3.3mm truncated conical. These were recorded flat and then also equalized with NAB.

The preferred version suggested by an audio engineer at George Blood, L.P. is the equalized version recorded with the 2.3mm truncated conical stylus, and has been copied to have the more friendly filename.

I'm trying to guess but can't imagine what the reasoning for this is. I've tried A/B/C/D testing a few tracks on some crappy speakers and can't discern any difference.

While it's certainly admirable to try and digitize it as thoroughly as possible, I just can't see how a difference of 0.5mm in the stylus width is worth increasing your work load 4x times over (having to record each record 4 times rather than just once).

mortalkastor 6 hours ago 1 reply      
The "Bibliothque nationale de France" (national library of France) did the same kind of thing with hundreds of thousands vinyl records from their archive, including international ones published in France: http://www.bnfcollectionsonore.fr/
ShirsenduK 20 hours ago 1 reply      
beaugunderson 9 hours ago 0 replies      
They had me make a Twitter bot that's tweeting out all of the 78s (with preview audio) as well:


0xcb0 5 hours ago 0 replies      
This is just great! Listening to these songs instantly sets me back to a relaxed inner state. Together with that sizzling noise of the gramophone record in the background, so calm and chilled.

I currently listen to "A Duke Ellington Panorama", just nice!

Thanks for that and keep up the awesome work!

jonah-archive 20 hours ago 0 replies      
Lots more info here for the curious: http://great78.archive.org

You can see a picture of one of the four-armed turntables here: http://great78.archive.org/preservation/

qrbLPHiKpiux 18 hours ago 2 replies      
sushisource 19 hours ago 0 replies      
More sample fodder for the EDM artists and rappers. Always a good thing.
e12e 20 hours ago 0 replies      
Certainly a bit of everything on there... :)


jrowley 19 hours ago 3 replies      
I have no experience with this stuff, but I wonder if they could use a laser record player to capture the record, and then replay it with different simulated stylus sizes. Not exactly kosher probably, but could be an interesting experiment. Plus scanning time could be greatly reduced I imagine.
menacingly 14 hours ago 1 reply      
Very cool that they offer 24bit flac downloads. I'm sure this sentiment is shared here, but I am always impressed by the efforts of this organization
daveheq 8 hours ago 0 replies      
Imagine after World War 3, the aliens sift through the remnants of humanity, find this archive of digitized 78rpm records, and turn into mustachioed corduroy-wearing hipsters.
Nav_Panel 17 hours ago 3 replies      
Some very very good stuff in here. I've gotten pretty into 20s thru 50s music over the past couple of years. I usually buy compilations on LP, though, so it's a treat to find these straight off the 78s. A big portion of the stuff never even makes it to digital.

Just at a glance, I'm seeing The Light Crust Doughboys[1], basically a string band supergroup. Multiple members would go on to found famous western swing bands (Bob Wills, Milton Brown). Very proto-rock-and-roll -- listen to that electric guitar -- Elvis would cover some Western Swing numbers[2] in his early days[3].

Also seeing some older stuff, including a few recordings by the (arguable) best banjo player of all time, Vess L. Ossman[4] (from 1907). Pretty cool to listen to these march numbers and then hear them evolve into jazz/ragtime only a couple years later[5] (this is a recording by Fred Van Eps, the second best banjo player of all time, from 1914).

EDITS: seeing some other personal favorites:

Hank Penny, a favorite western swing singer of mine[6]. He usually does it hot/upbeat/fun.

Blind Blake, a guitarist who could play the fretboard like a ragtime piano[7]!

Oh, and here's the WWII-era Bob Wills I was waiting for[8]. Got that classic Leon McAuliffe pedal steel playing. No Tommy Duncan vocals, unfortunately.

Neat! An old solo Art Tatum[9]! Widely considered the best pianist of all time... And another, a whole album[10]!

Really classic early electric guitar playing on a jump blues number by T-Bone Walker[11]. I actually believe he's one of the first to use the electric guitar in blues.

Great steel guitar playing on this Gene Autry cowboy number[12].

Looks like there's a lot of Django for all you gypsy jazz fans[13]. Never heard this take on Avalon before, I dig it.

Lot more to dig through and lot of obscure stuff I'd like to give a shot, but I'm out of time for now...

1: https://archive.org/details/78_pretty-little-dear_light-crus...

2: https://www.youtube.com/watch?v=4wGCTFWhoqQ

3: https://www.youtube.com/watch?v=8bSVEA0ZAVw

4: https://archive.org/details/78_policy-king-march_vess-l.-oss...

5: https://archive.org/details/78_notoriety-rag_van-eps-trio-ka...

6: https://archive.org/details/78_get-yourself-a-red-head_hank-...

7: https://archive.org/details/78_tampa-bound_blind-blake_gbia0...

8: https://archive.org/details/78_texas-playboy-rag_wills-bob-w...

9: https://archive.org/details/78_deep-purple_art-tatum-mitchel...

10: https://archive.org/details/78_art-tatum_art-tatum-james-swi...

11: https://archive.org/details/78_t-bone-blues_les-hite-and-his...

12: https://archive.org/details/78_silver-haired-daddy-of-mine_g...

13: https://archive.org/details/78_the-quintet-of-the-hot-club-o...

Finnucane 19 hours ago 1 reply      
Too bad it doesn't seem to be easily searched by label--from a historical perspective, it would be cool to be able to search for say, Paramount or Gennet or Okeh.
pier25 20 hours ago 1 reply      
Any sound restoration software would greatly improve these recordings.

For example this one from 1902: https://archive.org/details/78_medley-of-emmetts-yodles_yodl...

I'm sure Izotope would give the RX license for free in exchange for a blog post (or any other audio software company).

matt_wulfeck 13 hours ago 1 reply      
How would one go about removing the pops and clicks from recorded audio programmatically?

I really like some of the audio here but it needs some post processing. The only thing I can find to do it is audacity and it doesn't look very friendly to scripting.

tamersalama 19 hours ago 0 replies      
Looks like there are some recordings by Sergei Rachmaninoff himself [1]

[1] https://archive.org/details/georgeblood?sort=&and[]=subject%...

barking 5 hours ago 0 replies      
What did they smell of?It was really unusual.Tesco, briefly, had an own brand hand soap liquid in the 1990s with exactly the same smell.
kyledrake 19 hours ago 0 replies      
vinchuco 10 hours ago 0 replies      
Is there a way to stream these indefinitely on shuffle without having to pick each one manually?
sdsk8 16 hours ago 0 replies      
I don't know about everybody here, but i am listening to so much new things to me on this archive that i'll definitely donate to the archive team today, congratulations for this fantastic job!
orbitingpluto 10 hours ago 0 replies      
This is the crowning gem from the Internet Archive (from the 78 RPMs and Cylinder Recordings collection).

Cab Calloway, The Man from Harlem


fortyfivan 14 hours ago 0 replies      
Wow, this is great! I've been a serious record collector for 20 years, but never got into 78s.

My eventual life goal is to do something similar with my Brazilian record collection... have the skeleton of such catalog at: https://www.novedos.com/collection.

hmhrex 20 hours ago 2 replies      
Just curious, what's the copyright on this kind of material?
amelius 5 hours ago 0 replies      
Is it possible to search based on genre or geographic origin?
S_A_P 14 hours ago 0 replies      
So the obvious win here besides archiving art is that this is out of copyright sample fodder*

*IANAL and this may not be the case for all the material but I'm sure that there is mountains of inspiration to be mined.

kmeade 17 hours ago 1 reply      
I'm curious about something and I can't find the answer on the web site -- Why were these recordings played and digitized in stereo when the records were mono?
neelkadia 9 hours ago 0 replies      
New stuff for Machine Learning. GAN. Magenta.
cJ0th 18 hours ago 0 replies      
thanks for the heads up. this is just amazing!
anjc 9 hours ago 0 replies      
I am become The Avalanches, mixer of old songs

Very cool

Internet Draft: Let 'localhost' be localhost ietf.org
587 points by beliu  1 day ago   162 comments top 23
user5994461 1 day ago 8 replies      

 First, the lack of confidence that "localhost" actually resolves to the loopback interface encourages application developers to hard-code IP addresses like "" in order to obtain certainty regarding routing. This causes problems in the transition from IPv4 to IPv6 (see problem 8 in [draft-ietf-sunset4-gapanalysis]).
That does remind me of the times I was dealing with weird connection issues in some critical services.

It turned to be related to the use of "localhost" in the configuration. It resolves to ipv6 on some systems and that breaks everything because the target app is only listening to the ipv4 address.

Went as far as removing all references to localhost and added lint errors in the configuration system so that noone could ever be able to give localhost as a setting in anything.

nhance 1 day ago 9 replies      
If this doesn't happen or takes too long, there's always lacolhost.com and *.lacolhost.com. I own this domain, have registered it out until 2026 and vow that the domain and all subdomains will always redirect to localhost.

It's easy to type and easy to remember and should always do a good job of expressing intent of usage.

DonHopkins 1 day ago 1 reply      
There was the time that Keith Henson tried to explain the local loopback address to Scientology lawyers during a deposition...


Henson: (patiently) It's at This is a loop backaddress. This is a troll.

Lieberman: what's a troll?

Henson: it comes from the fishing where you troll a bait along in the water and a fish will jump and bite the thing,and the idea of it is that the internet is a very humorous placeand it's especially good to troll people who don't have any senseof humor at all, and this is a troll because an ftp site of't go anywhere. It loops right back around into your ownmachine.


jonathonf 1 day ago 4 replies      
I've had web browsers perform a web search for 'localhost', or even just redirect me to localhost.com.


JdeBP 3 hours ago 0 replies      
On the one hand, this isn't exactly a new idea and in the real world has been happening for years now.

* dnscache from djbdns has handled "localhost." queries internally all along, since 1999. It maps "localhost." to and bgack again. Various people, including me, have since added code to do the same thing with the mappings between "localhost." and ::1. (http://jdebp.eu./Softwares/djbwares/guide/dnscache.html) I implemented implicit localhost support in my proxy DNS servers for OS/2, as well.

* It is conventional good practice to have a db.127.0.0 and a master.localhost "zone" file on BIND that do this. This is in Chapter 4 of the book by Albitz and Liu, for example.

* Unbound has built-in "local zone" defaults mapping between "localhost." and both and ::1.

On the other hand, this proposal explicitly rules out all of the aforementioned existing practice, by demanding that both proxy and content DNS servers instead return "no such domain" answers for the domain name "localhost.". That seems like a fairly pointless deviation from what is fast approaching two decades of existing practice, for which no rationale is given and none is apparent.

tcbawo 1 day ago 1 reply      
At work someone once spent hours trying to resolve a network issue. Turns out he didn't have a localhost entry in his /etc/hosts and some sadistic person had created a VM named 'localhost' that registered a DNS entry via DHCP.
feelin_googley 1 day ago 1 reply      
At least on the OS I use, which is more IPv6 ready than most, /etc/hosts solves this "uncertainty" problem.

I have found that failing to include a localhost entry in the HOSTS file can lead to some strange behavior.

If there are "computers" out there that have no /etc/hosts or deny the computer's owner access to it, then maybe it might be time for an Internet Draft from Joe User.

There should always be a mechanism for the user to override the internet DNS. And applications should continue to respect it.

zanchey 1 day ago 0 replies      
We have two entries in our DNS which point to - localhost and elvis.

This enables the following on Solaris and similar systems:

 $ ping elvis elvis is alive

bryanrasmussen 1 day ago 0 replies      
this reminds me of a class I went to at a major company in 1999, we had problems following the setup instructions which included going to localhost/db-admin-path, after some sleuthing it turned out somebody 'in corporate' on the network we were using had named their computer localhost.
dmacedo 1 day ago 3 replies      
Also very important to point out; this same standardisation is missing on the TLD level.

Both for safeguarding internal use, and making a global TLD reserved on the global DNS zones.You'll find organisations using in production .local .dev (Taken by Google on 2014-11-20, followed by .app in 2015) *.zone (Taken by a LLC on 2014-01-09 ) as internal domains, with potential conflicts with the Internet's DNS resolution.

More importantly .dev [1] and .zone [2] are now valid TLDs, so watch out people!

[1] https://www.iana.org/domains/root/db/dev.html

[2] https://www.iana.org/domains/root/db/zone.html

sgtpepper43 1 day ago 0 replies      
Just add a line your hosts file mapping lolcathost to and you never have to worry about it again.

No that's not a typo

chr1 1 day ago 2 replies      
Does this mean that an entry in /etc/hosts assigning ip to localhost will be ignored?
ccheever 1 day ago 0 replies      
One time I was debugging a problem for a user of our desktop software (I work on https://expo.io) by sharing his screen and taking over his computer. And it turned out the reason the user was having problems was that in his /etc/hosts file, he had an entry pointing localhost at the IP address of some other computer on his network. Crazy. I have no idea how anything worked on his machine.

Took a while to track that was down. Was both bewildering and sort of satisfying to figure it out in the end.

eduren 1 day ago 3 replies      
Can anybody with more knowledge point out techniques that this would break?

Are there any software or networking patterns that currently rely on localhost _not_ resolving to the loopback?

EDIT: The RFC mentions that MySQL currently differentiates between the two, but that's it.

filleokus 1 day ago 3 replies      

 The domain "localhost.", and any names falling within ".localhost.", are known as "localhost names". Localhost names are special in the following ways []
Is this not implemented on macOS or am I just misunderstanding?

 ~ ping test.localhost ping: cannot resolve test.localhost: Unknown host ~ ping localhost.test ping: cannot resolve localhost.test: Unknown host

nailer 1 day ago 1 reply      
Surprised the more common .localdomain is omitted as a domain rather than having a .localhost domain.
ericfrederich 1 day ago 1 reply      
Sounds reasonable, but would probably break a ton of stuff. Does this provide enough benefits to outweigh the downsides?
age_bronze 1 day ago 1 reply      
There was no RFC for localhost yet?! That's pretty surprising... That this RFC have any practical meaning? People didn't actually register localhost. domain, did they? Is there an actual line of code that this should change? Are they just trying to promote writing localhost instead of
inopinatus 1 day ago 1 reply      
I would very much like to see this draft extended to cover SRV lookup as well.

Right now, section 3 of this draft would prohibit all SRV queries for localhost, which may hinder development and deployment of a SRV based application. That's an immediate problem.

But not only are there existing applications to which it is immediately applicable - it is a design error in HTTP that plain address records are used for resolution. One day this will be corrected, in which case measures like this should continue to apply.

agwa 1 day ago 2 replies      
To be clear, this is not an RFC yet. It's not even adopted by a working group, although I hope it will be.

Mods: can RFC be removed from the title? [Edit: thanks for updating the title!]

alexellisuk 1 day ago 1 reply      
Localhost resolving to IPv6 basically breaks with Docker they unless you give special instructions only listens on IPv4. With curl for instance you can use the -4 parameter but probably best we start saying test the site on in tutorials.
lolcalhost 1 day ago 1 reply      
This sucks. I have registered and am actively using a 'localhost' domain name under one of the new generic TLDs for for emails and account signups for quite some time now.
pmarreck 1 day ago 2 replies      
Why couldn't they just redirect "localhost" at the DNS level to
Show HN: Is the stock market going to crash? isthestockmarketgoingtocrash.com
790 points by truffle_pig  2 days ago   319 comments top 57
pdog 1 day ago 12 replies      
If you're looking for The Single Greatest Predictor of Future Stock Market Returns[1], here it is: http://www.philosophicaleconomics.com/2013/12/the-single-gre...

This is a long read, but it's worth it. The metric can be calculated in FRED[2], and as a predictor of future returns, it outperforms all of the most common stock market valuation metrics, including cyclically-adjusted price-earnings (CAPE) ratio[3]. (Basically, the average investor portfolio allocation to equities versus bonds and cash is inversely correlated with future returns over the long-term. This works better than pure valuation models because it accounts for supply and demand dynamics.)

[1]: http://www.philosophicaleconomics.com/2013/12/the-single-gre...

[2]: http://research.stlouisfed.org/fred2/graph/?g=qis

[3]: http://www.multpl.com/shiller-pe/

runako 2 days ago 9 replies      
I've never seen market valuation expressed as market cap as % of GDP. I'm not an economist, so I'll leave the detailed arguments to them. But it would be at least useful to explain why you think this is a meaningful metric as compared to those typically used to measure market valuation (e.g. P/E ratios etc.).

Your graph also ties your valuation metric to the 2000 peak and the 2008 peak. However, there were crashes in 1990 and 1987 as well. Should readers conclude that the 1987 peak level was also too high, and that therefore the last ~30 years have also been too high as well? (Abstaining from investing in the stock market at levels above the 1987 crash would have resulted in the loss of tremendous opportunity for wealth creation.)

There are a lot of opinions implicitly expressed in this site; it would be good to try to make those explicit.

uiri 2 days ago 2 replies      
For market overvaluation, it says: 9.1 / 10 "DEFCON 4"

DEFCON 5 is peacetime, DEFCON 1 is imminent nuclear war. For example, during the Cuban Missile Crisis, the US reached DEFCON 2. Should this say DEFCON 2 instead? Or is "above" normal readiness the intended meaning?

misja111 1 day ago 3 replies      
The metric used to calculate market overvaluation is interesting but it has little value for predicting a stock market crash.Let's take he last 3 major US crashes:

1987: this crash was caused by automated trading systems which could run wild in the absence of any prevention regulations such as circuit breakers

2000: the collapse of the dotcom bubble

2008: start of the financial crisis caused mainly by opaque credit default swaps and packaged subprime loans

Of those 3, only the dotcom bubble seems to be a bit related to the market overvaluation metric. And even right before the dotcom bubble crash there were plenty of economic guru's who argued that classic overvaluation metrics were not valid anymore because we were now in a 'new economy'.

The other two crashes were caused by black swans; occurrences that nobody was aware of and that were only understood afterwards. Most likely the next crash will be a black swan as well.

lr4444lr 2 days ago 2 replies      
Can someone with an actual economics degree explain to me whether it's a valid criticism of the "Market cap as % of GDP" metric that many US companies derive value from multinational labor and consumption, and if not, why not? Thanks in advance.
mendeza 2 days ago 5 replies      
What about student loan debt, how does that factor into the economy or the stock market being affected?

Right now student loan debt is at 1.4 trillion

source: https://www.debt.org/students/

daotoad 1 day ago 0 replies      
Good idea for a website, should be able to get you some nice revenue from intermittent visits. You probably want to focus on financial services for your ads.

I'm not going to say anything about your numbers and your models other than, without the ability to see how they looked at previous crashes, it's hard to see if the site is useful. To the innumerate masses and emotional investors the flickering numbers are persuasive enough. So they really don't matter.

On the bad side, your UX is god-awful. Use an oldish, slightly crappy monitor to look at it and you will discover that your background is indistinguishable from the foreground. The top bar of the box completely disappears, too. Also, a row of buttons is NOT a good tabbed interface--there is no indication that clicking on "Market Volatility" is going to reload all the content below the row of buttons. Maybe make actual tabs, at least make that stuff a distinct box.

This could be a nice little side product to make you some extra money. Get some GA on there, and slowly add features. I think a bit of interactivity and the ability to customize the predictive models through some drag and drop could actually make the page sticky and get people coming back.

pillowkusis 2 days ago 3 replies      
A site like this seems dangerous at best. Nobody can predict the stock market. Nobody can predict when a stock market is more likely to crash. This site tries to indicate otherwise. Whatever causes the crash it probably won't be one of the indicators listed here.
avip 2 days ago 1 reply      
I love the design and phrasing. This is just a well-done website.

It would be really interesting to see your collapse pyramid over time. How did it look in 2000? 2008?

benmarten 2 days ago 2 replies      
How is the heat matrix diagram calculated? It seems to be wrong. Public Debt has a 3.7/10, while it looks like its around 8.5 in the heat diagram.

Looking at the individual ratings:- Household Debt: 5.5 / 10- Market Overvaluation: 9.1/10- Market Volatility: 0.3/10- Public Debt: 3.7/10--> SUM = 18.6/40 or 46.5%

Also I noted: Drawing a linear trend line through the "Market Overvaluation" diagram, does make it look a lot better though. One could argue that people get used to certain levels, hence a growing trend over time.

Taking only these factors into account, it does not look like the market is gonna crash soon. In my opinion it's likely going to be caused by another factor not listed here ;)

indescions_2017 1 day ago 0 replies      
Correct answer, of course, is no one knows, because the future is opaque and unpredictable. And indeed you have some very smart professionals going to cash or directly betting on a 5-10% correction in the S&P500. And a set of equally smart fund managers calling for a 2600 target by mid-2018.

What we can say with some certainty, based on options activity, is that if a single day 3-4% drop in the S&P500 occurs it can trigger a massive unwind in short volatility positions:


And with several political risk factors on the near term horizon, including the possibility of a government shutdown in late September due to the failure of Congress to extend the debt ceiling (yes, they are arguing over who is going to pay to fund the border wall with Mexico). It certainly should surprise no one if a coming tomorrow could be very different than the extraordinarily low-volatility landscape we face today.

The Case For Long Volatility by Eric Peters


omg_ketchup 1 day ago 3 replies      
Site just displays a blank page. No error or anything.

I think that's a better statement than whatever the app actually does.

saimiam 1 day ago 1 reply      
I was (sort of) there when the 2000 tech crash happened and was in the thick of it when the 2008 crash happened.

This thread and a few offline conversations made me reexamine what I believe about the stock market and the nature of the 2000 and 2008 collapses. Of course, I'm not an econ nor do I have data to back up anything I'm saying.

All manias, from tulips to tech IPOs to housing bubbles are born when the common person joins the frenzy. On the flip side, the mania collapses when the common person walks away or never shows up the party. For the tech IPO frenzy of 2000, the common person never even showed up to use all those exotic new ideas which were getting funded and going public. During the housing bubble, the common person bought and sold houses which setup the flywheel. Eventually, the common person walked away from the asset in question bringing down the entire charade.

Today, the market is soaring. People are starting to wonder when gravity will reassert itself but in my view, this time the difference is that the common person cannot walk away. Unless adblocking and disdain for social media become extremely mainstream, the common person is so busy amusing themselves to death online that they are not going to leave the tech mania. Companies like FB and Google have made the web sticky.

Does this mean the stock market will rise indefinitely? I don't know. I do know that once there is a captive market comprising everyone online, no company is going to stop advertising or figuring out ways to reach buyers online.

We are in a new age where you just can't get away from the web. We are the product but we also have no way of exiting the dragnet.

aagha 8 hours ago 0 replies      
This is sooo cool! Great job.

One thing that might be helpful is to have a separate (informational) page that indicates what the diamond looked like at other period of economic failure--in fact, what it looked like leading up to the period of failure/crash would be really interesting.

I'm curious: How quickly can some of these variables changes? For example, it seems the VIX is at it's low end--how quickly can it spike to say, 30? How fast can the other vars change?

qubex 2 days ago 1 reply      
Economist here. You should really keep in mind that the same GDP must go both towards paying off the national debt and paying off household debt. Also you should track commodities (at the very least, the ratio between put & call options).
where_do_i_live 1 day ago 2 replies      
Your volatility section seems to be a very poor indicator of a future crash in the manner you are using it. Volatility is not a predictor, but instead a descriptor. An analogy I think is the weather stick - Is this stick wet? Then it is raining. It is a very poor item to use in your context.

Further, sustained periods of low volatility often are sometimes indicators of complacency among investors and indicators of higher chances of bubbles. Sustained periods of low volatility are at times indicative of higher future risk of a market crash, not a low predictor. I think you need to re-evaluate how you use volatility.

tveita 1 day ago 1 reply      
Normalizing household debt against the GDP makes the assumption that we are comparing the debt with the ability to pay for it.

But according to graphs like this, even though the GDP has been rising, median households have not been getting a corresponding increase in income: https://en.wikipedia.org/wiki/Household_income_in_the_United...

So the income we are adjusting against is not necessarily going to the people that are in debt!

ringaroundthetx 1 day ago 0 replies      
So VIX doesn't give an indication of much. The VIX formula has changed so many times, and the human behavior around the assets that VIX tracks has changed to reflect those changes and the new products those changes are based on.

Different people gamble in weekly S&P500 options than gambled in monthly S&P500 options. Different people gamble in the 5 consequetive week at any given moment weekly options, than gambled in the single week at a time weekly options.

The options market itself has had ebbs and flow in interest.

And the self fulfilling prophecy of keeping the market propped up when everyone buys PUT options expecting it to crash has disillusioned a lot of people from participating at all. People know what the central banks are up to, why pretend to have confidence in any of it. The Swiss bank is printing money to buy US stocks for free. Everyone's creating money through new bond issuances to buy things for free.

This all contributes to a lower VIX.

Nursie 1 day ago 1 reply      
"NaN% more overvalued than just prior to the 2008 financial crisis,"

I think there might be a few coding errors still lurking in there.

mxschumacher 2 days ago 2 replies      
American companies sell products & services outside of the United States. Comparing American GDP with the aggregate value of the US stock-market is deeply misleading, especially given a historical comparison: foreign markets such as China have gained in relative importance over timeframe under consideration.

When looking at debt, one should not just observe the nominal amount, but also the interest rates, which have never been lower. Large companies can tap public debt markets and borrow billions at 1.5% over a timeframe of ten years. Risk is thus lower than the website suggests (at lower interest rates, a company can carry more debt). Additionally, returns to equity will be higher (the I in EBIT is smaller, so profits are bigger).

timsayshey 2 days ago 2 replies      
Really cool idea. As someone that hasn't really investigated the market indicators for collapse this is really eye opening. It really breaks things down into plain english. Hope this goes to the top for some rational/interesting conversation.
anonu 2 days ago 1 reply      
As the site makes clear, nobody really ever knows if the market is going to crash. On the market valuation side they claim the current market is overvalued. But overvalued is a relative term... As you have to value versus something, and that something is usually something historical.

The way I see it though is the markets are a big voting machine.. and they're making predictions about the future and incorporating future expectations. With the current US administration still pondering over tax plans and infrastructure stimulus packages that are promised, market may be underpriced???

mathiasben 2 days ago 2 replies      
I feel as though the "stock market" following the 2008 crisis has become further insulated from the larger economies fundamentals. wages can continue to not keep up with inflation, savings rate continues it's downward slide, household debt service payments consume an ever increasing slice of disposable income, etc... all the while the type of dramatic dislocation event similar to 1929, 1987 are unlikely to occur. the market "circuit breakers" ensure any crash is a slow moving trend and not a single calamitous event.
bluetwo 1 day ago 2 replies      
The volatility index, or VIX, has become a popular measurement to reference in the context of predicting the market over the past couple years.

The problem is that it does not seem to have any real predictive power and I have yet to see any shred of evidence that the VIX has been shown to have predictive power over the future value of the stock market.

It is calculated from past price variance and is used in calculating the theoretical price of options, but that is it.

Does anyone have any evidence the VIX has value?

Kiro 2 days ago 1 reply      
I got a "Add Create React App Sample to your home screen" notification on my phone.
csomar 1 day ago 0 replies      
Does it make sense to have "marketcap" / GDP if the Nasdaq/DowJones has non US companies like Alibaba? Or is it taking these into account?
apsec112 1 day ago 1 reply      
I think you could estimate much more accurately with the prices of deeply out-of-the-money put options. Those are effectively a betting market on whether stocks will crash or not. We should expect option prices to take into account every major factor (not just these four), because if they didn't, people would get rich by trading on the "missing" info until prices corrected themselves.
lordnacho 1 day ago 0 replies      
One could argue that the volatility scale should be the other way round; that the diamond should be showing extreme values on everything other than household debt, which is middling.

The market is normally calm on the way up, which is why you might think its current upward movement will soon be interrupted by a volatile down-move.

brookside 1 day ago 0 replies      
A great read on how to capitalize on the upcoming crash! The Sale of a Lifetime: How the Great Bubble Burst of 2017-2019 Can Make You Rich[1]

Also good is the author's earlier book The Great Crash Ahead [2] "outlining why the next financial crash and crisis is inevitable, and just around the corner coming between mid-2012 and early 2015"


1. https://www.amazon.com/Sale-Lifetime-Great-Bubble-2017-2019/...

2. https://www.amazon.com/Great-Crash-Ahead-Strategies-Turned/d...

grandinj 2 days ago 1 reply      
The answer is of course: YES.

The more important question is when, and the answer to that is "who knows".

The market is a chaotic system, with severe non-linear responses. As such, it can remain stable much longer than people think, and crash much harder than anyone expects.

cs702 1 day ago 1 reply      
I love the idea, the simple design, and the humble tone of the byline ("no one knows for sure, but there are indicators that can help us guess. We can chart these indicators to give us the illusion of foresight.").

However, I have two suggestions. First, the numeric rankings (such as "5.5 / 10") need context: why not say something like "10 is the highest value reached in the historical record"?

Second, the explanations you give for chosing these indicators need a bit of work, as evidenced by some of the comments and questions on this thread. Most lay readers won't understand why the ratio of total stock market capitalization to annual GDP is important.

TekMol 2 days ago 3 replies      
The page strives solely on it's nice graphics, and sensationalist wording. There is little to no content of substance.

For example the page calculates "Market Overvaluation" as the US stock market value divided by the yearly US GDP. Hilarious.

yuhong 1 day ago 0 replies      
Yea, the US economy is based on constantly growing debt basically, which can't last forever. My favorite is the ad bubble now, and ads are basically designed to increase consumption. It is probably worth mentioning China too: http://www.zerohedge.com/news/2017-08-06/chinas-minsky-momen...
mcguire 1 day ago 0 replies      
Is this a psychological experiment? All I get on Android Chrome is a white screen.
Glyptodon 1 day ago 0 replies      
Question (as someone without domain knowledge): could someone explain what the expected relationship between GDP and total stock market value is? GDP represents non-publicly traded, and even non-private activity, while presumably the stock market's valuation is at least somewhat driven by expectations of future growth/profit, rather than current productivity. I don't doubt that there's a relationship of some kind, but what is the simple ratio actually showing?
lg 2 days ago 2 replies      
Could the fact that a lot of US companies book profits overseas and keep them there for tax reasons foil your assumption about the meaning of a high US market cap:domestic GDP ratio?
module0000 1 day ago 0 replies      
So, if the stock market is hypothetically predicted to crash in 10-25 days - what are you going to do? Short it now? Short it later? Buy?

Just curious what HN readers think. For the giggles...I'm going short when the tape says market sell orders exceed the rate of bid additions, and the opposite for going long. I like long-term analysis as much as the next guy, I just never, ever, ever, ever, ever make decisions based on it.

tome 2 days ago 2 replies      
Market Volatility section:

Current risk:

NaN / 10

"Calm waters"

(I'm using Edge)

AJRF 1 day ago 0 replies      
"We can measure Market Overvaluation by looking how much the stock market costs vs how much it is providing." Isn't this the opposite of what the stock market is supposed to provide? I assumed valuations for the most part are guided by what a companies outlook is for the future, not the present.
jostmey 1 day ago 0 replies      
"We can chart this to give us an illusion of foresight"

Got to respect the Author's humility in foretelling the future

coverband 1 day ago 0 replies      
Interesting analysis, but I'd not have included public debt as a risk factor. If anything, increasing public debt provides upward support for the equity market, regardless of whether the money goes to public investments, tax cuts or bad government spending.
sigmar 1 day ago 1 reply      
>The VIX is generally consistantly low (10 - 15) until it isn't. To get a sense of what a crisis would look like, we can compare to a few historical values.

What's the point of using a metric that can turn on a dime in a predictive model?

socrates1998 1 day ago 0 replies      
Low Volatility might actually be an indicator that the stock market is going to blow up, rather than stay calm.

Volatility tends to cluster, and periods with really low volatility are often an indicator that there is a big movement coming.

cm2187 1 day ago 0 replies      
Blank page for me. Don't know if it is there but a nice chart is size of the Fed B/S vs S&P 500, since 2005. Suggests a large part of the valuation of stock is generated by QE, which the Fed intends to start withdrawing this year...
neilwilson 2 days ago 0 replies      
'Public Debt' is a private asset. Why is having more wealth a bad thing?

The idea that being 'in credit' with a sovereign government with its own currency is a problem has been thoroughly debunked. Primarily by Japan.

Time to stop repeating the myth.

kmfrk 1 day ago 0 replies      
If nothing else, I like how this might stir some interesting discussions about the state of the economy.

One thing I'd like is a link to the cited data to make it a little more serious and conducive to debates.

peternicky 1 day ago 0 replies      
Why does this site report "the stock market is closed"?
forbiddenlake 2 days ago 0 replies      
Says "Stock market is closed" at 10:45AM EDT. Is it really?
neom 2 days ago 0 replies      
Distribution of household debt is to significant to look at the health of the economy in this way, especially so when you look at how the GDP is generated and who is generating it.
unknown_apostle 1 day ago 0 replies      
Cute site :-) Btw we have the added issue that the volatility of volatility appears to be rising. Meaning periods of apparent big calm turn into big price swings more rapidly.
artursapek 2 days ago 1 reply      
A high VIX would indicate that the market is crash-ing.
tambourine_man 1 day ago 0 replies      
Site's broken on mobile:


movedx 1 day ago 0 replies      
Can you please open source this under an MIT or some license you agree with?
franciskim 1 day ago 0 replies      
Lowest volatility ever in 27 or so years according to VIX apparently, which is actually a warning sign.
odammit 1 day ago 0 replies      
Nah, Trump says it's fine. Don't worry about it. It's the best. May see a dip in 2020.
malynda 1 day ago 0 replies      
Another pedantic remark: Next to the clock, you should include a timezone. Very interesting!
rrggrr 1 day ago 0 replies      
Household debt should be measured against household income and not against GDP.
HTML5 Version of the Tron:Legacy Boardroom Scene robscanlon.com
696 points by PleaseHelpMe  3 days ago   65 comments top 27
arscan 3 days ago 8 replies      
Glad to see some people are getting a kick out of this. I built it a few years back while learning webgl, css3, node.js, redis, and modern (at the time) js tooling. Source over at https://github.com/arscan/encom-boardroom

I didn't build this with any real practical application in mind at the time. But some people have reused components in their own projects over the years, particularly the globe (https://github.com/arscan/encom-globe).

thatcherc 3 days ago 2 replies      
Are there any efforts to make movie-type window styles for real use? The Tron:Legacy style would be a fun one to have, as well as the one used in the computers in Westworld (similar light-blue-on-black)[1] and Interstellar [2]. I'm sure it's a more difficult process than I'm imagining, but it would be really cool to be able to use the window and interface themes of the computers in your favorite show or movie, especially since some have such great designs.

[1] - https://www.youtube.com/watch?v=Ikup60uEg0c[2] - https://youtu.be/bmz9lMP6aQU?t=2m4s, visible for a brief second just after 2m4s

kang 3 days ago 0 replies      
DannyDaemonic 3 days ago 1 reply      
This is beautiful. There needs to be a plugin api for this so we can make our own using charts using the built in command line and associated widgets.
thinkpad20 3 days ago 0 replies      
The term "mad skills" comes to mind. I would love to see a breakdown of some of the techniques he used to construct this.
Splines 3 days ago 0 replies      
Subreddit of movie-fake-UI: https://www.reddit.com/r/FUI/
emilioolivares 3 days ago 1 reply      
What, the .js file for the globe itself is 43 thousand lines of code. Was this a weekend project? How do you find the time? Very well done my friend. (https://github.com/arscan/encom-globe/blob/master/build/enco...)
raykanani99 3 days ago 1 reply      
Holy cow. Did you use a charting framework for the stream feed? How did you get such a pretty globe?
nautilus12 3 days ago 0 replies      
If only we could get a version of this to render in the terminal so we could hack our workstations to actually look like this. I use tmux pretty heavily. Id love it if panes looked like this
hughw 2 days ago 1 reply      
My admiration is unbound. I have one reservation -- I wish it would handle history correctly. I wish each click e.g. github produced a new URL I could email to a friend so they could view the same scene. And then I wish I could hit "back" to recover the landing page. None of this diminishes how beautiful this is, and I'm sure it just wasn't part of what you were trying to explore.
collinmanderson 2 days ago 1 reply      
Fun to see Event Source show up here. I feel like WebSockets are all the craze, but Event Source seems to be a really simple alternative that uses only HTTP.
Animats 2 days ago 0 replies      
Check out Harvard's economic globe.[1] This looks like the globe from this demo, but it's a real interactive tool.

[1] http://globe.cid.harvard.edu/?mode=gridSphere&id=PH

pmattos 3 days ago 0 replies      
Very cool stuff... even `ls` works in the shell ;)
luord 2 days ago 0 replies      
Man, fantastic.

I should probably get into doing something like this in my free time instead of browsing reddit or tvtropes.

metmac 3 days ago 0 replies      
So cool to see this revived. I remember stumbling upon it awhile back.
mattnewton 3 days ago 0 replies      
This is super cool, I love the from aesthetics (if not the movie).

How do you make you or tapping not zoom again? Is there a HTML meta tag or something? It would improve the usability of the keyboard on mobile.

cjsuk 3 days ago 0 replies      
You got in :)
jarym 3 days ago 0 replies      
Really damn impressive!
Shinchy 2 days ago 0 replies      
This is fantastic, really fun to use and would make a cool portfolio website.
fizixer 3 days ago 1 reply      
You could add a video stream from a live online news channel on the top right? (on my side the top right was pretty much empty)
baalimago 3 days ago 1 reply      
I've not even seen tron since im too young... but all i can say is that i'm mighty impressed
bluescreenofwin 2 days ago 0 replies      
Man this brings me back.. Thank you for creating this!
th0ma5 3 days ago 0 replies      
They used Processing some I think in the movie which now has a WebGL version.
samgranieri 3 days ago 0 replies      
Holy shit! This is amazing. Great job
edpichler 3 days ago 0 replies      
Wow, beautiful!
cdevs 3 days ago 0 replies      
I love this.
Kenji 3 days ago 0 replies      
The funny thing is that it loads faster and has higher fps than many websites that display simple blogposts and a couple of images, but pull in literally megabytes of JavaScript and other bloat.

People, learn from this. Seriously. The web doesn't have to be slow if you put effort into it.

Big Companies and the Military Are Paying Novelists to Write Sci-Fi for Them newyorker.com
413 points by anthotny  1 day ago   159 comments top 36
tzs 19 hours ago 2 replies      
Companies commissioning stories has gone on for a long time.

For example, the Isaac Asimov story "My Son, The Physicist" was commissioned by an electronics company to run in an ad in "Scientific American".

Another Asimov example. "Think", IBM's in-house magazine, commissioned Asimov to write a story based on this quote from J. B. Priestly:

> Between midnight and dawn, when sleep will not come and all the old wounds begin to ache, I often have a nightmare vision of a future world in which there are billions of people, all numbered and registered, with not a gleam of genius anywhere, not an original mind, a rich personality, on the whole packed globe

Asimov write the story "2430 A.D." about a world where Priestly's nightmare had come true. (The title comes from his estimate of when human population at its current growth rate would reach the point where the Earth had so many people that there were no resources left for non-human animals).

The funny thing about this story is that "Think" rejected it, because they wanted a story that refuted the quotation. So Asimov wrote another story, "The Greatest Asset", that refuted Priestly, and sent that to "Think".

"Think" then decided they liked the first story better and ran "2430 A.D."!

I'm pretty sure that there was at least one other similar case with Asimov.

bbctol 23 hours ago 2 replies      
Reminds of this story from a while ago: the head writer for Call of Duty switched to working at a think tank, given his experience imagining the future of warfare. http://foreignpolicy.com/2014/09/22/call-of-duty-star-video-...
divbit 22 hours ago 3 replies      
I honestly don't care if there is a "sinister" purpose behind the good scifi I've read. Fund the next Iain banks please. In fact - throw me $50k to live for a year and you've got yourself a book - or I can at least guarantee some words on pages mentioning spaceships / robots, etc.
charlieo88 36 minutes ago 0 replies      
From the article:>> Companies that market directly to >> A.I. software, rather than to humans, >> might gain a competitive advantage."

I don't know why, but this is really disturbing.

"Hey Siri, find me a widget"

"I've found three locations with widget in your area. Would you like me t... WHOA! Check out the bytes on that ad!"

jermaustin1 22 hours ago 8 replies      
I do something similar. I pay writers to write me sci-fi and fantasy stories based on my prompts! It allows me to read what I want to read!
rahulpandita 23 hours ago 2 replies      
We proposed something similar for the field of software engineering research.


Pasting abstract here

"Software engineering researchers have a tendency to be optimistic about the future. Though useful, optimism bias bolsters unrealistic expectations towards desirable outcomes. We argue that explicitly framing software engineering research through pessimistic futures, or dystopias, will mitigate optimism bias and engender more diverse and thought-provoking research directions. We demonstrate through three pop culture dystopias, Battlestar Galactica, Fallout 3, and Children of Men, how reflecting on dystopian scenarios provides research opportunities as well as implications, such as making research accessible to non-experts, that are relevant to our present."

legohead 21 hours ago 5 replies      
Interesting they mention and get quotes from the author of the Three Body Problem series. I just finished reading that series, and the whole time I had the feeling that it was written more to convince people of something than to actually entertain the reader.

The ideas in the books were pretty amazing, unique, and imaginative; but the story and writing itself was quite sub-par.

edit: I listened (audio books) to the English versions. I thought the translation was excellent. I don't want to spoil anything, but the author has a very dark vision of mankind and kept having his characters ostracized to the extreme which got old pretty quick. I didn't feel like the books were missing anything descriptive from lack of translation.

vinayak 20 hours ago 0 replies      
Microsoft had released Future Visions Sci-fi series after inviting several sci-fi authors to their research labs - New Link - https://news.microsoft.com/features/future-visions-anthology...

The book itself can be downloaded at https://news.microsoft.com/futurevisions/

cpete 23 hours ago 1 reply      
I posted something similar on HN nearly a year ago: https://www.fastcompany.com/3063187/scifutures-probes-your-c...

It's a nice complement to this New Yorker article :-)

schlipity 19 hours ago 2 replies      
This is actually part of the plot of the movie (Three Days of the Condor) and book (Six days of the Condor). The protagonist works at a CIA site where they read basically everything released and break down the plots to see if it has any strategic value and/or state secrets.

If I had to guess, this still goes on at the state level, but this article is about making this sort of data (minus the state secrets awareness) available to the corporations able to afford it.

I'd also say that I think this might be an amusing reversal of the old mantra: Ideas have no intrinsic value. This business is pretty much about the expression of ideas. Maybe they have value because they are writing them down and marketing them.

tabtab 22 hours ago 1 reply      
"How I used Microsoft Cloud 365 to escape Darth Vader and blow up the Death Star! Uplinking R2D2 was totally seamless."
zitterbewegung 23 hours ago 0 replies      
Reminds me of a quote from Alan Kay: "A change in perspective is worth 80 IQ points."
KineticLensman 22 hours ago 0 replies      
If you want to read a concrete example rather than the generic relevant novels mentioned in the article the Directorate of Land Strategic Concepts, National Defence Canada, commissioned Karl Schroeder [1] in 2005 to write an SF novel to help explore them future doctrine and concepts. PDF is available at [2]. He wrote a second (post 2010) but I don't have the reference now (a 'friend' never returned it). I recall that it involved a CAF unit operating in a trans-national mega city (somewhere in Asia) having to subvert the all pervasive city AR so that they could complete their mission. I was impressed!

[1] https://en.wikipedia.org/wiki/Karl_Schroeder

[2] http://www.kschroeder.com/foresight-consulting/crisis-in-zef...

wodenokoto 23 hours ago 0 replies      
Kinda funny to think that maybe, just maybe, the social network was sponsored by Google or Twitter as an attempt to make the competing platform less desirable.
fencepost 23 hours ago 1 reply      
Visualizing and communicating how things will be used is very powerful and important.

My memory may be faulty, but I think it's Alan Cooper's "About Face" that basically advised not just describing the interface and how it'll be used, but how Bob the 73 year old luddite who hates all computers is going to interact with it (context in flight movie systems). Creating characters and having them interact with technology that doesn't exist yet is what SF writers DO.

fundabulousrIII 23 hours ago 1 reply      
Congrats. The .mil & .biz folks have been acknowledged in what has been implicit in every advancement I've seen in the last 40 years. sci-fi pop fiction postulation and conjecture based on the science community brain trust. Most of these guys were consultants or engineers anyway and empowered to write. There are iconoclasts like Ellison and the visionaries but the hard science guys always get paid for their visions.
pvaldes 20 hours ago 1 reply      
Last days the defcon issue, and this week for some particular reason we have a few similar articles here and there talking about how fabulous and great would be for a 'good nerd' to work for the military, government, etc...

Can we spot a pattern here? PR damage control?

stcredzero 16 hours ago 0 replies      
Back in the late 80's, I ran across a statistic that just the amount amount the military spent supporting its bands was as large as the entire budget for NPR. I think this is also covered in Manufacturing Consent. I remember flipping channels as a grade schooler and running across US military propaganda on the nearest independent TV station. (Which is now Fox, unfortunately. Sad, because this is the station that introduced me to anime.) That stuff gave me nightmares! An 80's era Soviet land assault is basically some nightmare perfection of the Blitzkreig. The whole horizon turns black and rolls at you like a wave, as rank after rank of smoke generating tanks charges you at high speed.
viach 20 hours ago 0 replies      
Interesting read, somewhat related:

A 19th-Century Vision of the Year 2000


Gotperl 23 hours ago 1 reply      
This is part of the plot by Armada by Ernest Client (which was also heavily influenced by Ender's Game)
dctoedt 22 hours ago 1 reply      
My wife watches police "procedural" stories (Law & Order, CSI, NCIS, etc.). I've long thought that some of their plot lines would make for interesting "what if" training cases for real cops.
exhilaration 23 hours ago 1 reply      
Hey, the article mentions HN-favorite Ken Liu, author of the Three-Body Problem.
AdmiralAsshat 22 hours ago 1 reply      
Not terribly surprising. People forget that Neuromancer was a commission.
runevault 22 hours ago 2 replies      
Based on the article about Neil Stephenson that was posted here not long ago I'd been thinking along these lines, didn't realize someone already made a business out of it. Interesting.
remarkEon 10 hours ago 0 replies      
Anyone read Ghost Fleet? It wasn't commissioned, as far as I'm aware, but it so spooked the Pentagon they hired the authors as consultants.
dogruck 23 hours ago 0 replies      
I wonder what the annual revenues are. I'm skeptical.
vonnik 23 hours ago 1 reply      
The US Army Lab has solicited sci-fi stories and awarded grants to build the tech they describe.
samstave 23 hours ago 2 replies      
Wasn't there something about Clancy being working with the CIA to massage their image?

Military and intelligence and politics have always manipulated media...

Also, recall "Americans army" was being ostensibly used as a recruiting/psyche-molding tool...


Finally: look at cyberpunk, and the top five writers in that category, as well as anime (ghost in the shell, etc) which have in-formed modern reality with all the millions of tech-workers from around the globe have worked since their childhoods to create aspects of those worlds into current reality...

Is the totalitarian police-surveillance state an emergent feature of such a reality?

All crazy military capability comes from able-minded imaginations saying "wouldn't it be cool if..." without the discernment of the far-reaching implications...

podiki 20 hours ago 0 replies      
Did no one else immediately think about the plot to Watchmen (the graphic novel, not movie)?
leoc 18 hours ago 0 replies      
There was a rash of initiatives of this kind after 9/11 as well.
mmaluff 22 hours ago 2 replies      
This is the worst story I've read all week. Capitalism is destroying our culture.
rurban 23 hours ago 1 reply      
"Sweet Tooth" Ian McEwan: CIA, MI5 and MI6 are sponsoring many writers, such as some companies sponsor open source devs
whyzoidberg 20 hours ago 0 replies      
Finally some good work for writers! But seriously artists have been kept alive by the state for millennia, no shock here
igorgue 19 hours ago 0 replies      
The author of this novel works at Magic Leap: https://en.wikipedia.org/wiki/Snow_Crash
lowglow 20 hours ago 0 replies      
I wonder if Neal Stephenson gets paid to do this. He's predicted parts of where we're headed with Diamond Age and Snow crash.
cryoshon 19 hours ago 0 replies      
i'd kill for opportunities like this.

but maybe we have to make our own opportunities, so: how about a startup that provides science fiction writing thought leadership as a service?

dibs on founder

Rest and vest: engineers who get paid and barely work businessinsider.com
480 points by SQL2219  2 days ago   317 comments top 48
boulos 2 days ago 8 replies      
There seems to be conflation in this article between two very different groups.

Group A is folks who are acquired and have outsized grants that say vest over N years (N between 2 and 4). It turns out the acquisition was probably a mistake, but the acquiring company made it (and won't own up to it). That's what's described in the Facebook and Microsoft examples. This is the classic "rest and vest" scenario (Note: an acquisition is not required, just any outsized grant).

Group B is "just" engineers at Google, Facebook, etc. getting paid really well for not doing much, while hanging out with the lavish perks. I've never heard of anyone refer to this as "rest and vest". In particular, I found this quote disturbing:

> There are a lot 'coasters' who reached a certain level and don't want to work any harder. They just do a 9-5 job, wont work to get promoted, dont want to get promoted.

At Google (and elsewhere), it's considered fine to reach a senior / terminal level and stay there. Is a VP or Director of Engineering lazy if they never move up? Of course not. The same is true of individual contributors.

Finally, the numbers mentioned for compensation are normal for very senior engineers at Google (and again, Facebook, Microsoft, etc.). This isn't "rest and vest", it's just business as usual. I don't particularly agree with the folks who spend their days in classes, taking long lunches, etc. but if they get their work done, what do I care?

lisper 2 days ago 4 replies      
It's not just SV that has this phenomenon. I worked at the NASA Jet Propulsion Lab from 1988 to 2000. I had risen to the rank of Senior Member of the Technical Staff, the second-highest rung on the technical career ladder. Beyond that there is the rank of Principal, which is very hard to attain. It's essentially the equivalent of getting tenure. It requires peer review. Most engineers never attain it, and I was not optimistic that I ever would. So in 2000 I decided my JPL career had peaked, and so I quit to go work for an obscure little Silicon Valley startup in Mountain View. ;-)

To my surprise, when I announced my departure, a bunch of people suddenly came out of the woodwork to tell me that they really didn't want me to go, including a number of very senior managers. So I used that as leverage to negotiate a deal for myself: I would come back after a year on the condition that I be promoted to Principal. Which is what happened.

The problem was that my promotion did not in any way coincide with JPL's strategic needs for my skills. One of the reasons I had left was because I had been on the losing side of huge political fight (http://www.flownet.com/gat/jpl-lisp.html) and when I returned I couldn't find a project that was willing to take me on. But they couldn't fire me because I was a Principal. So I basically spent the next three years getting paid for doing nothing, and getting pretty depressed about it. It's actually not fun to feel like a parasite, at least it wasn't for me.

asah 2 days ago 3 replies      
Can confirm.

But it's not a nefarious thing and the people aren't slackers: 90% of the people who end up in this position are ass-kickers who strive to have impact and get bored: most feel bad about slacking but their bodies and minds simply need a rest. They created BILLIONS in value and even providing tech support, they "pay for themselves" many times over -- that's why companies like Google keep them around.

Subtly: the kind of people who end up resting-and-vesting are precisely the kind of hyper-ambitious people who develop unique knowledge and skills.

YANT2017 2 days ago 0 replies      
This article is very misleading. It's not uncommon for engineers who've been instrumental to a key product or development to be given a light duty afterwards. This is primarily because these folks bust their ass and quite literally are exhausted once their project ships. The time with light duty is meant to retain this key talent and give them back some work-life balance. Also if your thing lands and it's big enough you usually get promoted and they want you to focus on soft skill development, literally making friends, so you can go on to do something bigger. My last half, my manager told me that all he wanted me to do this half was make friends. This is because he was giving me space to find the next big thing. When you shift from task oriented work to bigger picture stuff, you can't just start building stuff thinking people will use it. You have to spend time talking to people about what problems they have and see if you can come up with a way to solve them. It's really not unlike a startup in that regard.

There's also the old joke of the mechanic that comes to fix the machine by knowing where to tap with a hammer. So having people around who know where to tap is key. They are well worth what they are getting because sites like Google, Facebook, Amazon, etc... can't go down and if they do millions of dollars are burning for each minute those sites are down.

WalterBright 2 days ago 1 reply      
I attended Caltech in the era that Feynman was a professor there. I heard he was paid XXX a year. I opined that was ridiculous, who could possibly be worth that much?

An upperclassman laughed and told me that Feynman was worth that much to the university even if he did nothing. Attaching his name to the university brought in donations, grants, and top talent.

Of course, Feynman being Feynman, worked like hell anyway.

geff82 2 days ago 5 replies      
Ok, thanks for sharing this. I made this observation for the last 3 years myself, not as an employee even, but as a contractor. I had three positions at two companies, all paying quite high (150k$/year). I changed the positions because the workload was so low that I had an hour of work a day, then pretending I was doing work for the rest of the day which I can't stand for more than a few months. Now I changed again in the hopes of having real work to do, comes out that they contracted me only for "if there will be work in a few months". Interviewing several people on what I can do for them: essentially nothing. "Maybe you could google if using docker would make sense". On the one hand this kind of "work" feeds my family and hives me lots of freedom, but on the other hand it leads to nothing. And I am usually not the only one who has no idea why they are going to work.
ChuckMcM 2 days ago 1 reply      
And they wonder why house prices in the bay area are so high. :-(

I started work at Sun Microsystems on the Monday after they had IPO'ed (the previous Friday). It was about a year later when all of the various restrictions on personal stock sales had been lifted that I clued in that some people just didn't care any more about work and it was quietly explained to my shocked ears that these people were now multi-millionaires and working was no longer 'for a living' it was 'for the fun of it.' Or not. And I asked why they didn't just leave and the answer was simple, because it gave them something to do and their friends all worked here. Further many of them had been given additional "refresher" options and the more the stock went up the more they were worth thousands a month in additional value down the road.

I was fascinated to see how the different people responded to that new found wealth and the options it brought with it. For the good ones, it empowers a sort of fearlessness to do the right thing even if you boss doesn't think its the right thing. Or to advocate for an important point that might be politically inconvenient for the company. For some it affected their opinion of everyone else as if they were somehow so much "more" than folks who hadn't been there pre-IPO.

Fortunately most of the latter types left fairly quickly.

I could see how it could easily be the 'best' management choice to have someone like that not putting in too much face time at work. Bad managers control their reports by threatening to fire them, if you can't control them they are a threat to the bad manager, better to keep them far away from anything that could set them off.

That said, if you find yourself in this place the absolute worse thing you can do is to do nothing. Get healthy, learn something, use that 'free' time productively. It isn't like you can get it back later.

icelancer 2 days ago 0 replies      
>Medina said he experienced the high-pay, no-work situation early in his career when he was a software engineer in grad school. He finished his project months early, and warned his company he would be leaving after graduation.

>They kept him on for the remaining months to train others on his software but didn't want him to start a new coding project. His job during those months involved hanging out at the office writing a little documentation and being available to answer questions, he recalls.

This isn't a good example. The company budgeted X dollars over Y months for a total comp package of Z for an engineer they knew had a discrete timeline, and the engineer finished in Y-3 months. What should the company do, fire the engineer and save delta-Z? The company got what it wanted and more by having him stick around and answer questions and do documentation work for 8-10 hours a week of "free" labor.

ghettoimp 2 days ago 1 reply      
So working 9-5 is "coasting"...? Fuck you. I've got kids.
WalterBright 2 days ago 4 replies      
There's something corrosive to the soul about having no purpose in life.

I retired once. It lasted about 6 weeks, then I decided to create the D programming language. I plan to work until my mind no longer functions. I'm not interested in retiring.

bane 2 days ago 1 reply      
I did the work-from-home (wfh) thing for about 5 years across two different jobs. The first job was the worst kind of wfh situation because there simply weren't any boundaries between work and home and day work bled into night into weekends.

The second was the other worst kind, paid very well to do almost nothing, and again day nothing bled into night nothing into weekend nothing. I tried to use it to study things or learn other topics, but every once in a while I'd be needed for a few days, go and put a fire out and be back home doing not much at all. The reason for the situation was a disastrous corporate management. However, the situation was so great in theory (get paid top-10 metro senior pay to do nothing at all) that I actually had a hard time changing jobs because I kept telling myself I actually enjoyed screwing around.

Given a binary choice of one or the other I'd actually choose the second job again, but I'd structure my days very differently and try to be much more productive. The good news is that life isn't binary, and I'm in a place now where I work most days in an office, but can wfh when I need to, and rigorously control my schedule so work and home-life don't intersect. I took a pay cut, but I love this current work much more than either of those two jobs (and my wife is much happier as well) -- lessons learned I guess.

ChemicalWarfare 2 days ago 0 replies      
The article lumps few different scenarios under the sensationalist "look, ppl are making shit ton of $$$ and are barely working!!!" umbrella.

None of the scenarios are unique to SV or even IT world in general, the only "shock factor" is the compensation figures.

But again, most of the scenarios are pretty typical to corporate environments. Unless you're on some "kick ass all-star" team, once you start growing you can cruise if you choose to.

What struck me as odd is the "Just dont talk about it and everyone will assume you're on someone elses team" bit. Can't really picture an environment where a person doesn't show up the next day and everyone just "assumes" they are on a different team now...

IBM 2 days ago 4 replies      
This is probably why Apple, a hardware company, has operating margins that are higher than Google and Microsoft (even though their gross margins are almost half of Google's and Microsoft's).

The only person I can think of that might have this arrangement at Apple is Scott Forstall. I think that's why he's been radio silent until very recently (or he could just be very loyal to Apple). Maybe Katie Cotton when they changed their approach to PR from wartime to peacetime, but that could just be a regular retirement.

I mostly don't understand how Google and Microsoft employ so many people, or what they even do.

>"I've actually had a number of people, including today at Google X, ... send me pictures of themselves on a roof, kicking back doing nothing, with the hashtag 'unassigned' or 'rest and vest.' It's something that really happens, and apparently, somewhat often," the actor Brener told Business Insider's Melia Robinson last year.

Called it a year ago [1]:

>I've speculated for a long time that basically anything interesting Google says they're doing is essentially meant to be a jobs program to keep employees from leaving, PR for external stakeholders like investors, media, being attractive to potential employees, etc. They seem to have lots of formal ways to keep employees from leaving/close as well including investments off of Google's balance sheet (not GV or Google Capital) into ex-employee startups and just flat out paying people not to leave (which is the arrangement I'm guessing that Matt Cutts is under). It all seems very Microsoft of old.Can anyone at Google (or ex-employees) tell me if this is true?

[1] https://news.ycombinator.com/item?id=12410662

brendangregg 2 days ago 0 replies      
I know the rest and vest type. It can be demoralizing for others to know that some in the company aren't pulling their weight, because they got lucky in the past with stock offers.

It's one of the many reasons I like Netflix. We allow engineers to be paid almost entirely in cash. No one is resting and vesting that I know of -- not only because we wouldn't tolerate unmotivated people -- but because there's no vesting schedule that I know of. AFAIK, you can leave any time with everything. If you want to leave, then leave. We'd rather hold onto people that actually want to stay and get stuff done, and be self motivated.

pixelmonkey 2 days ago 6 replies      
One of the interesting trends of the past 10 years is the degree to which "big tech" has replaced "big finance" as the place for the elite to go to collect huge paychecks for relatively "nice" white collar work.

It makes total sense for top SV engineers to get paid well, IMO. But I am afraid working for these big tech firms is starting to have that feeling of "elite pedigree" that pervades complacent industries, like finance.

In 2002-2006, one could have written a similar article, but about top staff at Goldman, Morgan, UBS, etc. There were plenty of $300k-$500k salaries being paid for maintenance work for profitable business lines.

Options and RSUs are an interesting twist in Silicon Valley. To compete with the stock option packages given out by startups to early employees, Google and Facebook grant RSUs (and similar) instead. In Wall Street, the "golden handcuffs" used to be a near-guarantee of a year-on-year raise, an end-of-year cash bonus, and a track toward promotions that had built-in pay increases. No one wanted to throw away their time invested in a single firm. SV firms are different in that turnover is high, so vesting acts to counteract that. They have such fast-growing stock values, the stock grants can also be used in lieu of bonuses. Plus, to management, it really is "funny money" that does not actually increase operating expense.

Anyway, though the mechanics are different, it seems the net result is the same. "Golden handcuffs" are as real in tech as they are in finance.

The saddest reflection I have on reading this article is on how capitalism seems to value different professions wrongly.

These salaries are bigger than top specialist physician salaries. And physicians need 12-17 years of post-undergrad training, as well as often requiring $200k of medical school student loan debt.

It just seems like if Google and Facebook can afford to pay this price for engineers (who add leveraged value via their software contributions), capitalism should figure out how to pay doctors more, as well.

And go down the list of other "non-BS, but comparatively underpaid" professions like teachers, firefighters, etc. They could all use a compensation upgrade.

But what is the exact mechanism that is making it so finance and tech are among the only fields where labor compensation is commensurate with leveraged value-add?

capkutay 2 days ago 3 replies      
These engineers are worth more to them just sitting around relaxing, being content with their lives instead of taking a high octane job at a competitor or startup that will eventually compete with one of their smaller services (mail, ad analytics, etc).
user5994461 2 days ago 1 reply      
> There are a lot 'coasters' who reached a certain level and don't want to work any harder. They just do a 9-5 job, wont work to get promoted, dont want to get promoted.

Finally someone who understands how being an employee works. You are paid to be present ~ 40 hours a week, as stated in your contract.

Working twice as hard and killing your week end has no point and you won't get promoted. Don't bother.

seattle_spring 2 days ago 1 reply      
I would argue that the vast majority of software engineers in the US are actually overworked and make close to middle-class wages. It's really unfortunate seeing articles like this, because it reinforces everyone outside of tech's stereotypes that us engineers are lazy, overpaid slobs.
lettersdigits 2 days ago 1 reply      
> "Most of my friends at Google work four hours a day. They are senior engineers and don't work hard. They know the Google system, know when to kick into gear. They are engineers, so they optimized the performance cycles of their own jobs," one engineer described.

Is this really prevalent at Google?


zw123456 2 days ago 2 replies      
Yes, but the Wall Street CEO's are sooooo hard working. Give me a break. I once worked for a company whose CEO completely ran the thing into the ground and eventually got fired but got paid millions anyhow. I jokingly said I could have ruined the company for half what they paid that meathead.

I think the real scandal is the ridiculous amounts of money CEO's get paid for doing nothing in a lot of cases. The money these "high paid engineers" are getting is peanuts compared to the sums these CEO's are getting.

Sorry for the rant, but it just stuck in my craw a little.

NTDF9 2 days ago 0 replies      
Reminds me of the common saying,

"The hardest thing about working at Google was the job interview to get the job in the first place."

0xbear 2 days ago 1 reply      
This actually dates back decades. DEC invented "no output division": a team comprised of senior but bored people who would be unfashionable and dangerous to kick out. You give them some bullshit bling project and segregate them well from people who are actually doing meaningful work so that they don't get in the way. It's better if they do something meaningless for you than something meaningful for a competitor.
natch 2 days ago 0 replies      
I've posted before that I suspect this is the trouble with a lot of Google services that don't always seem to get the love they deserve. People who are well on their way to vesting just aren't hungry anymore, and can't be bothered to care. Not limited to Google of course.
spraak 2 days ago 0 replies      
Well, at my current position I am certainly resting, but not vesting, so I'm torn about what to do. On one hand, I have lots of time to do my own projects, but instead I've been very lazy and mostly read HN :/ and on the other hand, I am at the low end of the pay spectrum for my experience. I'd rather be paid more, but I don't want to have to work harder for it than I am now.

Wow, I am silly.

Edit: I mean that the company has not IPO'd and I don't own any stock/equity as a remote contractor.

hendzen 2 days ago 0 replies      
I wonder how much of this is due to non-voting shares being sold to the public that prevent an activist investor from being able to push the board to trim the fat?
jokoon 2 days ago 2 replies      
I wonder what kind of work they do that makes them so unique and indispensable. I also heard stories about engineers who did not share how their code worked, so that the company would not risk firing them.
Swizec 2 days ago 3 replies      
> she had been killing herself to make it more successful and protect her people from losing their jobs over it.

> As tired as she was, she couldn't just quit this job. She owed a big chunk of money in taxes thanks to that stock and needed her salary to pay those taxes.

> after getting violently ill at the thought of going to work

Burned out and trapped by debt. Not a great place to be even with the $1mm/year compensation. Most of which is illiquid I assume.

cylinder 2 days ago 2 replies      
All's good when the stock price is inflated. The reckoning (cost cutting) will come.
rb808 2 days ago 1 reply      
Google is sounding more like IBM every year.
master_yoda_1 2 days ago 0 replies      
I was at a startup (which pay high salary) and for 6 month I don't have work. I thought its a red flag and I decided to leave. 4 month after that the startup closed down the office because it did not get further funding.

So yes salary was good, benefits where good, there was no work, but "there is no free lunch".

kelvin0 2 days ago 0 replies      
Legendary slacker story:http://www.cnn.com/2013/01/17/business/us-outsource-job-chin...

Don't work hard, work smart :)

peterburkimsher 2 days ago 0 replies      
Is it unethical to keep a chair warm when my boss didn't give me new tasks to do?

For other areas of life (immigration), I need to get more years of continuous relevant work experience.

I come to an office every day, but my boss just doesn't have enough to keep me busy. My job title is "Project Engineer", which is vague enough to cover everything from DLL debugging to Node.JS programming to network monitoring to evaluating Advanced Planning systems. The latest task is to do some online course in machine learning, even though he didn't specify how the company will need it.

On bad days, I feel useless. But I reconcile the situation to myself by saying it's basically a "basic income" (the salary is not high; the minimum that people on my visa can have). I could think about changing after I have the years of work experience, but years just come with patience, not with productivity. I feel like my situation isn't "fair" because my friends are so much more stressed, but I need the years, not the results.

I also do a lot of side projects and post them online (e.g. learning Chinese - http://pingtype.github.io ), but my contract and visa specifically state that I can't have any other paid work. So all my projects must be free and open source.

If another rest-and-vest person wants to comfort their conscience, I suggest reading more about Basic Income theories.

rebootthesystem 2 days ago 0 replies      
I've seen a different kind of behavior. It consists of taking a project that could be done in six months and stretching it over two to five years. And, yes, it looks pathetic and absolutely ridiculous when viewed with the eyes of a "get shit done every day" entrepreneur, yet it seems that in some of these environments this can become acceptable behavior in some strange-as-fuck way. I think it's soul-sucking depressing.
shoefly 2 days ago 2 replies      
This happened to me once. I ended up coding my own personal projects at the "host" company to pass the time. Otherwise, I would have gone nuts.
jayd16 2 days ago 0 replies      
So...the age old concept of "Fuck You" Money?

This is more an indictment of the work hours/pressure on average engineer.

freyir 2 days ago 4 replies      
> "They are really good engineers, really indispensable. And then they start to pull 9-5 days"

Worthless slackers.

jaequery 2 days ago 0 replies      
I think this happens to EVERY companies out there. The higher up you go, there are just less to do since all you are doing is delegating your jobs.

It becomes a problem though when problems do arise and you coasted for so long that you have no idea what is going on and where.

banku_brougham 2 days ago 1 reply      
I notice ther was no mention of amazon, but that is a large tech company with 10s of thousands of engineers. Anyone have an amzn rest and vest story?
nodnyl 2 days ago 0 replies      
What strikes me about this as someone working outside all this, is how ridiculously profitable advertising is!. Its kind of weird that it is so much more profitable to show people pictures of things they could spend their money on, than actually taking their money.
maxxxxx 2 days ago 0 replies      
I think it's not only engineers. I know several people who are either corporate lawyers or other long-time managers who pretty much go to meetings the whole day because they have nothing real to do. They all are pulling good money but feeling like they are not doing much seems to take a psychological toll.
adamnemecek 2 days ago 2 replies      
So what exactly does indispensable engineer mean? How many people making more than 1M a year are at each of these companies?
ComodoHacker 2 days ago 0 replies      
Well, this highly connotates with https://news.ycombinator.com/item?id=14945045
feelin_googley 2 days ago 0 replies      
"It's a defensive measure."

"That's Microsoft Research's whole model."


p0nce 2 days ago 0 replies      
Sad to see tech giants not recapturing carbon with their unlimited money.
k__ 2 days ago 3 replies      
Aren't companies like McKinsey more efficient?

They have the rule that you either get better and a promotion or you will be fired.

So they only keep people that improve every year or they get new people.

JohnJamesRambo 2 days ago 2 replies      
Why are programmers called engineers these days? An engineer has an engineering degree and does something completely different than computer programming.
loeg 2 days ago 0 replies      
Where do I sign up for that job?
joejerryronnie 2 days ago 2 replies      
Or, as a recently bought out/retired colleague remarked, "My job now consists of being home all day and trying not to piss off my wife" - a much harder endeavor than any engineering project I can think of.
A checklist of marketing ideas for side projects sideprojectchecklist.com
600 points by karlhughes  2 days ago   58 comments top 23
FHMS 2 days ago 4 replies      
"Cold call ~20 people who might be good customers."

IMHO this is probably the first! thing you should do. To my own surprise people will give you money for your service even if you don't have a websites (and name, logo, slogan or anything else), and you send your 'product' via email.

And 3 out of 4 ideas don't survive these 20 calls - so you'll save a lot of time if you sell first and build later.

Emotionally that's not easy - of course - but it's what you will be doing all day anyway if it goes well, so why not start early?

mhoad 2 days ago 1 reply      
I do a lot of this stuff professionally so I was interested to take a look at an "outsiders" approach to this. Honestly, you've done an incredibly good job on this. Great suggestions!
albertgoeswoof 2 days ago 0 replies      
There are so many things to do here- on top of actually building the side project itself. It's probably worth reading and understanding the benefits of each activity and then prioritising the ones that work for your business.

Just be careful not to deprioritize items because they're outside your comfort zone.

tonyedgecombe 2 days ago 2 replies      
Don't forget to create the product as well.
cstpdk 2 days ago 2 replies      
Great list! Might there be a market / an interest in a dashboard + chatbot for taking care of these things? Keeping track of milestones, doing the trivial signups an presenting how the different initiatives are performing and can be tuned?
OjelaFunderbunk 1 day ago 0 replies      
This is a great start! I noticed that 90% deals with MARCOM strategy with a bit of SWOT thrown-in too.

As someone who is responsible for bringing products to market on a daily basis, there is a considerable amount of strat being omitted which will better prepare you.

Pricing. Determine where you want to position your product in the market. Identify the competitors that compete on price vs the one's that compete on quality. Determine where you want to position your products and which competitors to go after. This should help you determine which customers you want to market towards as well. It also helps you determine the effort you can apply to each channel. [Spend more time on the difficult customer to obtain]. Michael Jordan didn't become Michael Jordan by practicing his dunk. He purposely worked on the weakest part of his game to make him the greatest.

vram22 2 days ago 1 reply      
I like the way you have gone about this - very systematic and seems somewhat comprehensive (though of course more points may be added over time). Thanks for making it.
Roedou 2 days ago 1 reply      
Great resource. The one name in there I didn't recognize was '7search'; I clicked through, but it seems to have shut down within the last week, so you can probably remove that.
j_s 2 days ago 1 reply      
Do you have time to explain your choices and/or process as you used a static website generator + theme for this project's web site when it appears to be optimized for blogging?

Right now I'm looking at Hugo + Kube but there are so many not-normal things to use (Hugo + Github Pages all in source control but publishing different branches / folders, Kube using not-normal-Hugo stuff since it's got non-blog stuff, etc.) Is there a one-stop zero-to-hero guide anywhere for this kind of thing?

deerpig 2 days ago 1 reply      
This is brilliant, but it's in markdown. I've converted it to emacs orgmode so I can use org checklists and integrate into my todo list.


davkap92 2 days ago 1 reply      
Nice idea thanks! One suggestion maybe add some sort of priority marking/ time spent for each one, as I would say some are definitely higher priority than others
vitomd 2 days ago 0 replies      
Very well done. I really like checklists, I use them as my todo list, project organizer and general stuff. So I created Sorter to help me: is a webapp to organize ideas, tasks and information using bullet points and hashtags. Its open source if you want to check https://github.com/vitogit/sorter
huhtenberg 2 days ago 3 replies      
> Free Promotional Channels / Write and distribute a Press Release.

For smaller projects this does not work at all. All it does is it attracts spam from various Indian "wire services" and that's it.

> Paid Promotional Channels / StumbleUpon

Only if you want to see how a 100% bounce rate with sub-second page stays looks like. Absolutely useless otherwise, although it is very attractively priced.

pgeorgep 1 day ago 0 replies      
Wow, love it! Has anyone put this on PH yet?

Another one to add would be a LinkedIn bot for auto profile viewing. It's a pretty efficient way for people to see you viewed their profile, they view yours back, and check out your site/project.

icecoldrocks 2 days ago 1 reply      
Here's a shared workflowy of the checklist for anyone who wants to use it:https://workflowy.com/s/FSjJ.Z6V7qfO5CD
archon810 1 day ago 0 replies      
You should probably describe what Triberr is next to its mention, because their site is absolutely useless (at least on mobile).
nathan_f77 2 days ago 1 reply      
This is really, really helpful. I will try to do all of these things for the project I'm currently working on. But man, cold calling 20 people is going to be so much harder than actually building the thing.
ktian00 2 days ago 0 replies      
Very comprehensive in detailing practical options for each step. Thank you!
orblivion 2 days ago 1 reply      
People hitting F5 on HN/Show/New is a great place to find people interested in promoting their side projects. Well played :-) (and thanks)
patrickbolle 2 days ago 0 replies      
Thank you!! Launched a few weeks ago but still very useful.
pryelluw 2 days ago 1 reply      
I did not see under which license you released the downloadable content. Mind sharing it here?

If you havent licensed it, one from creative commons might fit.

placebo 2 days ago 1 reply      
That's neat, thanks.
mezod 2 days ago 0 replies      
very timely
Cheap Beijing Flights With a Dangerous Catch seat31b.com
597 points by msh  3 days ago   248 comments top 46
greenyoda 3 days ago 14 replies      
Companies like this have been around for years, and whenever I read about them I'm surprised that people would risk going to prison (or worse) just to get a discounted air fare. I wouldn't even carry a package for a friend unless I could see exactly what was inside, let alone carry a suitcase full of unknown stuff for a total stranger.

Also, an inevitable question when going through security checkpoints in some countries is "did you pack your own luggage". I assume that if you answer "no", you'd be subjected to a very thorough search for bombs or contraband.

Calling a company "Airmule" seems to be a particularly bad choice, since the term "mule" is commonly used to denote a person who carries smuggled drugs (sometimes concealed inside their body).

Someone 3 days ago 3 replies      

"Please note that, as stated above, the site, application and services are intended to be used to facilitate travelers and senders connecting and arranging item transportation directly with each other. Airmule cannot and does not control the content contained in any package and the condition, legality or suitability of any items and luggage. Airmule strongly advises each traveler to inspect each item carefully. If a traveler does suspects an item is illicit, do not transport and contact airmule. Airmule is not responsible for and disclaims any and all liability related to any and all available transportation. Accordingly, any inquiries will be made or accepted at the members own risk."

I don't know how long that has been there, but it is clear. They are brokers, but don't accept any liability.

wjnc 3 days ago 2 replies      
I cringe when I read such Trumpian tweets from a founder. The journalist reached out a few times and the response was unclear. So he writes his piece with a pretty clear warning to future customers. Get your PR and compliance straight if you want to avoid such pieces, don't complain afterwards while calling names.

Caveat emptor. Just those responses are a red flag, if the subject is legal risk surrounding smuggling to PRC. They don't want you to know.

jimjimjim 3 days ago 4 replies      
If you can't answer a yes no question without wishy-washy flim-flam pr doublespeak then you don't get to complain that an article doesn't have facts.

and while i'm ranting. what is with founders presenting them selves as "bro's at the bar"?If the founders had bios that looked like they were from upper management at ibm i might be more likely to use their service.

kaishiro 3 days ago 2 replies      
I find responses like those given from the co-founder here infuriating for some reason - far more so than I realistically should. I've always valued transparency when it comes to business, so when I see people dancing around straight answers and then lashing out when people take issue with said responses it just seems so remarkably childish.
zbjornson 3 days ago 2 replies      
I don't know anything about airmule's operation or China's customs/security, but the on-board courier industry is a legitimate one that has regulations and procedures that it's not clear the author of this article is aware of.

> "We have found contraband in [courier] shipments," says U.S. Customs official Bob Fischler, "but percentage-wise it is infinitisemal. And in any seizure we made, it was obvious that the on-board courier had nothing to do with it." In fact, at New York' JFK and at London's Heathrow airport, because of the sheer volume of courier shipments, all courier pouches go to a central location for clearance. The courier is typically dismissed before customs physically inspects the shipments.

- From Air Courier Bargains by Kelly Monaghan.

inertial 3 days ago 1 reply      
Quite a few companies operate in this space. This business idea & its risks have been discussed on HN more than once. I'm surprised that some of these are still around. A likely pivot for these could be to carry specific goods where there are no "dangerous" side effects e.g. importing smartphones, laptops etc. Although they still are not exactly legal.

- https://grabr.io/en/

- http://www.entrusters.com/

- https://backpackbang.com/home

- https://www.piggybee.com/en/

- https://worldcraze.com/

- http://www.canubring.com/

- https://www.manyship.com/

jstoja 3 days ago 1 reply      
Founder of a startup, having a major article killing your company and "don't have time on a Saturday with my family to engage".

I understand that family is important, but isn't a situation like this so important that you - at least - replace some time next week by 2hours now to answer to this article?!

Edit: made me think about this xkcd https://xkcd.com/386/

Sapph 2 days ago 3 replies      
There's another company that lets travelers subsidize their flight ticket / earn money for delivering US products to their destination:


Key difference is:

You buy the products locals ordered (locals pay for item + delivery fee upfront into escrow) so there's no risk of a third party hiding drugs or illegal materials.

marcosscriven 3 days ago 0 replies      
I can just see it at the airport. "Did you pack your bags yourself?". I guess at 40 I'm not the target audience for this, but I'd be worried a younger person trying to save money might end up paying a high price.The name 'airmule' doesn't do it any favours either.
blisterpeanuts 3 days ago 1 reply      
When I was studying in Taiwan in the early 80s, the island's high tariffs motivated travelers to carry suitcases full of stuff--Walkmans, cameras, Italian shoes, etc. Contact a guy in Hong Kong, he gives you a bag, a guy in Taipei picks it up and gives you NT5000, enough to pay for your ticket.

I never got up the courage to try it myself, but friends did. My girlfriend did it once.

Looking back, I realize how exceedingly stupid this was. Had there been heroin inside that camera, you were going to prison for the rest of your life. They didn't (and still don't) screw around.

I heard all sorts of stories. An Australian backpacker was caught at Korean customs with 50 Rolex watches stuffed in his shirt. He was sent up for ten years. Numerous young Americans and Europeans busted for drug smuggling were rotting in prison in Taiwan, Korea, and Japan. At the time, with these countries technically allied with us against Red China, execution was not a politically feasible alternative.

This Mule thing is just another respin of an old practice. Best to avoid.

Animats 3 days ago 3 replies      
It's significant that the business is about shipments from the US to China. That seems to be hard. Getting stuff shipped from China to the US seems to be ridiculously easy and fast. You can order stuff off Alibaba and get fast delivery via China Packet, which is a postal service with really good rates for China to the US. Delivery in the US is via the USPS.The other direction is much more expensive and slower.

The US needs to renegotiate postal rates with China. China is still getting the "developing country" discount from the USPS.

nebabyte 3 days ago 0 replies      
> Plus, you really have to love the founders of this company. I mean, as a startup founder myself, Im rooting for them. One is a hardcore gamer, the other is a former backup dancer for Gucci Mane, and the third loves beer more than you do. Im not making this upthis is what they say about themselves on their Web page

This guy clearly doesn't get it. Your startup page is where you post a phip relatable quirky attribute, whereas your actual qualifications go in single-phrase sentences on your twitter bio and after your name on quora answers

/s, hopefully obviously

chx 3 days ago 0 replies      
It was not so long ago that the CBP stopped a business courier off a flight from Guatemala who happened to carry nine pounds of heroin. Because he was a courier, he was not criminally charged, nonetheless he was barred from entry and banned for at least five years. And that's the USA, not China.


skrause 3 days ago 0 replies      
Before clicking on the article I thought that the dangerous catch was that you have to sit in seat 31B and wondered why. The article's title and site name should really be differentiated better in HN's title.
jfoutz 3 days ago 2 replies      
This makes me super curious about the pre flight baggage controls. How do they handle someone who picks up the extra bag, but winds up not taking the flight?

With baby formula, i'd just return the bag and apologize, eating the $99. Heroin on the other hand, i could probably move at a steep discount. $1k or so, not worth the risk. $10 or $20k? hmm. The bag needs to be worth at least $1000 in the target country, just to break even.

it seems like pretending to be a stoner, and setting up enough to buy a plane ticket could get you a lot of money for $99. Fake id and a prepaid credit card aren't that hard to come by. It's not like the ID needs to pass TSA inspection, as you're not taking the flight.

Seems like a very risky business. If your customers are willing to be pasties, it'll be ok. but just a couple of sharks completely change the risk profile. Doing stuff that precludes government enforcement of contracts is just crazy crazy risky.

BayesStreet 3 days ago 1 reply      
On their website they state "Airmule then manually inspects and verifies each item prior to packaging for a traveler."but I doubt this company that started last year has more experience finding contraband than law enforcement doing it their whole lives who have seen everything. Pretty asymmetric risk profile, save a couple hundred bucks for potentially your life.
smsm42 3 days ago 1 reply      
Looks quite shady, especially given we're talking about China. Anything being wrong with the package - not even drugs - that'd be insanely bad - but I'm sure there are many other things which require special papers to get into China, or are prohibited, and if something is wrong, it's the courier's ass on the line. I don't see how it could be worth the risk of being imprisoned in China. I mean it's one thing to be in a "gray area" as an American in the US, with all legal protections and ACLU and so on, and another thing doing the same in China...

And I wonder what TSA thinks about people transporting things that they have little idea about in their luggage?

csomar 3 days ago 0 replies      
The founder is lame and as /u/wjnc mentioned he has a trump-like behavior. The question is very simple: If there is drugs in the shipment, does the traveller get a FREE pass?

The article is lengthy and kind of make this question vague. In my understanding it is a single question: Who bares the responsibility?

Well, it is you the poor traveller. There is no way in hell you can accept such a deal even if you are flying for Free. In fact, if you are, ask yourself the question: Do free meals really exist?

reuven 2 days ago 0 replies      
I'm glad that the author of this travel blog is warning people against using Airmule. Someone is going to get in a heckuva lot of trouble.

First: I can only imagine, when checking my bags, getting the question, "Did anyone give you anything to bring on the flight?" and answering, "Yes, my entire 2nd bag belongs to someone else who is paying for half of my ticket."

That alone would be enough to give you extra-special scrutiny when checking in.

But let's assume that you get through security, go on your flight, and arrive. I've traveled to China many times, and have thus put my bag through the customs/airport scanner many times. If they find anything illegal -- and in China, that can mean all sorts of stuff -- you are in Big Trouble. I haven't ever seen anyone pulled aside when going through customs in China, but I don't envy them.

And sure, Airmule can say that they've inspected things, and that this is safe and fun, etc. Just try telling the Chinese customs officials that the drugs don't belong to you, but rather to a startup in Silicon Valley. I'm sure they'll be very attentive.

Airmule's site attempts to calm potential couriers' nerves by saying, "Read this Wikitravel article." (Reference: http://wikitravel.org/en/Air_courier) However, the article says, very clearly:

> You need to be very careful about the legitimacy of the jobs you take. The last thing you want is to be caught > transporting contraband (or worse) on a plane. A good way to avoid this is to use an agent (usually a > representative of the service you are working for), who will take you through customs and clear the contents.> Always check the reputation of the courier company before booking. None which are reliable and legitimate > would ever try to ship anything illegal.

Airmule doesn't promise to have an agent on the arrival side. They do promise that they'll "walk you through" things, but that's very different from physically being there in China and claiming the luggage and any responsibility for it.

The idea is a good one in theory, but as executed, it's half baked -- and might lead to executions of a more literal sort, if people aren't careful.

bogomipz 2 days ago 0 replies      
From their FAQ:

"We don't just ship any item that comes through our front door. Airmule only partners with TSA certified shipping companies.

"Under their Certified Cargo Screening Program, the TSA certifies cargo screening facilities throughout the United States to screen cargo prior to providing it to airlines for shipment on passenger flights." [1]

The TSA however does not search for drugs however from the TSA's site:

"TSA security officers do not search for marijuana or other drugs."[2]

[1] https://airmule.zendesk.com/hc/en-us/articles/115005116068--...

[2] https://apps.tsa.dhs.gov/mytsa/cib_results.aspx?search=marij...

zupa-hu 3 days ago 0 replies      
So, they claim they can have the cake and eat it too - as in, ship the bag as non-personal carrier stuff to avoid prison, and ship it as personal non-carrier stuff to pass customs. Bold.
grecy 3 days ago 0 replies      
I frequently fly international with zero checked bags and about 5lbs of carry on.

I would love to be able to do so for $99 if someone wants to on-sell my checked allowance.

After reading the article I see the pitfalls I had never thought of, and would obviously want some extremely, extremely clear legalities to make it very clear the bags are not mine, and I'm not bringing them into any country.

No, I did not pack them and, no, I am not bringing them into your country.

micah94 3 days ago 0 replies      
Wait...let me get this straight: You carry a package on a plane to China given to you by someone you don't even know? This is a joke, right?
bisRepetita 3 days ago 1 reply      
Real life example of a Guatemalan OBC unknowingly bringing heroin into the US: he was deemed not responsible quickly, and got expelled right away with no right to come back for 5 years.


fencepost 2 days ago 0 replies      
I haven't heard about air couriers for years, but I don't travel much. That said, I was under the impression that for basically all of them the package being couriered was never in the possession of the traveler but was instead packaged as freight and was delivered to the airline as such and handled as such at the receiving end. If the "courier" in question wanted to drop it off or pick it up themselves they'd still have to go to the appropriate air freight terminal.

If these folks are providing packages to travelers to be checked directly by the traveler then they're idiots and so is anyone who takes them up on the offer. If not for the defensive tweets, etc. I'd feel that (as someone else noted) this must be a satire of the 'gig' economy.

skinnymuch 2 days ago 0 replies      
Soylent being tongue in cheek with their name is one thing. But Airmule? Why would they want to associate themselves with the most common border crossing association with mules - drug mules? Besides the whole arrangement seeming bad, the name choice is horrible.
smegel 3 days ago 0 replies      
"We'll pay you to take this bag on the airplane for us.".


asdfologist 3 days ago 0 replies      
Risk of a death penalty for accidentally smuggling heroin? They couldn't pay me to take this flight.
overcast 2 days ago 0 replies      
I don't carry checked bags for the specific reason that I don't want to deal with checked bags. I guess this is for people who want to be as cheap as possible, while simultaneously being as inconvenienced as possible.
icbm504 3 days ago 0 replies      
I like the idea but in the world we live in (post 9/11), it is a major security violation.
chrischen 3 days ago 0 replies      
Why not just specifically operate in the market of importing baby formula in China.

You can easily subsidize a flight ticket with a checked luggage full of baby formula and for extra security the mule can go buy the formula him or herself.

Karliss 3 days ago 0 replies      
If airplane companies take into account that certain percentage of passengers will not arrive and overbook flights wouldn't they also take into account that most passengers will have less than maximum allowed baggage?
srathi 2 days ago 0 replies      
From their FAQ

Airmule then manually inspects and verifies each item prior to packaging for a traveler. We also guarantee that 100% of the items shipped through our service are safe for travel on commercial aviation.

This is classic lawyer speech (notice the words "safe for travel on commercial aviation"). This just means that there are no harmful things to a plane, but they don't say anything about 'safe for customs'.

jliptzin 3 days ago 1 reply      
If someone came to me with this business idea I'd chuckle and say haha, good one. Of course assuming that it is some joke. It boggles my mind that apparently 3 (presumably reasonable?) people have decided to seriously pursue this idea. It's so bad on so many different levels it may actually be the worst business idea I have ever heard.
thedogeye 2 days ago 0 replies      
m3kw9 2 days ago 0 replies      
So if they missed a small pack of drugs that some slipped in there, some ones life is ruined
rpmcmurphy 3 days ago 0 replies      
Some startups just need to die in a fire. This is one of them. (Theranos is another one).
illuminati1911 3 days ago 2 replies      
While I agree with most of the article, I don't understand the part where the author is complaining about the bios of the founders/managers.

It's a startup, not old slow mega-corporation where making a joke will get you fired.

bberrry 3 days ago 5 replies      
Is 31B a reference I'm missing? It didn't come up in the article body.
baybal2 3 days ago 0 replies      
China United Airlines once did the route for CNY488 with carry on only
seannyang 2 days ago 0 replies      
Cheers everyone, this is Sean Yang, CEO and cofounder of Airmule, the air courier startup with an admittedly strange name that you might have read about earlier today.

Is Airmule legal?

On Board Couriers (OBC) have existed over decades. Their purpose is to service cargo that needs to be delivered in a timely manner. Its quite an expensive service, often servicing auto parts, airplane parts, important documents, passports, NASA parts, etc. Costs can vary from a few hundred to thousands of dollars. To become an OBC is simple, just call any OBC company and register on their list. In our case, we simply request you to list your trip. The OBC companies size doesnt have to be the same as Fedex, UPS or any airline cargo department. As long as they follow the TSAs IAC regulation, only ship items from a Known Shipper, and have the cargo secured in a locked area inaccessible by outsiders. At Airmule, we have a surveillance camera over cargo 24/7. (49 CFR 1544.228, 1546.213, 1548.15, 1548.16, and 1548.7.)

So yes, OBCs are totally legal, as well as all OBC companies. I see many ask the answer to Did you pack everything yourself question. The answer is to be honest with Airline company No, I didnt. Im an OBC and I have the manifest, and I know whats inside my luggage.

Every single traveler will receive a manifest prior to receiving Airmule shipments Airmule is 100% responsible for the items on the list.There are items we dont accept if they dont comply with our policy or the destination country customs policy:

For example:

1. Powdery items.2. Pills, medicine, prescriptions3. Unclear liquids (wine, etc.)4. Live plants5. Animal products (elephant teeth, fur items, etc.)6. Counterfeit items

If the Shipment is for commercial purposes, we will declare through the proper channel. We contract with a professional customs brokerage company for every single country we service. In that case, couriers simple leave the item at customs, a receipt will be issued by Customs, and Airmule will handle it onsite, couriers will be relieved from duty at that point. Airmule is not Smuggling. We do pay duty on behalf of shippers. Our shipping policy is very restrictive with Senders responsible for all duty fees.

For those who likes the name, we cant do anything about it. Regardless of what we are called; it sounds like you dont want to work with us. We just hope one day, when you need something urgently, that Airmule is a better and more affordable option to help with delivery, and saves your day.

Airmule has been running for almost 2 years, weve helped thousands of travelers to see the world they never were able to see before. The deal we post is 100% authentic, but a very limited offer. In appreciation of your time to read this, wed like to give a bonus for $100 if you use coupon ISupportAirmule when list your trip. If you still have concerns, please do not hesitate to contact me directly at sean@airmule.com

gweinberg 2 days ago 0 replies      
What are the odds the whole thing is an elaborate joke?
skrebbel 2 days ago 0 replies      
Could this be performance art?
exabrial 3 days ago 0 replies      
This is 100% illegal no doubt
erikrothoff 3 days ago 4 replies      
Besides the quite unnecessary personal attack on the founders ("One is a hardcore gamer, the other is a former backup dancer for Gucci Mane, and the third loves beer more than you do. Im not making this upthis is what they say about themselves on their Web page") I feel the writer did nothing to back up his claim. The "clear as mud" answer from the founder "same as all OBCs" was a really weird thing to leave up to interpretation of the reader. Nowhere did the writer enlighten me about the actual rights of an On Board Courier. I found this article really lacking in substance, sorry.
Linux Load Averages: Solving the Mystery brendangregg.com
563 points by dmit  22 hours ago   76 comments top 26
ChuckMcM 20 hours ago 2 replies      
Awesome analysis, I have added it to my favorites list. Around 1990 or so when I was in the kernel group at Sun and a team had just embarked on the multi-processor kernel work that would later result in the 'interrupts as threads'[1] paper. During that time there was an epic thread on email which was something like "What the F*ck does load average mean on an MP system?" (no doubt I have a copy on an unreadable quarter inch tape somewhere :-(). If it helps, the exact same pivot point was identified, which is this, does 'load average' mean the load on the CPU or the load on the system. While there were supporters in the 'system' camp the traditionalists carried the day with "We can't change the definition on existing customers, all of their shell scripts would break!" or something to that effect. Basically, the response was if we were to change it, we would have to call it something different to maintain a commitment to the principle of least surprise. This has never been an issue for Linux :-).

As a "systems" guy I am always interested in how balanced the system is, which is to say that I am always trying to figure out what the slowest part of my system is and insuring that it is within some small epsilon of the other parts. If you do that, then system load is linear with workload almost regardless of task composition. So disk heavy processes load the "system" as much as "compute heavy" processes and "memory heavy" or "network heavy." In an imaginary world you could decompose a system into 'resource units' and then optimize it for a particular workload.

[1] http://dl.acm.org/citation.cfm?id=202217

siebenmann 20 hours ago 3 replies      
This is great work in general and excellent historical research.

As an additional historical note: in Unix, load averages were introduced in 3BSD, and at that time they included processes in disk IO wait and other theoretically short-term waits that weren't interruptible. This definition was carried through the BSD series and onward into Unixes derived from them, such as the initial versions of SunOS and Ultrix. At some point (perhaps SunOS 3 to SunOS 4, perhaps later), the SunOS/Solaris definition changed to be purely runable processes.

(I'm not sure what System V derived Unixes such as Irix, HP-UX, and so on did, and their kernel source is not readily available online for spelunking.)

As of early 2016 when I last looked at this, the situation on FreeBSD, OpenBSD, and NetBSD was somewhat tangled. FreeBSD load average only included runable processes, but NetBSD and OpenBSD counted some sleeping or waiting processes as well.

simonjgreen 7 hours ago 0 replies      
Under Better Metrics the author discusses ways of drilling down to find the source of a high load average. I feel like this section should mention `atop`, which is imo a really underrated single-pane-of-glass view into everything your system is doing, now and historically.

If you haven't tried `atop`, give it a go.

This historical analysis in this article though is great, because while Load Average has been an oft discussed and we'll understood topic for a long time, the decisions that got us there are not.

Twirrim 10 hours ago 1 reply      
Several years back the company I worked for ended up picking up some work for a client. Every quarter we'd download a huge trove of TIFFs from some source, and then do some image conversion work before shipping transferring them to the customer's infrastructure.

There was a java application that powered the logic side of things, calling out to ImageMagick to do the actual processing and conversion. For whatever reason, after careful benchmarking we settled on a java thread count that happened to get us the peak throughput, but also caused system load average to hit around 400 and keep steady at around that level.

The day that happened, and I could show that no application on the server took a performance hit, was the day that I finally persuaded my boss that load average is an interesting stat, but it's not the be-all and end-all, and that a high load average doesn't necessarily correlate to an actual problem.

sreque 19 hours ago 1 reply      
One source of high load average spikes that I've seen in my job is when a process crashes and generates a core dump. While the core dump is being written, all threads in the process are in the TASK_UNINTERRUPTIBLE state even though they are doing absolutely nothing, and as such they all count towards the load average as if they were spinning on on a CPU core. If the total virtual memory of the process is large, say in the multi-GB range, coredumping can take on the order of a minute, and Linux will report an unreasonably high load average if that process had a lot of running threads.

Things like the above scenario make me treat the load average metric with a lot of skepticism. I would much rather use other metrics to infer load.

saalweachter 18 hours ago 0 replies      
If it was bothering anyone else: yes, the parenthesis in the patch in the email are unbalanced, and the code was checked in as:

 if (*p && ((*p)->state == TASK_RUNNING || (*p)->state == TASK_UNINTERRUPTIBLE || (*p)->state == TASK_SWAPPING))

vfaronov 4 hours ago 0 replies      
Worth remembering that essentially the same issue exists at a lower level: the %Cpu number as shown by top includes not just the share of time spent actually executing your instructions, but also the share of time waiting on memory access.

As explained by the same author: http://www.brendangregg.com/blog/2017-05-09/cpu-utilization-...

mnw21cam 6 hours ago 0 replies      
Good article. However, it is missing the reason why load averages include tasks waiting for disc/swap.

One of the things that the load average is sometimes used for is to work out whether it is appropriate to start some more processes running on a system. For example, make has a "-l" option, which prevents more parallel jobs being run while the load is above a supplied number. When a system is thrashing due to insufficient RAM, then the load average will be high, and this option will appropriately prevent more tasks being started which would make the thrashing worse. If the load average was just based on CPU, then it would be low while thrashing, and using that make option could lead to complete system collapse.

solarengineer 8 hours ago 0 replies      
When I'd asked Brendan via Twitter for an article on Load Averages in Linux, I hadn't expected such a detailed response. I've worked on a few projects where I've had to show that even though the "load" on the Linux system was low, it was really the steal% and the iowait that were killing performance. I'm sure that from now on, so many system and support engineers will have a good article to reference. Thanks, Brendan!
Florin_Andrei 21 hours ago 2 replies      
> As a set of three, you can tell if load is increasing or decreasing

That could be accomplished with a set of two.

A set of three could in theory give you acceleration.

hathawsh 16 hours ago 0 replies      
This analysis cleared up a mystery for me. I've noticed that when a server app is under heavy load in Linux, the load average goes high if the bottleneck is the CPU or the disk, but the load average goes low if the bottleneck is network resources (like databases or microservice calls). I know why that happens, but it's very unintuitive and it confused me when I was new to Linux. I thought load average would measure the CPU load only. Now I know the historical reasons for measuring system load instead of CPU load.

I kind of like it the way it is since it's handy to be able to distinguish network load from CPU+disk load just by looking at the load average. However, since the load average includes other stuff as well, sometimes I still don't know what the load average really means.

sytringy05 12 hours ago 0 replies      
My company took over production support of an ESB from another company for a client a couple of years ago. The worker nodes had about 100 JVMs running on it and its resting Load Avg was around 30. This on a 2 CPU RHEL vm.

Out of morbid curiosity, I restarted one of the test servers and ran top. Load Avg was in the order of 2200 for about 3 hours.

The worst part was that the guys we took it over from didn't even think it was a problem.

ty_a 21 hours ago 3 replies      
Holy crap, Brendan Gregg's site went down. Proof he is human I guess?
ge96 21 hours ago 1 reply      
Why isn't there one for ram in i3? I read something about how it's hard to gauge ram usage despite htop displaying it as well as inxi in general on Windows you look at task manager there is memory usage.
mnarayan01 21 hours ago 0 replies      
Page swapping seems like it makes a lot of sense to include in the load average. Disk I/O seems like something more towards the opposite end of the spectrum, though TASK_KILLABLE (https://lwn.net/Articles/288056/) presumably mitigates this where used.
mobilethrow 20 hours ago 2 replies      
OT: what could cause a system to have a load of 1 when idle?

I have one (unimportant) Linux system that idles with a load of exactly 1. The issue persists through reboots. It is a KVM virtual machine and qemu confirms nothing is going on in the background.

Any ideas how to find out what's causing it?

fanf2 21 hours ago 2 replies      
I thought that including disk wait in the load average was a common Unix feature. Sadly I can't go spelunking through the archives right now, but it would be interesting to see what Solaris and BSD do, for comparison with systems a little bit closer to Linux than TENEX :-)
gciruelos 19 hours ago 0 replies      
there's a very good (and old) article about linux load averages here: http://www.linuxjournal.com/article/9001?page=0,0
caf 10 hours ago 1 reply      
brendan, you could consider adding an option to offcputime that merges all kworker stacks together, since they're really just separate workers in the same thread pool.
js2 18 hours ago 0 replies      
It's been years but I really remember that Solaris load avg used to similarly be affected by I/O, particularly NFS.
JaggerFoo 20 hours ago 0 replies      
Great article. Interesting, insightful, and actionable.


faragon 21 hours ago 0 replies      
It incredibly detailed, including references and historical investigation. Mind blowing. Kudos, Brendan Gregg.
Steeeve 10 hours ago 0 replies      
This is an incredible analysis! Well done!
shimon_e 9 hours ago 0 replies      
netdata is a good tool if you are looking for precise data on where the bottlenecks are on your server.
SoMisanthrope 19 hours ago 1 reply      
Brilliant. Time to patch it back to CPU loads.
Apple staffers reportedly rebelling against open office plan bizjournals.com
379 points by V99  18 hours ago   359 comments top 60
seorphates 17 hours ago 8 replies      
Working in open office plans is simply awful.

Personally I believe remote work, for any tech-enabled employer, makes the most sense. The impact on infrastructure by removing commuting alone could maybe help save the planet. And our collective sanity.

Wouldn't it be nice to have ISPs that can provide an infrastructure that could actually support that? I think so.

The hideous effects of cluster-fucking hundreds of thousands of people daily just needs to stop. Tech companies are guilty. They're huge and, humbly opined, are idiots for making it worse and not really needing to. Top that off with an open floor destination and.. damn, work is beat.

Joeri 4 hours ago 0 replies      
Apple has insisted in presentations to the city of Cupertino that the open floor plan designs are conducive to collaboration between teams, per Bloomberg. But the high-level executives, including Apple CEO Tim Cook, are exempt from this collaborative environment and have offices on the fourth floor of the new building.

See, this is exactly what's wrong with open plan offices in most places. If a CEO honestly believes open plan is better for collaboration, then they need to eat their own dog food. That CEO needs to be sitting right in the middle of things. If they find they can't get anything done as a consequence of the collaboration they are in the right place to take action to fix that. And if they are able to achieve productive outcomes, they are also in the right place to argue against people who say it's not possible. Letting upper management avoid all the downsides of the open plan layout causes problems with it to fester and will bring overall worker satisfaction and productivity down. In short, it is bad management to treat management in a special way.

inetknght 16 hours ago 10 replies      
My company only has offices for upper management. Everyone else is at a table. Tables are arranged in groups of four.

Now, I get it, some people like open office environments. Good for them.

Me? Well, I've told many coworkers that I can't work from home because I wouldn't work from home. There are too many distractions at home, so I need to be at the office to be productive.

But this open office?

There are days where I am convinced I would do more work, be more productive, and feel more satisfied if I worked from home.

I went and bought some noise cancelling headphones. They help, but definitely not enough. My table is by the main door. With a room of 40+ engineers, there's constant distracting traffic. Some people make snide comments about my choice of operating system, keyboard, language, editor, typing noise, attire, whatever. Or to chat about the games that I missed last night, something happened at the not-company-sponsored-happy-hour that I didn't get the invite to, or something about lunch that, you know, you should have been there and if only you wouldn't leave the office for lunch. Or about how your racing car is in for the shop because, well, actually I don't even care why. It's just in the shop (I know! you told me!) and you expect me to care about car parts too, and shame on me for not knowing the difference between a maserati and a miata.

On the other hand, any time I mention to my boss that I'd like at least a cubicle the response is "it's not going to happen". Thanks, boss! I'm glad you've got my productivity concerns on your plate. I'm glad they can just, you know, be heard. Not addressed, just heard. It's really helpful to be heard. All day. It's real helpful to hear everyone's discussions while I'm trying to do work.

Honestly, guys, if you like an open office environment, that's good for you. Not everyone wants one and not everyone works well in one.

loco5niner 17 hours ago 2 replies      
Hopefully, more and more companies experience backlash from this. It is a horrific mistake to add distracting elements to most programmers environments. Even worse, in my open office plan, they put our very loud finance group right next to us. Absolutely no thought of noise management was considered, except for putting in horrible "white noise" generators that set off my tinnitus Thankfully, my direct manager is understanding and let me turn off the one directly over my head. And by directly over my head, I mean about 4 feet.
nemo44x 17 hours ago 4 replies      
It all seems so backwards. Instead of having collaborative working spaces with private rooms for meetings, doesn't it make more sense to have private rooms for working and collaborative meeting spaces?
hkmurakami 17 hours ago 1 reply      
It's really kind of amazing to me how in 20 years we've gone from laughing at the cacophonous, claustrophobic, diseases-transmission-inducing, open office plans of other economic regions (ex: the traditional Japanese office http://1.bp.blogspot.com/-CadIFZ3h638/T7yGtzdxVDI/AAAAAAAABe..., or the Wall Street trading floor), to precisely emulating their layouts (with better superficial aesthetic design), inheriting both their economic efficiency and productivity inefficiencies.

I'll take a cube farm with 5 feet walls any day over an open office.

chmaynard 15 hours ago 2 replies      
I worked at Apple during the years when the company designed and built its first campus at 1 Infinite Loop in Cupertino. As I recall, Apple R&D employees were considered stakeholders and participated in the design of the interior spaces. Apple wisely decided to give each engineer a private office. There were open areas near the offices with comfortable furniture and whiteboards for engineers to meet and collaborate. I worked in one of these buildings from 2001-2007, and I can confirm that the work areas were beautifully designed and ideal for fostering productive work. It's sad to hear that Apple apparently abandoned this approach in its new campus.
aetherson 17 hours ago 1 reply      
I am fairly close to someone who works at Apple. His team is avoiding the new spaceship building. He mentioned wanting to keep his office, but that was just one part of several different complaints, including just "it turns out that the building isn't big enough for most of the people who work at the HQ in Cupertino," and "My team would probably have to split up in awkward ways because not everyone would be able to work in the spaceship (due to space constraints)."
nashashmi 17 hours ago 8 replies      
Man, I remember in college when we would be working long hours in the library on a computer lined up in a row of computers. Every one would be intensely working on what they needed to. Sometimes two would work together. This was especially true before presentations when we were trying to put our stuff together. It was neat. It was collaborative. It was fun. And we were happy.

Open floor plan is reminiscent of those days, but it isn't working. And I cannot figure out why. What's missing? Intensity? Work? Stress? Team building therapy? Or just trust? Whatever it is I hope we figure it out.

sidlls 17 hours ago 2 replies      
Open offices diminish workers to cattle status. Most work, even the kind many developers would not think of as being so, in tech companies requires thoughtfulness often and collaboration less often. I consider open office plans to be disrespectful and an indicator of second-class status.
minwcnt5 15 hours ago 0 replies      
Headphones are a poor solution to the noise problem in open offices. I find it uncomfortable to wear them for 8 hours at a time, and it means I can't overhear the conversations I do want to overhear. Sitting elsewhere only works if I have a task I can do on a laptop; for serious development work I need a lot of screen real estate. That solution also has the same problem as headphones where I might miss important conversations because I'm too busy hiding from noise created by people doing work completely unrelated to mine.

There's a pretty happy medium, 2-10 person offices (with 4-5 being the most common size) with glass walls. Google used to have a lot of these before completely open plans became en vogue, and it was very rare to hear complaints. They allow frequent interaction with your most common collaborators while blocking out conversations from distant teams. They reduce visual distraction while still allowing in lots of natural light and inviting conversation. Doors were usually left open, so it was pretty comfortable to walk into another office and start up a conversation.

With the giant, open, chicken-farm style floorplans, people feel too self-conscious about dozens of people overhearing to have small 2-3 person conversations near their desks, which means more formal meetings with all the associated overhead, and fewer impromptu questions like "hey does anyone know of a tool to do X?" And then you're still more distracted anyway due to all the typing, people walking by, large groups being loud when gathering to eat lunch or go to a meeting together or whatever.

I only see two advantages of completely open floors: slightly cheaper (glass offices can be made almost as dense, but not quite, and I guess the glass partitions aren't free), and better circulation to dissipate bad odors more quickly.

kevinburke 16 hours ago 1 reply      
One solution to this problem would be for Apple employees to form a union and collectively bargain for better working conditions. Probably just threatening to do this would lead to significant concessions.

Any Apple employees interested in this should contact Maciej Ceglowski on Signal at +1415-610-0231.

pimmen 8 hours ago 0 replies      
"But the high-level executives, including Apple CEO Tim Cook, are exempt from this collaborative environment and have offices on the fourth floor of the new building."

Because private offices offers control over your working environment; if you need to collaborate, use a conference room, if you need a quick discussion, call them up on Slack.

I'm not going to touch wether or not the CEO has earned the best working environment, but let's bring attention to the fact that the CEO is promoting less control over your working environment for his employees and claim open-office plans offers all kinds of benefits, while the C-level management chooses to opt out. Either that's very noble of them to sacrifice all the benefits of open-office, or they're being a bit disingenuous about why almost everyone else gets an open-office plan.

chank 17 hours ago 1 reply      
My company recently switched to an open floor plan. It's done nothing but increase distractions and office gossip. Everyone I know tries to get away from their desk as often as possible. Ducking into side rooms, attempting to work from home, and just plain using any excuse to escape the zoo.

Management loves open plans because it's the cheapest seating arrangement. They claim that it will increase collaboration while exempting themselves from having to deal with the environment. The truth is that just being able to see someone without walking over to their desk isn't going to magically make you communicate with them more or make your output higher. Some people like open floor plans but it's been my experience most people don't and just grin and bear it while slowly dying inside.

knorker 17 hours ago 1 reply      
I recently watched the movie Office Space.

Oh, such a wonderful working environment. To have the privacy and isolation from distractions and interruptions that a cubicle gives. What I wouldn't give to work in such a great office space.

borplk 15 hours ago 0 replies      
> open floor plan designs are conducive to collaboration between teams

This is just an overused cover-up story to avoid stating the real reasons which is cutting costs and monitoring employees.

They use "collaboration" so that you can't voice your opposition to it easily.

If you do that they will beat you with the "not a team player" and "not a culture fit" sticks.

Then in reality unhappy employees sit next to each other with noise cancelling headphones whose job has been unnecessarily harder than it already is because now a part of their mental focus and capacity is actively going towards ignoring distractions.

a3n 13 hours ago 1 reply      
In (almost) all open office environments, people above a certain level have private offices.


Why don't they want to be as productive and collaborative as their reports? Conference rooms and phone rooms are just as available to them as they are to the rest. They can probably even afford much better head phones than the rest.

I just don't see enough of a difference to justify it.

brudgers 16 hours ago 1 reply      
Good architecture does not come from curved glass and 1mm joints between materials. It comes from human habitability. Why build a building that makes people unhappy? It seems to miss the point.
skc 16 hours ago 0 replies      
Every article I've read about this building in the past has gone to great pains to point out the artistry, elegance and taste that was applied in building it.

I now find it highly amusing that at Apple, form over function won out yet again.

maxxxxx 17 hours ago 4 replies      
It seems a lot of managers live in that phantasy world where people do nothing but collaborate. Do they really think that code gets written that way?
nupertino 15 hours ago 0 replies      
I wonder if anyone will make a claim about necessary workplace accommodations under the Americans with Disabilities Act for ADD/ADHD. I already take medication which makes it almost OK for me to share an office - a recent change for me after 20 years. But I'm still freaked out by someone literally 3 feet away from me. My social anxiety and borderline asperger's really make me seize up until I can be alone in the late afternoon / evening.

When I had my own office, I was able to do things like coordinate health care, talk to my wife, and eventually the divorce lawyers, but with the knowledge that I could close my door and have privacy - now I have to escape to a staircase to have a private conversation.

Plus, I'm terribly annoying to be around. From my mechanical clicky keyboard to a desk overflowing with artifacts and fidgets of various ilk, sharing a workspace means subjecting everyone else to my idiosyncrasies, mumblings and offensive body oder.

1_2__4 14 hours ago 0 replies      
Every single company does this now and it's a fucking nightmare. They'll give you a million useless and stupid perks, but they won't give you a fucking place to actually do work. It's infuriating beyond words.
aphextron 16 hours ago 1 reply      
tarikjn 9 hours ago 0 replies      
I knew this was going to happen when I saw some of the office picture/renders a few months back.

Highly relevant article:http://timharford.com/2017/02/what_makes_the_perfect_office/

And for a bit of history about cubicles, their first incarnation was actually a developer's dream:https://www.wired.com/2014/04/how-offices-accidentally-becam...

LyalinDotCom 17 hours ago 0 replies      
from the story ... "Prominent Apple podcaster and blogger John Gruber passed along rumors that some high-level Apple staffers are unsatisfied with the companys open floor plan which has many company engineers working at long tables with co-workers, instead of in cubicles or offices."

wow, "long tables" for lots of devs to work at, what can go wrong right? and i thought Microsoft open space had its issues, this sounds much worse.

When do people focus again?

cordite 15 hours ago 0 replies      
I've worked in both, and both have their benefits. However, my evaluation of open office may be biased because we also use slack.

Regarding open office plans:Focus does suffer in an open layout. Creativity does suffer too. In the face of a fire in production, an open office creates a low friction environment for task distribution to handle it. A factor that makes up for this is that we can work from home one day a week. I find these times to allow me to be most creative for planning long term solutions. Occasional remote work is possible and effective thanks to several technologies including Slack.

Regarding individual or paired offices: focus is easy to accomplish, and it is easier to be creative. It can be quiet, but it sure feels lonely when my team members take 3-6 minutes to walk to. Unfortunately, meetings, ad-hoc visits, and email were the communication methods here. Remote work was near impossible and impractical with just email for peer involved processes. This was also in a very corp-legacy environment and my ability to make an impact was unsatisfying. So I feel my creativity was often wasted and unvalued.

Overall, I think I like what I have now, which is mostly open office, but still occasional time for individual creativity.

oddity 17 hours ago 2 replies      
Surely, they'll all realize that it takes courage to embrace the office layout of the future?

Jokes aside, this was a problem five years in the making, and as far as I can tell there was no secrecy about the plan. I'm surprised the complaints are only coming now.

Animats 17 hours ago 1 reply      
Bench seating, work tables and open cubicles.

Apple? Famous for not letting their developers talk to people outside the project?

mmanfrin 17 hours ago 1 reply      
I hope they win, open offices are the stupidest thing since cubicles.
jansho 16 hours ago 2 replies      
bipson 4 hours ago 1 reply      
Everybody arguing for open office plans and stating that they or "some people" thrive in such environments should finally come around to read Peopleware [1].

Although they might base some statements on assumptions I do not fully agree with all the time, and before reading I was had not decided if I was strictly for or against open office plans, their conclusion is spot on: open plans do not foster collaboration or communication. They may cause a constant buzz and seem productive, but nobody will be smart, creative or productive in that environment, compared to a silent, uninterrupted workplace.

All you multitaskers and procrastinators (including me): You are lying to yourself.

[1] https://www.amazon.com/Peopleware-Productive-Projects-Teams-...

booleandilemma 14 hours ago 2 replies      
My company has an open office plan and I feel like the area directly behind my chair was designated as the company's unofficial meeting space, but no one ever told me.
Bonge 7 hours ago 0 replies      
The open office arrangement works for some industries but for others, each company would need to evaluate how specialized their workforce is to choose a suitable arrangement.

I would prefer a glass office with the freedom to have it closed much of the day without any "judgement".

I am developer with an office now, but if i close the door I seem to be keeping people off, and not "social" or "accommodating". Leaving the door open exposes me to a lot of distractions (noise, visual-people walking etc, people just stopping by or spying on me) which are unhealthy and reduce my productivity. I have since learnt to ignore as much distractions as I can.

Working previously in an audit firm, the open space worked well, because there is constant collaboration with multiple audit colleagues and all tasks complement each other hence the need to constantly keep tabs. But in development,if I have my specs or requirements , I don't need to keep in touch unless when giving updates, requesting for a resource, asking fr help or something else that is really pressing.

meddlepal 11 hours ago 0 replies      
In the future when I do my next job hunt I'm giving serious extra consideration to any company with private offices or high wall cubes even if the comp is worse. Open office plans suck. It's time these companies start being penalized
jacobr 6 hours ago 0 replies      
We have screen walls (~2 meters) around teams of 2-4 people. I think 2-4 people is a decent compromise, you get friction-less cooperation with the people you work with but you don't get to overhear 20 different conversations about what someone did on their weekend.

I am still bothered by other people's conversations across the screen-walls though, and I prefer to not listen to music all day.

Our CEO is sympathetic to the issue with open offices and maybe if we move to another place it will have a different layout, but converting an open office to separate rooms is not that easy. Does anyone have any suggestions when management would actually accept taking measures to avoid the negatives of open office plans?

Are there 4m high screen walls, with doors..?

nottorp 5 hours ago 0 replies      
Hmm. In a field like software development, where the good people work wherever they want, this means they will lose their best staff.

Time to take a look at current desktop Linux again :)

siculars 9 hours ago 0 replies      
"Gruber continued, When he [Srouji] was shown the floor plans, he was more or less just 'F--- that, f--- you, f--- this, this is bulls---.' And they built his team their own building, off to the side on the campus My understanding is that that building was built because Srouji was like, 'F-- this, my team isn't working like this."

Ya, what that guy said ^^.

zitterbewegung 17 hours ago 1 reply      
I have heard that open office plans are justified by decreased cost. Does it make sense that they would build a 5 billion dollar office with open floor plans? I suppose given that large price tag there would be motivation to cut costs that way other than the fact that the open office plan as a fad still exists.
auggierose 14 hours ago 1 reply      
I guess there will be two classes of people at Apple. Those with their own office, and those without. Although it has its rough edges, I am becoming a big fan of Swift. I hope Chris Lattner has his own office.
jmull 15 hours ago 0 replies      
Open office spaces is something that will be laughed at in the future.
eecc 6 hours ago 0 replies      
As a freelancer I'm looking forward to the day my customers will accept that I'm just as effective (actually more so) sitting in my own shared office space located just a short commute from home rather than their own desk and office, alas a grueling journey away.

I guess they don't trust us children doing the work we are assigned and want to smell us sweating away at their stinky enterprise codebases

cylinder 16 hours ago 1 reply      
I'm sure Jony Ive's office is nice and quiet.
Someone 17 hours ago 0 replies      
IMO, it is all in the execution. Open offices can be terrible, but they also can be good, just like private offices.

There's a huge difference between a few dozen desks in a bare concrete hall without any dividers between desks or a few dozen desks in a room with sound-dampening dividers between desks and lots of sound proofing on the walls and ceilings.

I work in an open office, and barely hear it when people three meters away make a phone call.

deathanatos 9 hours ago 0 replies      
> Apple has insisted in presentations to the city of Cupertino that the open floor plan designs are conducive to collaboration between teams

Oh, please. A trip to Wikipedia would have told you that[1]:

> A systematic survey of research upon the effects of open-plan offices found [] high levels of noise, stress, conflict, high blood pressure and a high staff turnover. The noise level in open-plan offices greatly reduces productivity, which drops to one third relative to what it would be in quiet rooms.

> Open-plan offices have frequently been found to reduce the confidential or private conversations which employees engage in, and to reduce job satisfaction, concentration and performance, whilst increasing auditory and visual distractions.

Further, open office plans spread disease more readily[2]:

> elevated risks [for disease] were found among employees in all three traditional open-plan offices

An open office floor plan robs you of the ability to control the noise level in the environment. There is literally no way for me to convince enough of my coworkers that they should:

* Either take their phone with them, or silence it if leaving it at your desk.

* Stop having meetings in the aisle immediately next to my desk.

* If you're going to video conference in the meeting rooms, and have the other end at full volume, close the damn door. If you don't know enough about video conferencing to understand what a feedback loop is, and want to spend the first 10 minutes of the meeting generating them, close the damn door. Stop looking at me like I'm rude when I close the door for you.

Calling people out gets a typical "oh, sorry", but not an actual change in behavior.

Maybe it encourages me to talk to the team nearby. Maybe. But is it worth the losses? No. (The team next to me is sales. They're not bad people, but they are fairly noisy. (And I'm sure they'd say the same of us, in fact!))

[1]: https://en.wikipedia.org/wiki/Open_plan

[2]: http://www.tandfonline.com/doi/pdf/10.1080/00140139.2013.871...

icanhackit 16 hours ago 1 reply      
John Kullmann, who ported OSX from PPC to x86, was able to work from home. I wonder if it's set up so that when you need to get shit done you work from home and when you need to collaborate you visit the spaceship?

Or perhaps only the superstar engineers get to pull that kind of thing.

b34r 9 hours ago 0 replies      
God I hate open office layouts. The only job I've ever had with a proper cube was a marketing agency, and it was glorious.

My team recently got little flags for our desks that explicitly say "open for business" and "busy, come back later"... but even with those people still bother you!

MikeTLive 17 hours ago 0 replies      
they screwed up my award winning office design.common section down the middle with individual offices on opposite walls, large enough for pair programming, one end of the alcove has windows above shelves, other end has multimedia.offices to be used as small conference rooms and manager offices, without windows, are in a perpendicular central hallway. i designed this over 10 years ago and a virtually identical design was used for my company's buildout the following year. no one is assigned to the common central table. offices can have the door open or closed depending on the occupant need/desire to be heads down or passively participate with others. each alcove houses a functional team
chewrocca 12 hours ago 0 replies      
I worked at a place that had the floor to almost ceiling cubicles. It was nice and quiet. Then we went to the open floor plan. Then the Nerf guns arrived.
Cozumel 17 hours ago 1 reply      
Investing in everything but the people who work there.
dogruck 10 hours ago 0 replies      
When was the first "rebellion against an office plan"? Can anybody link a report from the early 1900s? The 80s?
mathattack 16 hours ago 1 reply      
I'm ok with an open office, but my job is to be interupted. It's less good for deep thinking jobs, or heavy phone call jobs. Our engineers don't seem to mind though, maybe they just plug in their headsets?

Joel Spolsky wrote about this [1] though his main citation is experience at Microsoft.

I do have to admit Open Floors are a shift for a company focused on secrecy.

[1] https://www.joelonsoftware.com/2008/12/29/the-new-fog-creek-...

signa11 8 hours ago 0 replies      
'peopleware' by demarco and lister, goes over this and lot of other issues in great detail. imho, it should be required reading for s/w managers at the very least.

edit-001 : added author info for the book.

jorts 12 hours ago 0 replies      
Putting on headphones and listening to white noise like rainfall blocks out just about everything for me. I don't like open floor plans due to noise when I'm not wearing headphones, but with them on it's fine. When I don't put my headphones on it allows me to engage with my team when I am not hyper focused on something.
romanovcode 4 hours ago 0 replies      
Well I would also be pissed if I had an office and then was told to give that up.
marze 16 hours ago 1 reply      
Apple is famous for pixel perfect prototyping, of numerous possible solutions, before choosing.

Did they prototype and test various office layouts? If not, why not?

pasbesoin 17 hours ago 0 replies      
Apple has the money to afford whatever it wants. If it's like any other place I've seen, I expect there's longstanding communication of one or another sort from high performers that they want distraction-free environments.

From what I've observed of such high performers, they are not anti-social nor anti-collaborative, nor are they "crippled" in either respect. Rather, many of them are the most capable in these areas, because they actually pay attention and focus on getting things done -- and done as well as time and resources allow.

The fact that Apple, like many workplaces I've observed, chooses to ignore this and push a paradigm that increases their stress and decreases their effectiveness and efficiency?

Well, as I learned in my own experience, over the years: This is just a fundamental level of dis-respect.

I don't know anything about Apple work internals, specifically; the last time I intersected with those peripherally was in the early '90's.

But when you blatantly disregard what employees tell you -- and in this case, "professional" employees who have a high degree of training and awareness about the tooling they need, including their work environments, to be most effective. Well, that's just disrespect.

And employers who persistently engage in such, deserve what they get. I hope -- because at some point, this counter-productive... "ideology" needs to die.

P.S. Those employees that want cubicles or open-space? Fine, give it to them. I don't want to dictate environment, either way.

Trust your employees to select what works best for them.

And measure the results. Objectively, not in the typical performance review ex post facto rationalization and justification.

In my own experience, top performers cautiously (politics) leapt at the chance to work from home and otherwise gain undistracted blocks of time to adequately focus on complex problems and program management.

Those who embraced the cycle of endless meetings, interruptions -- including environmental -- and superficially-addressed delegation? They faced the same problems, month after month, cycle after cycle.

synicalx 11 hours ago 1 reply      
We're going one step better at my work with our new/future office, "Activity Based Working". All the trappings of "open plan" but with even more features to make Government work more soul-crushing and complicated.

One office, with desks for 80% of the staff (because the other 20% need to take the hint and resign). Each desk only has one monitor, keyboard, and a mouse. If you've got certain ergonomic requirements, or need a colour accurate monitor, or a large monitor, or several monitors then you're just a naysayer who is obviously not productive enough to understand the ways of the future.

No one 'has' a desk, instead you grab your laptop out of your locker each morning and go find one. Or you might be allocated a desk via a morning raffle, not sure on this one yet. At the end of each day you clean every surface with alcohol wipes, which you then queue up to place in the singular bin that services the 300ish staff. Anyone who sits at the same desk twice will have to complete "Activity Based Working" training, in much the same way intoxicated road users may attend a DUI class.

There will also be no car parks for staff, who are being encouraged to use public transport. The fact that this public transport doesn't actually exist yet is just a "growth opportunity", but who's growth we're referring to here is not yet clear.

This might all sound like a joke, but the sad thing is it's 100% serious. Literally all of the above has been set in stone by minister that our department reports to.

carapace 14 hours ago 1 reply      
I will take $20,000 off of my pay if you will give me my own office with a door I can close.
peeters 16 hours ago 1 reply      
> bench seating

Wait long tables is one thing, but what on earth does bench seating mean? Like it's one long picnic table?

molestrangler 17 hours ago 2 replies      
This is only a rumour, I would wait for some photos to appear to confirm this.

Anyone got any recent images now people have started moving in?

There Have Always Existed People Whove Simply Wanted to Be Alone hazlitt.net
422 points by fern12  3 days ago   165 comments top 28
nabla9 3 days ago 9 replies      
Being alone and hiking and camping in the wilderness without human contact for longer period can be amazing experience. There can be initial anxiety and intense desire to go back after romance goes away and your internal shit comes to light. When there is constant need for do chores to survive but also free time and no human contact, no books, radio or music, mind gradually settles into itself.

It's like coming from bright light into a dark room. Gradually your eyes adjust and you start to see more. Coming back into the civilization is similar to someone pointing flashlight into your eyes. So much external triggers for behaviour. Realizing that I'm not actually me with other people and I'm disappearing into network of others. Me with others is mainly just bunch of triggers that fire based on conditioning.

If I can feel intense otherworldliness from just week or month alone, I imagine that if someone spends decades alone, civilization might seem like miserable alien ant colony. Everybody is responding to commands from others and carrying stuff they don't care about.

ps. It also can trigger psycosis, panic or some kind of madness (prairie fever, cabin fever) in some people. Romanticizing it as escape from all your problems might give people the wrong idea.

white-flame 3 days ago 3 replies      
It baffles me that people think it's so shockingly outlandish that there are people who don't reactionarily buy into the tribalistic pressures around us to simply act like everyone else. Humanity is not homogeneous, yet some notions like these are always projected out to be considered an immutable, inescapable constant. I can only guess that's that same fear of being different shining through.

I presume that on sites like this, there's a higher percentage of people who attempt to be more intentionally decisive about themselves and their lives. We don't necessarily have to toss out everything like Knight did, but looking at life and all the weird social rituals and expectations built up, the dichotomy between those and what seems actually beneficial becomes apparent. That conflict causes a choice, we would seek to do the "better" thing, and that draws many people outside the superficial social norms.

I especially bristle at this quote: "Why dont we want to be alone? Because the stuff thats down there is stuff you dont want to see." Anybody who tries to intentionally better themselves knows what's down there. You have to assess what you are if you're going to change. Sure, you can deny and hide from all that and simply find comfort in floating along with everybody else in social inertia, but that seems to me to be a shameful waste of those conceptual abilities which (apparently) make us uniquely human.

evervevdww221 3 days ago 5 replies      
I have the impulse to become a hermit myself, fundamentally because I'm tired of living up to other people's ideologies: going through schools and finding a job in a cubical. getting married at the right age and then raising the right amount kids, saving for their college fund and then for retirement.

but why?

I can't help but compare with my surroundings, even I have quit Facebook for many years. I can't be myself when I'm around others, but become a money maker for things I don't need. I can't concentrate on what makes me happy.

I recall what made me happy. it was when I finally understood some papers, some equations, some code. I just want to find a quiet place to do these. I hope to become an awesome painter and a guitar player too.

I just want to have enough to survive and focus my energy on these things. I don't care if I have successful kids or fancy cars.

grabcocque 3 days ago 5 replies      
The idea of introverts and extraverts being distinct groups of people with completely different neurological responses to social situations is a largely false one, created by self-help woo merchants to unhelpfully pathologise the feeling that EVERYONE has from time to time that they want to be left the fuck alone.

You know what? It's a normal, neurotypical part of life as a homo sapiens to want to be by yourself sometimes. And equally, it's a normal, neurotypical part of life as a homo sapiens to want to socialise sometimes.

We're a weird species like that, the way sometimes we want something and other times we want the opposite.

Mikeb85 3 days ago 3 replies      
The title IMO is far more interesting and thought provoking than the actual story. So this guy lived in the woods somewhat near people, and stole to get by...

I've personally always been fascinated by the topic as I have met several hermit monks, have a friend who lived as one for half a year, and contemplated it for myself. The history of religious asceticism and hermits is quite interesting, and many of history's most famous philosophers/religious leaders/prophets were either hermits, or had periods of reclusion. And nearly every single religion has these hermit figures.

Anyhow, the downside is that being alone is tough. Physically and mentally. Humans are social creatures. However I have personally benefited from periods of isolation and reflection, even if I'd much rather be around others.

factsaresacred 3 days ago 5 replies      
> Years ago, I went to India for a ten-day, silent retreat. I wanted to make myself go where I was afraid to godeep down, inside my own head. I found it terrifying. Why dont we want to be alone? Because the stuff thats down there is stuff you dont want to see.

We live in a world in which who we are is defined by what we do. We are a role - parent, engineer, carer. Strip that away and all that remains is a who. That's the reward of solitude: a situation wherein you have nobody to bounce your 'self' off, nobody to define yourself in relation to, allowing you to surface.

Turns out that who you are is simply a sequence of reactions to experience - the external kind as well as that which bubbles up internally. Rather than terrifying, this should be seen as profoundly liberating.

tray5 3 days ago 4 replies      
My personal theory for explaining hermits throughout history is simple, these people had/have aspergers. I have aspergers myself, and I can very easily see someone who has aspergers who for whatever reason no longer wants to socialize anymore getting up and doing their own thing out in isolation. I don't truly believe that any neurotypical person, and for that matter many aspies could do it, but if you're brain is wired in a way that socialization doesn't provide that reward that it does for most other people, either because you don't understand social interaction and have no desire to learn the rules so you can play the game, or simply because you have discovered the rules and simply have no interest exhausting the effort, going out into isolation and spending the rest of your days pursuing other things that give you fulfillment.
dahart 3 days ago 1 reply      
Snap Judgement did a nice podcast version of this story. http://snapjudgment.org/north-pond-hermit

The letters Knight & Finkel exchanged add an interesting angle.

Can't say I'm a fan of the author's choice for title of this post. It's Chris Knight's story, and nothing in this post presents any evidence for anyone else at any other time, aside from this single sentence "Think of Jesus, Mohammed, and Buddha: they all spent very long periods of time alone before introducing their religions." I totally believe there have always been people who want to be alone sometimes. Pretty much everyone wants to be alone sometimes. But ugh, this sentence & title seem to strain credulity and are so completely unnecessary and tangential to this story.

booleandilemma 3 days ago 4 replies      
He portrays a man who, without a shred of formal outdoor training, survived through ingenuity and remarkable self-discipline

The man burglarized people's houses for supplies.

dr_bloodmoney 3 days ago 2 replies      
I read about this man when the story first broke and found it extremely fascinating. I can relate to wanting to be alone and living an isolated existence. I love being in nature, away from the world and have often thought about pursuing such an existence permanently. But I just cannot comprehend his methods. Move to Alaska. Learn to hunt. Carve out a place for yourself somewhere. What he did tells me he was just insane - live near people and steal. To put it bluntly, this is fucking nuts.
Chiba-City 3 days ago 1 reply      
FWIW, hundreds of thousands of Christian and Buddhist monks alive today all over our world live near to one another in cells or caves. A farming monastery in Arizona started 20 years ago is just beautiful and thriving. WV now has a Buddhist monastery. Buddhist monks are considered "ordained" but not most Christian monks. Lives of quiet or social Renunciation are everywhere and growing in number. Some even have fast WiFi. Go look on YouTube. I have been considering and schematically budgeting an urban ecumenical working monastery in Washington DC that would support itself with OSS testing, documentation and language localization.
cJ0th 3 days ago 0 replies      
> Years ago, I went to India for a ten-day, silent retreat. I wanted to make myself go where I was afraid to godeep down, inside my own head. I found it terrifying. Why dont we want to be alone? Because the stuff thats down there is stuff you dont want to see.

I find it interesting that silent retreats work for so many people. While I do get some benefits from meditation it doesn't really lead me to terrifying situations. From time to time it feels like I am having some epiphanies wrt to my shortcomings but meditation seems too gentle to call those moments a confrontation. There is always this nice, cool distance between me and my thoughts. Throwing me into an impro theater group might be more beneficial (and terrifying).The required spontaneity would force the "actual me" to live through uncomfortable situations and perhaps grow.

KhanMahGretsch 3 days ago 1 reply      
This looks like a good thread to recommend one of my favourite YouTube channels, "Primitive Technology", which features all manner of tools and dwellings built caveman-style.

It's creator is a hacker in the truest sense; his forge-blower contraption, for instance, is simply ingenious.

Don't forget to turn on captions, the subtitles describe what he's doing :)


FrozenVoid 3 days ago 1 reply      
I don't want to live in a wilderness, but i would want to greatly reduce mandatory social interactions i have to do daily. Its emotionally draining and stressful.Almost everything we do can be automated, but people still insist on face-to-face interaction(or at minimum voice/video chat) and there is this herding behavior that forces people to adjust their beliefs and thoughts to conform to current in-group paradigms(the comparison with ant colonies ITT is on point).All the 24/7 media exposure and rat race of consumerism eventually take their toll on mental health(the polar opposite of "hermit slowly losing their minds") with people becoming psychotic and dependent on pills to function.
DannyDaemonic 3 days ago 2 replies      
There's an evolutionary advantage to having people who are isolated from the rest of the community. And not just in terms of sickness transmission. Things such as war, famine, and natural disasters can wipe out whole population groups.
gozur88 1 day ago 0 replies      
But Knight was never really alone. Sure, he didn't live with or talk to people. But he couldn't survive without stealing from the people around him.
Zuider 3 days ago 0 replies      
"Man is by nature a social animal; an individual who is unsocial naturally and not accidentally is either beneath our notice or more than human. Society is something that precedes the individual. Anyone who either cannot lead the common life or is so self-sufficient as not to need to, and therefore does not partake of society, is either a beast or a god"

From Aristotle, The Politics.

mythrwy 3 days ago 0 replies      
Maybe "introvert" or "extrovert" depends largely on the potential company.

All people (and cultural groups for that matter) are not equally pleasant to be around.

taw55 3 days ago 1 reply      
Here is another, longer form article about Chris Knight by the same author.


mkhalil 3 days ago 2 replies      
"Why dont we want to be alone? Because the stuff thats down there is stuff you dont want to see."

This really resonates with me. The "Fear of missing out" is something I try to avoid like the plague, but sometimes I wonder why do I even care?

edit: replaced the acronym FOMA

asherkosaraju 3 days ago 0 replies      
And there is nothing wrong with it. Most people think being alone is equivalent to being antisocial. A common misconception that needs to be addressed. The person doesn't need therapy, they just want to be left alone.
mark_l_watson 3 days ago 0 replies      
Great story. I wonder how spiritual/ religious Chris is. I understand being alone in nature and the desire for solitude but Chris's life style probably has a strong spiritual component. I am going to have to read the book.
aaron695 3 days ago 0 replies      
There have always existed people who have suffered from severe depression. There have always existed people who are sucidial.

I find the title a bit dangerous, but an interesting topic.

lngnmn 2 days ago 0 replies      
Since the time Vedic seers, I suppose...

BTW, prolonged solitude has been considered by most major Eastern schools (both Hindu and Buddhist) as necessarily precursor for spiritual transformations.

Indolat 3 days ago 0 replies      
A man can be himself only so long as he is alone; and if he does not love solitude, he will not love freedom; for it is only when he is alone that he is really free.

Arthur Schopenhauer

fundabulousrIII 1 day ago 0 replies      
Always bring a cat.Otherwise why not?
JumpCrisscross 3 days ago 0 replies      
How does one find, or plan, such retreats?
stcredzero 3 days ago 0 replies      
True story: in the early 90's, I walked up to and witnessed this: There was a young woman running a "Museum of Elvis" in a storefront in Portland. She wasn't making it financially, so unable to pay off her student loans, she took to sitting in the storefront in a chair, with a sign and a donation box saying, "I just want to be left a loan."
Jeff Deans Lecture for YC AI [video] ycombinator.com
428 points by danicgross  1 day ago   55 comments top 6
iandanforth 1 day ago 3 replies      
The notion of running one giant model that has many sub-talents is epic. I can imagine that all the disparate models they run today could fuse into a giant network that melds predictions and guides computation as required by the task. That seems like a very Jeff Dean scale endeavor.
sputknick 1 day ago 8 replies      
If Tensorflow becomes the default library for Deep learning, is this a good thing or bad thing? Does it help in that all researchers can focus on what's important (the data and results) or does it hurt in that Google now controls an important paradigm for the next generation of computing?
litzer 1 day ago 2 replies      
As somebody who's recently starting to learn more about ML, a lot of the work of an ML engineer does seem to be automate-able (not doing research or pushing boundaries but just applying ML to some product need). For example, choosing hyperparameters, evaluating which features to collect, etc seem to be things that can be automated with very little human input.

His slide on "learning to learn" has a goal of removing the ML expert in the equation. Can somebody who's more of an expert in the field comment on how plausible it is? Specifically, in the near future, will we only need ML people who do research, due to the application being so trivial to do once automated?

hallman76 1 day ago 1 reply      
As a ML enthusiast, this is incredible to watch!

I'm completely blown away that Google was working on full-scale physical architectures that were optimized for these problems. Talk about being two steps ahead of the game!

bluetwo 1 day ago 7 replies      
If a doctor misdiagnosis an eye ailment, they might end up with a malpractice lawsuit. If an ML program misdiagnoses an eye ailment, what is going to happen?
deboflo 1 day ago 1 reply      
Once, in early 2002, when the index servers went down, Jeff Dean answered user queries manually for two hours. Evals showed a quality improvement of 5 points.
Software Engineering Computer Science (2009) drdobbs.com
470 points by nreece  2 days ago   295 comments top 24
alkonaut 2 days ago 7 replies      
Software engineers need to know to recognize and classify problems in CS. You need to know what algorithms and data structures exist, what their properties are, and what they are called. The areas that come up will come from Math and Computer science (which are closely related). A solid computer scientist person knows how to derive some Dijksttra algorithm from first principles. A good software engineer recognizes the problem at hand, and recalls the algorithm to pick when presented with the problem.

What is that problem in front of you? Gradient descent? Tree traversal? Multiple dispatch? Path finding? What structure represents the data or algorithm? Ring buffer? Blocking queue? Bloom filter?

You rarely need to remember a pathfinding algorithm or trie implementation by heart. What's important is that you a) recognized the problem at hand as "path finding", "bin packing" or whatever. Terminology is important here. The good software engineer needs to know the proper names for a LOT of things. Recognizing and labeling problems means you can basically look up the solution in no time.

So CS is definitely very relevant for software engineering - but you need a broad understanding instead of a deep one.

There is always the argument that a lot of devs basically to monotone work with SQL and some web thing in node and rarely even reach for a structure beyond a list or map. That's true - but sooner or later even they bounce into a performance or reliability issue that's basically always due to incorrect choice of data structure or algorithm. I'm only half joking when I suggest that most of todays "scaling" is compensating for CS mistakes in software.

jolux 2 days ago 5 replies      
I don't understand why the author is so suspicious of formal methods. Other engineering disciplines are based on the application of solid, well-understood principles from the natural sciences to practical problem domains. There are few solid, well-understood principles in computer science that are directly and obviously applicable to software engineering so far.

I vigorously contest the idea that software engineering cannot be rigorous and so shouldn't try.

whatnotests 2 days ago 4 replies      
100% agree.

So unless you spend all day writing compilers from scratch or calculating Pascal's Triangle, please stop with the ridiculous CS questions in interviews.

Software Engineering is more of a trade, and requires vocational knowledge and experience. A mountain of theory may not always be required to Get Shit Done.

peterburkimsher 2 days ago 7 replies      
Here's the graphic transcribed as text for non-English speakers.

Software Engineering: Requirements, Modifiability, Design Patterns, Usability, Safety, Scalability, Portability, Team Process, Maintainability, Estimation, Testability, Architecture Styles.

Computer Science: Computability, Formal Specification, Correctness Proofs, Network Analysis, OS Paging/Scheduling, Queueing Theory, Language Syntax/Semantics, Automatic Programming, Complexity, Algorithms, Cryptography, Compilers.

In my opinion, some of those could be on the other side of the line (estimation could be CS, language syntax/semantics and network analysis could be SE). But I agree with the general division.

I studied Electronic Systems Engineering, but somehow always found jobs in software companies. One problem I struggle with is the division between DRY (Don't Repeat Yourself) and WET (Write Everything Twice) coding styles.

Most programmers hate it when code is repeated. They prefer to spend days trying to integrate external libraries instead of just copying the necessary functions into the main branch. There are good reasons for this (benefiting from new features when the library gets updated), but there are also risks (the code breaking when the library gets updated).

Software Engineering priorities include Safety, Portability, Modifiability, and Testability. I interpret that as a WET programming style. "If you want it done well, do it yourself." There's no arguing about responsibility then - the code is mine, and I should fix it if it breaks.

ChuckMcM 2 days ago 2 replies      
I've seen similar articles to this one, both in print and on web sites. I used to explain it to people as the difference between 'coders' and 'engineers' but I think my own hubris at having a degree got in the way of my thinking on it.

Over the decades I've met a bunch of people who program computers for a living, and there is clearly a spectrum where on one end is a person who spends the weekend benchmarking different sort algorithms under different conditions for the fun of it, and the guy who left the office at 5PM once an integration test passed on a piece of code that he pasted in from stack overflow was deemed to have no regressions. There are many different disciplines that have such a wide dynamic, from chefs who spend their weekends trying different flavors to cooks who take frozen patties out, reheat them and serve. Painters who throw human emotion into a painting and painters who lay down a yellow line on a road according to a template for $20/hr.

It seems to me that most, of not all, of the 'theory' stuff in computer science is just math of one form or another. This is not unlike all the 'theory' stuff in electrical engineering is just physics. You can do the tasks without the theory, but you rarely invent new ways of doing things without that understanding.

But just like carpenters and architects there is a tremendous amount of depth in the crafting of things. That brilliance should be respected, college trained or not, so trying to 'split' the pool doesn't lead to any good insights about what being a good engineer is all about.

halfnibble 2 days ago 3 replies      
I didn't study Computer Science in college. Not one single course. But I'm not stupid. I made straight A's in math through Calculus III. So a lot of these comments frustrate me. I've taught myself literally everything I know. I've read dozens of books. I practice coding obsessively--it's my passion. Do I "get shit done"? Yes, absolutely. Do I not care about the efficiency of my algorithms? No, I care deeply. I don't always know the "computer-sciency" term for things. But my goodness, get off your high horse and tell me what you want accomplished. Chances are I'll implement a solution that's just as efficient and arguably much better than most "engineers" can. And no, I'm not going to be obsolete at age 40. By the time I reach age 40, PhD's will be coming to be for advise. Because I didn't study computer science in college. I'm studying it for life.
ajarmst 2 days ago 0 replies      
I'm convinced that the only useful definition of a Software Engineer is "someone who has 'Software Engineer' in their job title". Most other Engineering disciplines are far more rigorously defined. That said, observing a disconnect between theory and application is hardly novel or unique to software disciplines.
amw-zero 2 days ago 3 replies      
> all computer hardware is essentially equivalent.

This is quite inaccurate. Hardware directly influences software. "if" statements, functions, and threads didn't exist at one time, and all require explicit hardware support. I believe that as we come up with different abstract constructs at the hardware level, we'll influence the possible software that can be written.

hestefisk 2 days ago 0 replies      
Software engineering is where the rubber hits the road in terms of requirements definition, creating a solid design, fitting stuff into an existing legacy environment (SAP anyone? Java EE?), iterating prototypes with stakeholders... and usually in large corporations. It was out of many years of budget overruns in defense procurement that software engineering cornerstones such as CMMI emerged.

To me, the essence of software engineering is that 20% is about building the 'good' solution itself, e.g. architecture, code, release / deployment, ... the remainder of the engineering is navigating / tolerating the inherent corporate messiness of politics, opinions, power, and everything else... engineering the solution is the easy part; engineering good requirements and quality is tough.

tim333 2 days ago 0 replies      
The author has a slightly funny use of the word engineering. If you look at its use in a conventional field like making cars then the science bit is the basic physics and chemistry of how gasses expand etc, the engineering is designing the machinery so the brakes work, the engine produces enough power and doesn't break and the like and then human issues like whether the workers go on strike or the end users are idiots and crash are not engineering.

Similarly I'd say in software the engineering bit is making reliable systems that are fault tolerant and secure and so on and then the people bits like the user interface are something like design and psychology, not engineering.

partycoder 2 days ago 0 replies      
Let's revisit the definition of "engineering", in a simplified form:

 Science -> Engineering -> Technology
Engineering borrows scientific[1] knowledge to create[2] technology[3]

[1]: or empirical knowledge

[2]: or maintain or implement

[3]: or processes

The relationship between science and engineering has been clear for a while now, even before the appearance of software engineering.

There's a lot of science at work in existing software, so it would be inaccurate to say that software is "unscientific". However not many people get to work on those projects.

A vast majority of people can make a decent living working on user facing technologies built with existing technology. At that level appealing to non-technical stakeholders has much more weight than applying engineering rigor.

But that's not the reality for everyone.

Chiba-City 2 days ago 0 replies      
These are complicated terms. Harvard's CS was part of their Applied Math department. There are Applied Maths of scheduling programmatic Engineering outcomes for sure. Fred Brooks taught us all that.

I studied Russell, Godel, Tarski and Quine and then compiler and runtime logic (as a Philosophy major). Back then CS was mostly a realm of 3-Page proofs on alpha renaming or newfangled Skip List speed/space utility.

As an old VAX/Sun or 512K/DOS C programmer working in DC for decades around lots of TC, datacenter and transaction processing folks, an SE MUST have basic speed/space, set theoretic, programming by contract, data integrity and MTBF abstractions in their heads while they plan and develop. Both accuracy and performance against test and measure just matter for the business cases 24/7.

Content software developers patching together framework components on 2 day schedules for consumer Web bloatware rarely understand something like data integrity needs of billing system logic embedding in redundant switches failing over on rough schedules. Typing commands is not even Software Engineering.

Software Engineering is not an individual identity phenomenon. SE is how groups show responsibility for stakeholder outcomes unrelated to paychecks. First rule of SE is everyone on the team passes the bus test. Nobody is essential. Unless we seek luck, we can't improve what we don't measure. Learning how and what to measure takes real training and group method application. So many out there never know what they are missing.

Business competition minus lucky windfalls is largely based on COST ACCOUNTING. Successful operations will discover heat dissipation costs challenges. Basic CS speed/space, contract covenant assertions, data integrity and MTBF logic in Software Engineers translates very easily into understanding business innovation problems.

drawkbox 2 days ago 0 replies      
Most developer jobs contain parts of both, with more time spent in software engineering.

Software development, app development, game development, web development are all probably 90+% software engineering and 1-10% computer science depending on the project. Specific projects may differ such as writing standard libraries, engines, data, standards, teaching, etc. In the end most of it is production and maintenance as part of shipping.

autokad 2 days ago 2 replies      
rarely has asymptotic complexity mattered to my code. usually the most important factor is modularization and readability. i spend more of my time reading or re-using code, and my time is more expensive than a computer. plus, highly optimized code can sometimes be unreadable and lead to bugs, which are also more costly.
k__ 2 days ago 1 reply      
In Germany we have Informatik, which was treated as CS&SE long time.

Lately there are sprouting more and more SE degrees.

On the other hand we also have universities of applied science, where Informatik is often more like SE

thomasbachem 2 days ago 0 replies      
We also wrote an article about this about a year ago: https://code.berlin/en/blog/computer-science-software-engine...
KirinDave 2 days ago 0 replies      
I neither agree nor disagree with the article. I think it conflates a lot of stuff.

But look, what the math and science sides of the room throw at us definitely informs the engineering. In every other engineering principle from architecture to ditch digging, there is a feeder system from a variety of mathematical and scientific disciplines. While many other engineering disciplines are well established, they are not immune to this and in general don't begrudge it.

Doctors are required to keep up on the state of treatment. Architects need to keep up on materials science AND new mathematical modeling techniques and tools. Car designers care about new discoveries in lighting, battery and materials technology.

Here's a good example of the kind of stuff we all should be on the hook for. I've tried to push this paper up to the front page a few times now because it's roughly the same as if someone walked up and calmly announced they'd worked out how to compress space to beat the speed of light:


Folks are generalizing linear sort algorithms to things we thought previously were only amenable to pair-wise comparison sorts without a custom programming model and tons of thought. No! And then a famous engineer-and-also-mathematician made an amazingly fast library to go with it (https://hackage.haskell.org/package/discrimination).

We're seeing multiple revolutions in our industry made of... well... OLD components! While deep learning is starting to break untrodden ground now, a lot of the techniques are about having big hardware budgets, lots of great training data, and a bunch of old techniques. The deep learning on mobile tricks? Why that's an old numerical technique for making linear algebra cheaper by reversing order we walk the chain rule. O(n) general sort is arguably bigger if we can get it into everyone's hands because of how it changes the game bulk data processing and search (suddenly EVERY step is a reduce step!)

We've similarly been sitting on functional programming techniques that absolutely blow anything the OO world has out of the water, but require an up-front investment of time and practice with a completely alternate style of programming. But unlike our fast-and-loose metaprogramming, reflection and monkey patching tricks in industry these techniques come with theorems and programmatic analysis techniques that make code faster for free, not slower.

Even if your day job is, like mine, full of a lot of humdrum plug-this-into-that work, we can benefit from modern techniques to build absolutely rock solid systems with good performance and high reliability. We could be directly incorporating simple concepts like CRDTs to make our systems less prone to error.

It's our job (and arguably it's the hardest job of the field) to dive into the world of pure research, understand it, and bring what's necessary out to the world of modern software. That means more than just tapping away at CSS files, or wailing about NPM security, or shrugging and saying, "Maybe Golang's model is the best we can hope from in modern programmers."

z3t4 2 days ago 0 replies      
What is software Engineering !? Making an excel sheet ? Making a web site ? Writing SQL ? Using programming language X, Y, Z ?
cpburns2009 2 days ago 0 replies      
Software Programming != Computer ScienceSoftware Programming and Science != Engineering

While we're drawing distinctions stop calling yourself an engineer unless you're legally licenced as one. Programming may share similarities with engineering but it lacks the professional accreditation and liability.

sytelus 2 days ago 2 replies      
Computer science is neither about computers nor is a science :).
hprotagonist 2 days ago 3 replies      
This is not a particularly new observation.

My half-assed analogy:

CS is to SE as Physics is to Mechanical Engineering.

In both cases, it's unwise to trust one category with screwdrivers...

lngnmn 2 days ago 0 replies      
Sure. Engineering is an applied science. So, these cannot be equal.
j05huaNathaniel 2 days ago 4 replies      
oneplane 2 days ago 0 replies      
Yeah, no shit...

Truck Driver Road Planner

832 TB ZFS on Linux jonkensy.com
319 points by beagle3  3 days ago   156 comments top 18
rsync 3 days ago 4 replies      
"I ended up between the Supermicro SSG-6048R-E1CR60L or the SSG-6048R-E1CR90L the E1CR60L is a 60-bay 4U chassis while the E1CR90L is a 90-bay 4U chassis. This nice part is that no matter which platform you choose Supermicro sells this only as a pre-configured machine this means that their engineers are going to make sure that the hardware you choose to put in this is all from a known compatibility list. Basically, you cannot buy this chassis empty and jam your own parts in"

This is a major departure from the Supermicro business model and practices and basically broke all of our next generation expansion roadmaps.

This was not a technical decision - it is the same old economic decision that every large VAR/integrator/supplier has succumbed to for the last 30 years. They aren't the first ones to try this trick and they won't be the last.

We (rsync.net) are not playing ball, however. After 16 years of deploying solely on supermicro hardware (server chassis and JBODs) we bought our first non-supermicro JBOD last month.

kev009 3 days ago 1 reply      
There are a couple needful tweaks to this BOM for anyone wanting to follow this..

Only populate one CPU socket. Zone allocation between two NUMA nodes is kind of hard, especially since Ubuntu 16.04 zfs is pre- OpenZFS ABD where memory fragmentation is reality.

I would recommend better NICs like a Chelsio T5 or T6. Aside from better drivers and a responsive vendor, you can experiment with some of the iscsi offloads or zero copy TCP.

Supermicro seriously under-provisioned I/O on that chassis. I'd add LSI/Avago/now Broadcom cards so you can get native ports to every drive. Even if it's just a cold storage box, it will help with rebuild and scrub times and peace of mind. The cost of this is not bad compared to the frustration of SAS expander firmwares. 2x24 or 3x16 and 4 drives on the onboard if you can skip the backplane expander. Supermicro will usually do things like this if you insist, or an integrator like ixSystems can handle it.

More subjectively, I would also recommend FreeBSD. It seems their main justification for Ubuntu was paid support, which can be had from ixSystems who sell and support an entire stack (Supermicro servers, FreeBSD or FreeNAS or TrueNAS, and grok ZFS and storage drivers to the tune that they have done quite a bit of development.

twiss 3 days ago 0 replies      
> I purchased these units through a vendor we like to use and they hooked us up, so I wont be able to share my specific pricing. (...) If you build the systems out on there youll find that they come in around $35,000 (USD) each.

That devided by 52 x 8 = 416TB is 0.084$/GB. For comparison, the Backblaze Storage Pod 6.0 [1] claims 0.06$/GB for the version with the same hard drives. Although this version has a bunch of extra features like 2 x 800GB SSD's for ZFS SLOG, 8x more RAM for a total of 256GB, etc.

[1]: https://www.backblaze.com/blog/open-source-data-storage-serv...

4ad 3 days ago 6 replies      

 you can run ZFS on Ubuntu [...] You could also build this on Solaris with necessary licensing if you wanted to that route but itd be more expensive.
I find it bewildering the author didn't even consider illumos or FreeBSD, where ZFS is a first class citizen.

sandGorgon 3 days ago 2 replies      
The most important line for me was "Today, you can run ZFS on Ubuntu LTS with standard repositories and Canonicals Ubuntu Advantage Advanced Support. That makes the decision easy."

Its highly interesting that Canonical does this with ZFS. I'm not sure why they dont market this more.

vc00000 2 days ago 0 replies      
We bought our initial 2 TrueNAS servers from IX Systems (SuperMicro) back in 2011, have been upgrading over the years and they have been very reliable servers.

Currently each server has 63 drives (4TB HGST NL-SAS) with 1 hot spare, configured as RAIDZ.

Right now there is 200TB of usable storage, we initially started with 29TB and have been expanding as needed when it hits about 79%, I buy 18 drives roughly every 6-8 months, 9 drives per server and expand the pool.

To say that we never had issues is lying, we did have some major issues when upgrading from versions, but this was early on, now it is a rock solid storage system.

Although there is less than 300 active users connecting to the primary server, there is a lot of very important pre & post production high dev videos.

Reboot with 63 drives is around 10 minutes or less.

Resilvering could take 24-48 hours, depending on load, depending on how much data the failed drive contained.

Performance has been great, reliability has been great, support has been great.

Sadly IX Systems can no longer provide support after the end of this year, they've extended support beyond the expected lifetime of the hardware.

jjirsa 3 days ago 9 replies      
Zfs on linux and huge single servers, what could go wrong?

It's like a blog written by a 22 year old straight out of college that's never dealt with a real production deployment/failure

Zfs on Linux has data loss bugs. There's at least one unpatched and there are bound to be more.

Single huge servers eventually fail. Maybe it'll be a drive controller. Maybe it'll be CPU or ram with bit flips as a side effect. Downtime would be the least painful part of the eventual failure.

guroot 3 days ago 2 replies      
Can I just ask.Why not use FreeBSD?
BigIQ 2 days ago 0 replies      
Biggest question: why?

At that scale something like Ceph would be more reasonable. Just because ZFS can handle those filesystem sizes doesn't necessarily mean that it's the best tool for the job. There's a reason why all big players like Google, Amazon and Facebook go for the horizontal scaling approach.

mikekij 3 days ago 0 replies      
Almost big enough to archive SoundCloud!
jaytaylor 3 days ago 1 reply      
What about cooling? Will the lifespan of the high-capacity platter-dense hard drives be drastically reduced by clumping them together like that with what looks like little airflow?
cmurf 3 days ago 1 reply      
Do either of these project spec hardware that would work for this use case?

opencompute.orgBackblaze storage pod, they're up to v 6.0 now

(Netflix open connect specs supermicro hardware)


andreiw 3 days ago 1 reply      
I wish Supermicro had a similar chassis around the Cavium ThunderX. That would make a lot of sense for network-attached storage, regardless of whether one goes with SATA or drops in a SAS adapter or two. Does anyone know if any of the Cavium accelerators (crypto or compression) can improve ZFS perf?
rurban 3 days ago 0 replies      
I'm not a HW guy but those drives seems to be far too close together. A few more millimeters space will keep the temperature down much better I assume.
z3t4 3 days ago 3 replies      
Anyone else addicted to acquiring servers and high bandwidth connections ? Any ideas on what to do with the over capacity ?
SoMisanthrope 3 days ago 0 replies      
Very impressive! It's amazing what people are doing with OTS technology.
notyourday 3 days ago 0 replies      
Very good experience with 45drives.com storinator XLs.
yest 2 days ago 1 reply      
>Its hard if not impossible to beat the $/GB and durability that Amazon is able to provide with their object storage offering.

what the actual fuck?? AWS S3 is a abominable rip off. After I rented to my own dedicated server, I am paying several times less.

Eager to Burst His Own Bubble, a Techie Made Apps to Randomize His Life npr.org
402 points by ZeljkoS  2 days ago   169 comments top 36
et-al 1 day ago 9 replies      
To a certain extent, this how we used to travel back in the day.

You hung out in a hostel, had conversations with other travelers (instead of thumbing through Instagram), and let the randomness of other people and life, not apps, dictate your itinerary. You walk down a street, "oh hey that looks interesting", and wander down a quiet alley that leads to cute cafe, or jump in the back of a tuk-tuk headed to a waterfall that may or may not really exist, but who cares? You're riding the wave. One of the main reasons for travel/holidays is to break from routine, and the single most significant one can do, bear with me Silicon Valley, is to put away that smartphone. Try exercising your intuition instead of apps.

Many folks nowadays have optimised their lives so much that they've needed to create a noise-generator to bring back some humanity.

reustle 1 day ago 4 replies      
I actually met Max while we both happened to be in Thailand in 2015. We spent the day on bicycles following the directions his script told him, without hesitation. Regardless of where it was in the city, that's where we went next (a laundry mat, daycare, cafes, and the zoo iirc). Nothing was skipped, because it was what the software told us to do.

Here are some pictures from that day https://goo.gl/photos/gyCNRz2rs7zLJrt79

maxhawkins 1 day ago 2 replies      
If you want to try this out check out my Facebook group, The Third Party:


You can send us a message to receive a randomly selected event near you. People from all over are attending the events and posting about their experiences in the group.

simmons 1 day ago 3 replies      
This is great. I think that the illusion of time speeding up as we get older is due to getting into a routine where we do the same thing every day, and the brain begins compressing memories as our everyday experiences become less novel. Using a "randomizer lite" program to shake things up might be a good start to breaking the routine.
owenversteeg 1 day ago 2 replies      
Ah, but what was his source of randomness? Perhaps he didn't have enough entropy and now he's got to do the last few years all over again ;)

I think I naturally do a bit of that myself: whenever I have some empty time, I fill it with something "random", only instead of choosing randomly I often choose the cheapest option. For example, I once booked a flight to Iran departing hours after I bought the ticket, simply because it was the cheapest option for an interesting place to fly (under $180 round-trip.)

I think the design choices really impact the end result, though. One minor design flaw might result in completely eliminating a whole lot of interesting places or things, which is what I'd be scared of. For example, that cheap fare to Iran was only on one travel search site, which didn't have an API. By selecting one booking website as an API, and letting the algorithm decide for me, I wouldn't have gone to Iran.

Similarly, there are a lot of things that wouldn't seem like an "option" to a computer but are an option to me. I've wanted to see Greenland from the air, so I've been taking a lot of flights that pass over Greenland on the way to places I needed to go anyway. But if the algorithm decided for me, it would probably have booked air tours over Greenland - substantially more expensive in terms of both time and money. It wouldn't be able to say "hey, you know that trip across the Atlantic you have in a few weeks? Why not pay $25 more to have it fly over Greenland?"

Mz 1 day ago 2 replies      
This "I'm bored with my life. I shall appify it and travel to random countries" stuff is a whole level beyond First World Problems. It is something like First World Problems of the Jet Set. Yet, this article doesn't cast him as a member of the Jet Set.

Modern Life has gone to some rather weird places that were simply inconceivable until incredibly recently.

dogruck 1 day ago 6 replies      
Max should meet Luke Rhinehart -- The Dice Man: https://en.wikipedia.org/wiki/The_Dice_Man
throwaway2016a 1 day ago 4 replies      
This is an awesome idea.

I've spent a lot of energy making apps to automate my life and management my schedule. Now I'm tempted to also have it throw in something random now and again. I couldn't go to these extremes (I couldn't pick up and move to another country for instance) but it would be cool to say do things less extreme... like pick a random place for dinner or watch this random Netflix movie.

ZeljkoS 2 days ago 0 replies      
bartread 1 day ago 1 reply      
My immediate reaction to this was, "hey, this is awesome," and actually it really is. I think it's an interesting and creative solution to a problem a lot of us share - admittedly whilst recognising that for large swathes of the world's population this would be a great problem to have.

Still, there's that nagging little voice in the back of my head: part of me can't help wondering what will happen, and how people will come to view it, if/when his apps catch on (as I suspect they will).

zer0th 1 day ago 0 replies      
I've noticed that problem in my life as well. As a result, I decided to consume media a bit more randomly. My harddrive has a folder called "todo". This folder contains subfolders labeled "watch", "listen" (which includes "music" and "podcasts"), "read" and (a fun one) "pilates".The folder "watch" includes mostly movies and documentaries, sourced from my public service broadcaster's VOD service (which I scan weekly for interesting stuff) or youtube. The "music" subfolder of "listen" contains first and foremost dj mixes I've collected from eclectic rss feeds."read" has all kinds of PDFs and HTML pages (mostly scientific papers, long-winded articles and e-books). The "pilates"-folder contains videos of individual exercises (I find it more exciting to not always do the exact same routine)

On top of that I've written a script which prompts me to choose a category (i.e. one of the folders delineated above). It then randomly opens a file from the respective directory. When I am done using a file the script asks me whether I'd like to archive it, keep it (in case I am not done with it yet) or delete it.

driverdan 1 day ago 1 reply      
I love the idea of random surprises.

I created a simple app that picks random items from a Chinese ecommerce site within a set budget. My long term goal was to use ML to learn what each user liked and send them random items on a schedule, selecting from multiple sites.

I never finished it because other things took precedence but the random selection part works.

Anyone interested in this as a service?

mcrad 22 hours ago 0 replies      
I'm not a developer but I did once build a prototype that sent text message inquiries throughout the day (within hours configured as "open" on my calendar). The questions were customized around a few goal areas, as a way to measure my progress. I never found much support for it, but the concept was intended to give techies with the hyperactive schedul tools a chance to leave "generic" hours on the calendar while still being accountable to being productive during those times.

Tried a similar idea with a diet-specific set of goals. Aiming for 100% comprehensive tracking of your calories seems a bit crazy to me.

narrator 1 day ago 1 reply      
This is the plot of the indie movie called "Buggaboo": http://www.imdb.com/title/tt0206610/

The main character, who is an Indian engineer living in Silicon Valley, says to his friends : "What if you could randomize your life?". That's all I'll say. It's a good movie. I'm not going to spoil it for you guys.

rnprince 1 day ago 0 replies      
Reminds me of undergrad class registration, when everything satisfying a general education requirement would fill up and the server would crash the instant it started.

I didn't choose to take History and Religion of Ancient Israel, but it was the class I got the most out of.

monksy 23 hours ago 0 replies      
I've been looking to do something like this for my own side project.

Unfortunately Yelp is pretty restrictive about working with their review data. (They only give you 3 reviews per business on the APIs, also they're hostile against scrapers)

Hasn't anyone seen projects that do similar things with Facebook/Foursquare/Yelp data?

avip 1 day ago 0 replies      
Inspiration from Borges maybe?

https://en.wikipedia.org/wiki/The_Lottery_in_Babylon(it's in the highly entertaining Ficciones collection)

zoom6628 1 day ago 0 replies      
Brilliant way to actually take a break, or holiday, for those with the means to do it. I only plan my holidays down to which airport to arrive/leave and where to stay (yes im picky about that). Everything else in between arrival and leaving i prefer to depend on what I see/find/hear when walking around. That is my idea of a real holiday. Each to their own - YMMV.
darioush 20 hours ago 0 replies      
I don't like NPR's attitude of conflating social anxiety of meeting a random bunch of people and whether they're going to accept you with racial issues.

This makes me like them significantly less.

clamprecht 1 day ago 1 reply      
See also, Geohashing: https://en.wikipedia.org/wiki/Geohashing

"Geohashing is an outdoor recreational activity inspired by the webcomic xkcd, in which participants have to reach a random location (chosen by a computer algorithm), prove their achievement by taking a picture of a Global Positioning System (GPS) receiver or another mobile device and then tell the story of their trip online."

JunkDNA 1 day ago 0 replies      
If your life needs more randomness, I highly recommend having or adopting children. Having three of them 7 and under, I can attest that the entropy in my life is off the charts. I have no need for an app.
lolc 1 day ago 1 reply      
Well he won't meet me. I live in the No-Facebook Bubble.

Though sometimes the parties I help organise are listed on the Facebook by third parties. So there's that slim chance...

gehwartzen 1 day ago 0 replies      
The past eqivilent was flipping to a random page in the yellow pages, throwing a dart at a map, or flipping a coin. It's funny that we need a app to randomize our life at all. I suppose it is largely just how we acces information now.
Tade0 1 day ago 2 replies      
Great idea, but I don't need an app for that. My fiance takes on this role - we've been living in another country for almost two years because of one idea she had. I'm picking the next place once both of us get bored with this one. Suggestions, as always, welcome. Has to be in the EU though.
dubin 1 day ago 1 reply      
Anyone who's interested in more ideas along these lines should look at Tyler Cowen's list of how to be less complacent: http://tylercowen.com/complacent-class-quiz/
jpatokal 1 day ago 0 replies      
The Degree Confluence Project is an interesting version of random travel: people visiting arbitrary lat/long intersections.


628C6l0 1 day ago 2 replies      
i've been doing it for almost four years now, and it's amazing to see how wrong and how frequently wrong your preconceptions and expectations can be.

you don't need an app to do that though. a spreadsheet is perfect for that purpose (and telling Google Assistant to 'flip a coin' or 'give me a random number between 1 and 20') and gives unlimited flexibility and less reliance on developer to implement features you want. not a lot of people appreciate this, but sometimes trying to accomplish a task with the general-purpose tools you have at hand can lead you to discover solutions so good it is in fact superior to any dedicated tool.

cconstantin 1 day ago 0 replies      
Just imagine someone hacking into this algorithm
somberi 1 day ago 0 replies      
To quote NY Times - "Algorithms lead us to Anagrams"
barsonme 1 day ago 0 replies      
some people use apps, others of us have ADHD that works just as well :-)
thebigspacefuck 1 day ago 0 replies      
"Peculiar travel suggestions are dancing lessons from God"
ghosttie 1 day ago 0 replies      
So basically diceliving
allard 1 day ago 0 replies      
like John Cage
stillhere 1 day ago 5 replies      
user432468 1 day ago 4 replies      
NPR so diligently points out white privilege. It couldn't be that he dresses well, has good hygiene, is educated, well off, and speaks politely. It is only because he's white? In San Francisco of all places?

> At first, he was nervous: What if people wouldn't let him in? But, as a kind of unassuming white guy, he actually didn't have this problem. (And Max acknowledges this privilege.)

A Tech Leads New Project Checklist insimpleterms.blog
370 points by LeonigMig  1 day ago   41 comments top 9
JackC 1 day ago 4 replies      
"Understand the importance of the Trinity of delivery: Delivery manager, product owner, tech lead"

I'm only familiar with teams too small to have separate roles like this. How does good software planning scale down to smaller teams -- say 5 or 10 people in the whole organization? Ultimately someone has to be responsible for the same concerns, but I wonder how it maps.

In general I'd love to see a comparison of software teams at different sizes. What are the key, identified roles in a company of 5, 50, or 500? What are habits that smaller organizations ought to borrow from larger ones?

fundabulousrIII 22 hours ago 1 reply      
How about fuck this paradigm?

Scrum = Shite verbosity about agile process and training for mgmt larva.Agile = Hyperbole. AWS = I don't know better and my developers are < 35 years old or on the make (+ stock). My customers are just stupid.Security = big $$, modest skills, certs for the neglected comptia and northcut industures. I'm a CEH!! So is every script kiddie on the internet.tester: can't write working code, useless to the degree that you come in when they need you to justify junit crap numbers you generated when drunk. This will scale if 12 coors == 12 saudi virgins.analyst: I'm an experienced tester.tech lead: best resume and best bully of the bunch. I can understand stack exchange!QA: I like your doxygen docs and the test numbers but I have a problem with connecting to server(x) at ip(x) from greenland at 2pm on a sunday. Any ideas? team: Bunch of backstabbing co-moderators.

megamindbrian 1 day ago 0 replies      
Here is the other one from today. I should turn this in to an app-style application for employers/recruiters to fill out before they contact me.https://www.sideprojectchecklist.com/marketing-checklist/
bastijn 1 day ago 0 replies      
Might want to add company size and team size(s). Many of these points show it is most likely for small to medium sized companies. For an enterprise the checklist does not entirely hold due to more specialized roles. Though there are some good tips.
Domenic_S 1 day ago 4 replies      
Whats the budget and the value proposition? should be #1. Projects without a clear purpose (happens way more than you might think) are sinking ships you've got to get away from.
dbg31415 1 day ago 0 replies      
Ask about approved licenses and also make sure you know what you've purchased when it comes to platform service levels.
tomcam 1 day ago 0 replies      
My primary requirement is a demonstration that the site can be torn down and recovered any time you never know when you will have a DNS problem or someone take over your domain
anotherevan 1 day ago 0 replies      
Unrelated minor rant: Pocket is hopeless when dealing with bullet points. It so often skips list items, especially if they have links in them. I understand that they are trying to avoid including navigation stuff, but overzealous way too often.

I wish I could integrate Instapaper with my Kobo ereader instead.

throwme_1980 1 day ago 5 replies      
here is what i usually do:

-identify weak people in the team with a stinky attitude, those who dont learn and are toxic, do your best to get rid of them.

-next identify those mediocre people doing 9-5, maximise their output during those hours, don't give them no slack.

-finally identify your super stars, cherish them, buy them coffee/lunch and give them a lot of slack.

for this is a meritocracy, no damn Disneyland.

How do you find integer solutions to x/(y + z) + y/(x + z) + z/(x + y) = 4? quora.com
444 points by jordigh  2 days ago   93 comments top 12
mrkgnao 2 days ago 2 replies      
For anyone looking to learn more about elliptic curves, which are astoundingly "well-connected" as math topics go, here's a good book that should be accessible to anyone with some calculus under their belt:


It's not a textbook, which is both good and bad. In my case, it did a good job whetting my appetite for more!

A really well-written and not-extremely-difficult undergrad textbook on elliptic curves:


(Non-affiliate links, just so you know.)

rsj_hn 2 days ago 1 reply      
This is a great article. Little puzzles like these are often the gateways to enormous mathematical journeys. Here, I only wished the journey was more detailed -- you could motivate all of these operations -- but that would take many pages.
AbacusAvenger 2 days ago 2 replies      
The Wolfram Alpha results for this one are pretty intimidating:


jfaat 2 days ago 3 replies      
> Next up: what is the degree of our equation? The degree is the highest power of any term showing up. For example, if you have (a^2)b(c^4), thats a term of degree 7 = 2 + 1 + 4

I thought that equation was degree 4, which aligns with what the author says later on. Am I missing something? It seems odd that he would write out that equation accidentally, maybe just crossed wires though. I'm sure we've all been there.

bmc7505 2 days ago 11 replies      
What are some other problems that appear deceptively simple, but are extremely difficult to solve?
DrTung 2 days ago 0 replies      
Agree, good article. Thanks to it I think I understand more why cryptography based on elliptic curves (like Curve25519) is considered pretty safe for now.
tim333 1 day ago 0 replies      
A bit tangential but I wonder if something like that could relate to the slightly odd collection of fundamental particles we find in physics. They have properties that have to come out integral like spin x 2 and charge x 3 and have odd values like muons and tao particle being basically electrons but approx 200 and 3500 times heavier.
chegra 2 days ago 3 replies      
x = 702

y = -390

z = 858

[ http://www.wolframalpha.com/input/?i=(702%2F(-390%2B858))%2B...) ]

Found the above solution using a hillclimbing algorithm.


Houshalter 2 days ago 1 reply      
I wonder if a SAT solver would be able to solve this faster than brute force.
theophrastus 2 days ago 4 replies      
using the rearranged form:

 ((x + (((2*y*z) + (y*y) + (z*z) + (((z*z*z) - (z*y*y))/(x + y)))/(x + z)))/(y + z))
I find by brute force (no pride!) the first solution triplet: 35, 132, 627

Edit: of course, this is not a solution. It's now just an example to others to beware of floating point errors.

ramshanker 2 days ago 0 replies      
Now eagerly waiting for Letsencrypt to make Elliptic curve as default. Respect to the author of the article.
masthead 2 days ago 2 replies      
Detecting Chrome headless antoinevastel.github.io
369 points by avastel  4 days ago   155 comments top 24
westoque 3 days ago 10 replies      
Your solutions in detecting Chrome headless is good.

But someone who really wants to do web scraping or anything similar will use a real browser like Firefox or Chrome run it through xvfb and control it using webdriver and maybe expose it through an API. I find these to be almost undetectable.. The only way you can mitigate this is to do more interesting mitigation techniques. Liie IP detection, Captchas, etc.

edit: when I say real browser, I mean running the full browser process including extensions etc.

shakna 3 days ago 3 replies      
> ... to automate malicious tasks. The most common cases are web scraping...

I really don't think scraping should fall onto that list.

There isn't even a consensus in the IT world whether or not scraping should be able to be legally restricted.

stevefeinstein 3 days ago 2 replies      
So again someone wants to punish all the legitimate people using a web site to get some marginal benefit from detecting the remaining <1%. The inevitable false positives don't affect the "malicious" users. Only the legitimate ones.And how much will this bloat the page load by? Adding more code to an already overly large page isn't helping anyone.

Just let the web be the web, and stop trying to control it.

JoshTriplett 3 days ago 2 replies      
This looks like a list of bugs that need fixing; ideally, headless Chrome should be completely indistinguishable from ordinary Chrome, so that it gets an identical view of the web.
josteink 3 days ago 1 reply      
> Beyond the two harmless use cases given previously, a headless browser can also be used to automate malicious tasks. The most common cases are web scraping

I guess I disagree with the premise of this article.

How is web scraping fundamental malicious?

What rights/expectations can you have that a publicly accessible website you create must be used by humans only?

sorenbs 3 days ago 0 replies      
Leaving aside for a moment that many "malicious" use cases are actually fairly common and totally legitimate.

Headless Chrome is awesome and such a step up from previous automation tools.

The Chromeless project provides a nice abstraction and received 8k start in its first two weeks on Github: https://github.com/graphcool/chromeless

fforflo 3 days ago 1 reply      
Since when is web scraping a "malicious task"?
XCSme 3 days ago 2 replies      
If someone wants to scrape your site he will do it, just find workarounds against your "protection". It is impossible to tell the difference between a real user and an automated scrape request, you can only make their job a bit harder.
tyingq 4 days ago 1 reply      
I wonder how many of these were deliberate, and how many were missed. Google has a vested interest in bot detection.

And by releasing headless chrome, they killed off some of the competition. (https://groups.google.com/forum/#!topic/phantomjs/9aI5d-LDuN...)

PascLeRasc 3 days ago 2 replies      
I don't want to start an argument here, but can someone explain why web scraping is considered malicious?
tomatsu 3 days ago 0 replies      
> var body = document.getElementsByTagName("body")[0];

You can just use document.body.

I also suggest to use a data URL instead. E.g. "data:," is an empty plain text file, which, as you can imagine, won't be interpreted as a valid image.

 let image = new Image(); image.onerror = () => { console.log(image.width); // 0 -> headless }; document.body.appendChild(image); image.src = 'data:,';
> In case of a vanilla Chrome, the image has a width and height that depends on the zoom of the browser

The zoom doesn't affect this. It's always in CSS "pixels".

netsharc 4 days ago 1 reply      
Shouldn't the first block of code have "HeadlessChrome" instead of just "Chrome" as the search term?
tscs37 3 days ago 0 replies      
I do hope that these methods get patched, I tend to archive my bookmark collection with chrome headless to prevent loosing content when such a site goes offline. I hate it when a website requires me to play special snowflake to scrape them for this purpose.
jdc0589 2 days ago 0 replies      
dumb question from someone who's written a ton of scrapers and scraping based "products" for fun:

at what point does it make more sense for companies to just start offering open APIs or data exports? Obviously it would never make sense for a company who's value IS their data, but for retail platforms, auction sites, forum platforms, etc... that have a scraper problem, it seems like just providing their useful data through a more controlled, and optimized, avenue could be worth it.

The answer is probably "never", it's just something that comes to mind sometimes.

skinnymuch 3 days ago 2 replies      
How many of these can be faked with some additional code with Chrome headless?

Regardless as others are saying, using complete Chrome or Firefox with webdriver solves all these, right? Is there a way to detect the webdriver extension? That's the only difference I think from a normal browser.

revelation 3 days ago 1 reply      
The irony of using JavaScript to detect scraping or bots when the majority of them not used to trick ads don't ever execute any of it because they are a better curl.
askvictor 3 days ago 0 replies      
All of these could quite easily be overcome by compiling your own headless chrome. It wouldn't surprise me if there is a fork to this effect soon.
DannyDaemonic 3 days ago 0 replies      
I'd be willing to bet that missing image size variance is more of a bug or oversight, and is something that will be fixed.
hossbeast 3 days ago 0 replies      
"Beyond the two harmless use cases given previously, a headless browser can also be used to automate malicious tasks. The most common cases are web scraping, increase advertisement impressions or look for vulnerabilities on a website."

Cheating an advertiser I'll grant you, but the other two are 100% legitimate.

userbinator 3 days ago 0 replies      
Those who want a more "authentic" experience would do better to use a real normal browser, and control it from outside.
assafmo 3 days ago 0 replies      
"... a headless browser can also be used to automate malicious tasks. The most common cases are web scraping... "

Since when web scraping considered malicious? Companies like Google are doing billions because they use web scraping.

codedokode 3 days ago 2 replies      
What about mining cryptocurrency on a page load as a solution against scrapers?
fiatjaf 3 days ago 1 reply      
Isn't it possible to detect a bot by tracking some events like random mouse moving, scrolling, clicking etc.? Why weren't these kinds of detection tried in place of captchas, for example?
asveikau 3 days ago 1 reply      
The Kolmogorov option scottaaronson.com
363 points by apsec112  8 hours ago   310 comments top 29
habitue 7 hours ago 9 replies      
Let's say this is about the Google memo. And let's say, for the sake of argument, you're a person who thinks Damore had some good points and some bad points but you think the hysterically censorious response to him was way over the line. But you don't want to become a pariah yourself, so you stay quiet about it. The argument Damore was making was fiddly, kind of subtle and takes a long time to explain, it's not worth the trouble you're going to get into. You take the Kolmogorov option and decide to wait out this insane time period.

Only it turns out, when you don't decide to argue for that subtle and qualified defense of Damore, a bunch of alt-right internet trolls make some terrible fallacious defenses of things he didn't say. Suddenly, the original censorious instincts seem much more righteous and justified. After all, "Now there are only full-throated red-pillers arguing in Damore's defense! We were right all along!"

Now there are two sides to this issue, and they're both identity politics and brain-dead shouting. Because no one stopped and offered a third option: actually discussing his argument, acknowledging where he was right, and discussing what he got wrong.

tpeo 7 hours ago 6 replies      
Besides ideological self-righteousness, Aaronson left out one important component of oppressive regimes, which I think is actually far more important: it takes a relatively large amount of people to topple a regime, while small groups as well as individuals do nothing but expose themselves by rebelling. So no individual with any regard for his own personal safety has any incentive to rebel, and will instead go along with the flow for as long as it's tolerable to him.

But this, on the other hand, would create an odd situation where actually the vast majority of people might actually wish for rebellion, but none of them actually acts out on that wish. Which I also think is actually much more likelier than what he's putting forward. True crusaders are rare, most of people are "just following orders".

wisty 7 hours ago 1 reply      
A bit of game theory might be useful - https://object.cato.org/sites/cato.org/files/serials/files/c...

Riots and revolutions are difficult to start because of a number of obstacles:

* It is not clear how many sympathizers you have.

* Even if you believe you have the numbers, being the first to act may result in you being made an example of by the authorities.

It's smarter then to rely on more ambiguous signals. Playing stupid is a good way to do this. The problem is, the people who want to "stick it to the man" are generally young people who still process emotions with their amygdala - they can't keep their ego in check and pretend they don't realize they are contradicting the official doctrine.

It's fun to think you can be like Martin Luther, nailing The 95 Theses to the church door. We only hear about Martin Luther because of a quite literal survivor bias. Also, Luther survived because he claimed to not be challenging the Vatican's authority, but simply having a scholarly debate. He was also quite lucky that the secular authorities were lenient (perhaps for political reasons - there were lessor nobles who weren't so keen on the Holy Roman Emperor's authority).

softbuilder 7 hours ago 1 reply      
What I find myself wanting to fight is not the prevailing views on a topic, it's the willingness to lie and mislead to promote those views. I saw that this past week with the Google memo situation. I also saw a shocking amount of it last year with the US election. Sloppy journalism feeds the people who seem to want to believe the worst in everything, who in turn feed entire social networks. Eventually sufficient people have heard a thing that the network as a whole believes it. Individuals have little influence. I suppose public consciousness (and social media in particular) could be thought of as a big NN.
mseebach 7 hours ago 3 replies      
I don't know anything about Kolmogorov, so I can't evaluate the assumption that he obviously saw through the lies and was horrified by the brutality - but I will note that a great number of prominent intellectuals, who were in absolutely no private danger from the soviet regime whatever they'd believe and say, and who definitely had access to information about the brutality, who still chose to support and defend the regime.

Without evidence to the contrary, it's plausible that Kolmogorov simply supported his government and found the brutality to be acceptable collateral damage.

If we've learned anything from the 20th century, one thing should be the lesson that even very smart people can be found supporting extreme brutality in support of an ideology.

guillaumec 7 hours ago 2 replies      
I really enjoyed reading the The Gulag Archipelago by Aleksandr Solzhenitsyn about the state of mind of the population during Stalin leadership. He relates the story of a conference of the party where nobody in the room dared to be the last one to stop applauding in honor of Stalin.

After a while everybody started to realize how stupid the situation was, yet they couldn't stop clapping because they all knew that this would be a death sentence for the first one to do so.

jfoutz 7 hours ago 1 reply      
My only complaint is the 'truths everyone knows' but doesn't say. Most people just don't care. there are a few thousand people that think deeply about worldview A, and a handful of others that think deeply about worldview B. Everybody else is worried about milking their cow, or pilgrimage to the holy land, or picking the kids up from soccer practice.

When worldview A is dominant, problems are framed in terms of A. It's literally hard to have those 'hey, that's funny' moments. And when I do, it's easy to dismiss them. I haven't thought as deeply as the worldview A people, i'm probably forgetting some minor detail. Worldview B people are trained scientists, who spend years honing the 'hey that's funny' detector. I can't compete with that.

Maybe i'm weird. I don't think so. I've had my insights about programming problems, but there are so many topics where i'm simply ignorant. There are so many subjects where i'm a fish, i can't see the water.

The author brought up sexual orientation. I am not a human sexuality researcher. I don't care to be one. My information comes from the culture i live in. I was incredibly happy for friends that were able to marry. If i lived a hundred years ago, i strongly doubt i would hold that same view.

I agree with the author, because culture creates defaults. Push culture when you can. Make better defaults. Most people don't change the factory settings. They are lazy. Or busy. Or really just don't care.

scandox 4 hours ago 3 replies      
I want to say this as respectfully as I can, but it isn't easy: the quality of discourse on these topics is really low on HN.

At the same time I do think it's really important for them to be discussed.

Time and again, though, I'm disappointed by how reductive and unmeditative these discussions are. I'm also surprised by how angry people seem.

This is a site used by a lot of very smart people, but I think there is an element of people transferring the confidence of their domain expertise from one area into another, without noticing that they've crossed that border.

I think people need to approach these topics with a humility they wouldn't need to bring to say, hashing algorithms or distributed databases.

edit: typo

KKKKkkkk1 14 minutes ago 0 replies      
I'm curious about the remark that Kolmogorov was gay. This is the first time I hear of this. As far as I can tell from Wikipedia, this claim first surfaced in an article at The Mathematical Intelligencer from 2001. Are there any accounts from his contemporaries that would confirm this?
smackay 5 hours ago 1 reply      
One problem with creating an island of reason in what is apparently a sea of chaos is that the values that are cherished: reason, truth, rationality and merit are quite possibly the very things the regime is trying to suppress - or at least use a fuel for their outrage in order to promote their views and desire to be seen and hear at the expense of everything else.

The problem I see here now is that the regime is much more distributed with no apparent central authority to work against, however quietly. Comparisons have been made with the Soviet Union, the Vatican and the Nazis but they were all based on an ideology. Social outrage, social signaling and self-righteousness are more fundamental characteristics of humans' group behaviour than politics and for that reason it will be much harder to oppose and even harder to change.

A more apt example might be the effort it took/will take to topple slavery - which despite the change won in the Americas still unhappily exists today in other forms. Other fundamental beliefs can be so deeply rooted in culture that they persist for many hundreds of years and change, even it if is superficial, can take a very long time.

jancsika 2 hours ago 2 replies      
Two things:

* for a society that enjoys even minimal levels of freedom of speech, the Kolmogorov option isn't easily distinguishable from opportunism or careerism.* for a society like the U.S. that enjoys minimal levels of freedom of speech, what is an example of an unspeakable truth? Charles Murray's research exists. Brendan Eich is still rich. Richard Stallman has an extremely unpopular opinion about child porn on his website and he still does talks to people who worship the ground he walks on.

Frankly, it seems like the height of narcissism for a tenured professor to defend not speaking out against Trump by appealing to Kolmogorov.

It would be so refreshing to see the author instead appeal to Occam's razor and just call it cowardice.

Edit: grammar

fnord123 5 hours ago 1 reply      
First they came for the Socialists, and I did not speak outBecause I took the Kolmogorov option.

Then they came for the Trade Unionists, and I did not speak outBecause I the Kolmogorov option.

Then they came for the Jews, and I did not speak outBecause I the Kolmogorov option.

Then they came for meand there was no one left to speak for me.

-With apologies to Martin Niemller.

abecedarius 59 minutes ago 0 replies      
There's an interesting book about that dynamic where some unknown number of people privately doubt a core public value -- "preference falsification": https://www.amazon.com/Private-Truths-Public-Lies-Falsificat... I read it after the 2016 election, but it's pretty old -- is there newer work anyone can recommend?
thiagoharry 3 hours ago 0 replies      
"As far as I can tell, the answer is simply: because Kolmogorov knew better than to pick fights he couldnt win."

This is documented in some place and represent Kolmogorov views, or the author is just projecing it's own views and opinions? Because kolmogorov could just have being neutral about the politics in its country (like the majority of people) or perhaps even had some degree of support at the time. This opinion "its obvious that the soviets were E-V-I-L" are opinions coming from the other side of the cold war.

Tossrock 6 hours ago 0 replies      
Very similar to pg's "What you can't say" essay - http://www.paulgraham.com/say.html
TheAceOfHearts 4 hours ago 1 reply      
This quote comes to mind:

He who knows only his own side of the case knows little of that. His reasons may be good, and no one may have been able to refute them. But if he is equally unable to refute the reasons on the opposite side, if he does not so much as know what they are, he has no ground for preferring either opinion... Nor is it enough that he should hear the opinions of adversaries from his own teachers, presented as they state them, and accompanied by what they offer as refutations. He must be able to hear them from persons who actually believe them...he must know them in their most plausible and persuasive form. John Stuart Mill, On Liberty

tristram_shandy 7 hours ago 1 reply      
The difference, of course, between Galileo and Kolmogorov is that people are aware of Galileo. Acquiescing keeps intellectuals comfortable, but that's about all you can say for it.

There's been a few of these articles linked on HN recently, including Paul Graham's original http://www.paulgraham.com/say.html, with a more or less identical thesis (be quiet) that camouflages real message: that the author is signalling support for the heresy, carefully.

zem 7 hours ago 1 reply      
The fact that some geniuses were laughed at does not imply that all who are laughed at are geniuses. They laughed at Columbus, they laughed at Fulton, they laughed at the Wright brothers. But they also laughed at Bozo the Clown.

-- Carl Sagan, "Broca's Brain"

minipci1321 6 hours ago 0 replies      
I think in his intrepretation of the Kolmogorov's life, Aaronson might be totally missing the evolution of views of someone born in 1903 in Russia and having lived through what followed. It is not like Kolmogorov was parachuted overnight from some free country into the oppressive regime.
macrael 7 hours ago 14 replies      
Tell me, truly, is this a view held by many here? That, as the church denounced Galileo's observation that we turn around the sun as heretical, so too our modern PC culture suppresses legitimate inquiry into wether gender is a determinant of programming aptitude? This is a comparison worth drawing?

Galileo was a scientist. He published papers, books, treatises. He devoted his life to the pursuit of the truth, found a big piece of it, and was punished by society for it. He is one of the giants whose shoulders we stand on.

Kolmogorov sounds cut of the same cloth. He built a research center, helped other scientists on the way up, and left a legacy.

James Damore is a junior programmer who wrote a 10 page 'manifesto' accusing his colleagues of having inferior genes. This created a textbook hostile work environment,[1] leading to his firing.

The amount of sympathy I've seen here is dismaying. It is illustrating just how far we have to go until equality is the rule of the day. Please, before you throw in your lot with him, consider how ridiculous the analogy OP made here is. The Google Manifesto is manifestly not A Dialogue Concerning the Two Chief World Systems.

I will say, some of the heavyweight commenters here do give me hope. tptacek linked this graph[2] on one of the early threads on this. It's a pretty solid rebuttal to any and all concerns that women are innately unsuited to computers rather than that our computer culture has driven them away.

[1]: https://medium.com/@scurphey/googles-response-to-employee-s-...

[2]: http://www.npr.org/sections/money/2014/10/21/357629765/when-...

myth_drannon 6 hours ago 1 reply      
I'm surprised no one is mentioning Jordan Peterson and the attacks on him. He is quite successful in defending himself speaking his truths.
thanatropism 4 hours ago 0 replies      
Yeah, but many of us, even if aspiring to be our best, will never reach the importance of a Kolmogorov, or a Terry Tao, etc. Our optimal mix of political iconoclasty and mathematical progress may be much much different.

There's a sense in which way the ideal mathematician (or ultramarathoner, or classical composer...) is like a monk. Many of us are not built like that.

red_admiral 4 hours ago 1 reply      
> This is where you build up fortresses of truth in places the ideological authorities dont particularly understand or care about

Isn't this exactly what safe spaces were originally meant to be?

hyperpallium 8 hours ago 4 replies      
What's the hinted present-day ideology he dare not challenge?
aatchb 7 hours ago 2 replies      
So he's willing to concede space on the scientific landscape in the face of increasingly oppressive 'political correctness'? Don't research something if it may hurt someones feelings! Great.
yaacov 8 hours ago 0 replies      
Looks like his blog's down? Guess I'll try again in the morning

Edit: seems to be working now

jdoliner 7 hours ago 2 replies      
> if I dont understand what is or isnt hurtful, then Ill defer to the leading intellectuals in my culture to tell me

This essay was so promising, and then turned out to be a pledge of allegiance to the thought police.

frinxor 7 hours ago 0 replies      
Good read - thanks
igravious 6 hours ago 1 reply      
Quick question.

Every mathematician believes that he is ahead of the others. The reason none state this belief in public is because they are intelligent people.

The others here, does it refer to other mathematicians, or other people in general?

The BitTorrent Protocol Specification v2 bittorrent.org
311 points by Nekit1234007  1 day ago   107 comments top 9
jzelinskie 1 day ago 5 replies      
This update to the spec is a modest change that's largely a preemptive reaction to SHA1 being broken; large portions of BitTorrent are designed around the 20-byte length of a SHA1 checksum. They've decided to move forward with SHA256 truncated to 20 bytes to avoid incompatibilities with existing infrastructure such as the Mainline DHT.

Beyond the hashing algorithm, some important additions that were previously proposals without widespread use (e.g. merkle tree for hashing pieces) are becoming required. The focus has mostly been on optimizing latency for the P2P protocol and making sane improvements to the file spec. I feel like trackers were largely overlooked in this update, but I'm biased because I work on a popular tracker.

Ideally, BitTorrent would be broken down into separate specifications that could be used together or in separate systems: one for the file format and piece representation for sharing files, one for the P2P protocol, and one for discovery (trackers, DHTs). I want to believe that there would be far more interesting P2P projects if you could just lift robust primitives from BitTorrent.

Scaevolus 1 day ago 4 replies      
1) Chunks don't span files. Each file is validated by the hash of its merkle tree. This is the biggest user-visible change, since it means you can download one file without downloading others.

2) SHA1 is replaced with SHA2-256 (2x longer hashes and not broken).

3) Files are represented by a tree structure instead of a list of dictionaries with paths-- this reduces duplication in deeply-nested hierarchies.

4) Backwards compatible-- you can make a .torrent file with both old and new pieces, and a swarm can speak either. This requires padding files from BEP47, which most clients probably don't support.

Per-file metadata increases pretty significantly, from ~19B (just length) to ~68B (length + hash).

Klathmon 1 day ago 6 replies      
I have to admit, BitTorrent is one of the things I took for granted.

I never really thought about the details of how it works, or the really really impressive feats that were accomplished to get it to work. I knew it was a really good technology, but reading this and the comments here puts it on a whole other level.

Why isn't this technology talked about more? Why are blockchains the big "thing" right now with people trying to use them everywhere to see where they fit best, but torrent networks are kind of just... ignored?

The decentralized nature of it seems to open so many possibilities at first glance, is there a reason they aren't being taken advantage of? Is there some kind of "great filter" kind of thing that is preventing widespread usage of something like a torrent network?

richdougherty 1 day ago 0 replies      
redm 1 day ago 1 reply      
I just don't see this technology ever going mainstream. I first deployed this type of application in 2003. It was named Redswoosh and did effectively the same thing as BitTorrent, just in a closed client. I was also a very early adopter of BitTorrent using it personally.

Users hated it for general use, even when downloading big files. 1) They didn't like having to install/run some special software to download a file. 2) They didn't like the effects of uploading to others and it slowing down the connections.

Consumer networks are asymmetric having far more download capacity in upload capacity. This makes sense since 1) most users download and want to use the available bandwidth for faster downloads, and 2) it prevents commercial applications on consumer circuits. This is far from ideal for applications like BitTorrent.

I'm not saying there isn't an application for this technology, I'm saying all the good applications don't want to ask the users to pay for distribution to other users. Thus it's relegated to mostly piracy, open source, etc.

Bittorrent Inc. has been trying to commercialize this for a decade now, I just don't see it happening. If there was anyone who could commercialize it, it was Travis Kalnik, and while he exited for 20m, he was very lucky, (and happy) to get out of that market.

0x0 1 day ago 1 reply      
What's the stuff about "proof layers", is that new in this v2? The paper briefly talks about proof layer requests. Is this something merkle-tree related? What is the purpose? Is it to prevent clients from lying about having pieces they do not have by requesting a verifiable random hash chunk?
lowglow 1 day ago 2 replies      
Can someone diff the spec from the previous version? What's the changelog? :)
shmerl 1 day ago 1 reply      
Do all Bittorrent clients support it already?
smegel 1 day ago 2 replies      
Pity we will never see a genuine version of uTorrent that will support it. That was a real loss.
Prostitution decriminalized: Rhode Islands experiment newsworks.org
311 points by MaysonL  2 days ago   244 comments top 27
andrewstuart 2 days ago 9 replies      
It's weird seeing this. In Australia prostitution is completely legal. There was actually a publicly listed brothel for a while. Strange to think other societies criminalise it.

I think being legal is best for the health and wellbeing of the sex workers and minimises the criminal involvement.

There are brothels all over the place but mostly they are pretty low key, often hidden in backstreets or warehouse areas with only a red light and a sign to indicate they exist.

I wish we had such an enlightened position on drugs, which remain almost completely criminalized, whilst other parts of the world move towards legalization of various aspects of drug law.

Iv 2 days ago 4 replies      
Either as a society we accept that some people will trade sexual favors for money or we refuse it. I can see arguments both ways. In the end, that is a stance on how you weight two values:

1. The morality to add financial consideration in the decision to provide consent to a sex act.

2. The amount of freewill that a person has within the job market.

What is interesting is that you can have different liberal or conservative position on both of these considerations and still be unsure about the side the scale weights in:

As a conservative:

1. Social conservatives tend to consider sex like a serious matter, set apart from the rest, so they will say that consent is not to be sold.

2. Fiscal conservatives tend to consider freewill in the job market is close to absolute: you are always free to refuse a job, so people deciding to go into prostitution really chose it without constraints.

Depending on how you weight sex-is-special vs job-market-is-free, you can be against or for prostitution as a conservative.

As a liberal:

1. Social liberals will tend to consider that consent is important in every part of life but that sex is (or can be chosen to be) an activity like any other. If someone is really willing to sell sexual favors, under what principle forbidding it? (and it turns to be a fetish too)

2. "Fiscal liberals" (is that an expression? Not sure, non-native here, I mean the opposite of "fiscal conservative) tend to consider wages and employment to be a tool of oppression and control. In that respect, they will be less likely to consider that someone who works for money chooses to do so freely.

Depending on how you weight you-are-free-to-sex-as-you-want vs wage-is-a-way-to-control you can be a liberal and be for or against prostitution.

geff82 2 days ago 4 replies      
As a German where prostitution is just seen as a regular trade, some general points.1.) While paid sex is surely available at many places, we still do not live in one big brothel.

2.) Why should people have to film their act of having sex and then publish it (porn is legal in the US...)

3.) There is a problem with human trafficking that has to be dealt with. But dealing with this problem in some manner is surely the better way than making prostitution illegal. As long as there are hormons in this world, prostitution will exist. Learn to live with it.

tryingagainbro 2 days ago 3 replies      
I never understood this: it's your body so you have the right to an abortion, but...

Someone does it for free, someone for a promise to marry, someone for marriage, someone hoping it leads to something, rent paid etc etc. Who are to decide what "currency" is legal and what isn't for "her body"?

kolbe 2 days ago 1 reply      
I've always thought about how brutally sexist it is to take this industry where women have a massive competitive advantage, and criminalize it.
RealityNow 23 hours ago 1 reply      
Summary of results from prostitution decriminalization experiment:

* Female gonorrhea rates decreased 40%* Reported rape offenses decreased 30%* No evidence that decriminalization increases human trafficking

> She has a theory, though, in that while she knows for some men rape is about power, "I think the argument that we're making is that that might not be true for all men, and for some, these activities could be substitutes."

> In other words, for some men, rape may be just about sex. And if there's a legal and accessible market for it, the number of rapes in a community may go down. This has not been a popular theory or study. And for many, it challenges the notion that rape is about violence and power, and not sex. "So I consider myself a feminist, but I think this finding angers a lot of feminists," Shah said. "It is a very controversial idea."

What the hell is this "rape has nothing to do with sex" nonsense and why does challenging this absurd assertion anger feminists? How removed from reality do you have to be to believe that forcing sexual intercourse on someone has nothing to do with sexual attraction?

But back to the topic of criminalization, it's downright disingenuous to conflate prostitution with human trafficking. And I don't believe the government should be in the business of imposing its moral whims on the decisions of consenting adults.

wcummings 2 days ago 1 reply      
Rhode Islander checking in. I'll share a story I heard from someone downcity many years ago, before this law was reversed in '09.

Buddy's (7-time wiseguy mayor of Providence, and 2-time convicted felon) usual driver was unavailable, and a detective was assigned to escort him. The mayor gives his new driver an address. The detective recognizes it, bit can't quite place it, and gets on his way.

When he gets there, it hits him. "Buddy, this is the address of a known brothel".

Buddy quips back "you must be a detective" and disappears into the building.

cestith 1 day ago 0 replies      
Banning anything that has a high demand mainly gives criminals control of the market. It has little impact on demand but is an abrogation of any responsibility to regulate.

The same was true of the US 18th Amendment, of the "War on Drugs" in the US now, of the banning of sex work, or most anything else. When the government decides to just ban something, others step in to control the market without any rule of law or responsibility to the public.

jessaustin 1 day ago 0 replies      
TFA: Still, perhaps even more surprising than the decrease in gonorrhea was another public health development. Sexual violence, or rapes, dipped dramatically. And this wasn't just amongst sex workers. It was across the board, according to FBI crime reports and jurisdiction level data.

"Reported rape offenses decreased by about 30 percent," Shah said.

That's another big decrease. Shah says, if anything, you'd expect rape to go up as when prostitution is decriminalized, sex workers are more likely to report rapes. She compared this to neighboring states, too. The drop was only in Rhode Island. So she examined other crime data in Rhode Island, like burglaries and murders, to see if there had just been a drop in crime generally.

It didn't match.

vitro 4 hours ago 0 replies      
I highly recommend the document Whores' Glory by Michael Glawogger which gives you a different look on what prostitution can also be like.
jaemison 2 days ago 6 replies      
I love how in the past hour, a NYT op-ed discussing motherhood and women's agency is flagged as irrelevant but an article on decriminalizing prostitution is at the top of hacker news with a lengthy discussion on the nuances of policy and practice /s
jacknews 2 days ago 0 replies      
I neither endorse nor reject this paper, but it is an interesting argument:http://assets.csom.umn.edu/assets/71503.pdf
spodek 2 days ago 0 replies      
Does anyone else see the article presenting for a mutually agreed on act as women innocent / men predatory?

For example

> The Swedish or Nordic model takes aim at the demand, making it illegal to buy sexual services but not to sell it.


> In a class action lawsuit in the U.S. District of Rhode Island, COYOTE's lawyer argued that the state's law on prostitution was too broad and discriminated against women, given that female sex workers were arrested far more than male customers.

Saying "takes aim at the demand" for a mutually consenting interaction just seems a way of targeting men without explicitly saying it. Different countries and situations, but the reporter doesn't note the different treatment by the state when one sex is criminalized versus the other.


> "You see the silencing of victims," Hughes said. "It is just very harmful to women. It really is a libertarian approach, but the ones who get freedom are the pimps, the sex buyers, the businessmen who then can rent properties to the massage parlors, and to the sex buyers. There's very little freedom for the women."


> As for Robinson, the online escort who moved to Rhode Island? She was angry.

> "It pissed me off. I didn't know nothing about activism, I didn't know what a sex worker rights organization was," she said.

> Robinson changed that. She became active with COYOTE, the sex worker union and advocacy group.

> "Criminalization is a punishment for women who won't conform," Robinson said. "And we're just supposed to go live in the streets in poverty and not complain about it, and be good women."

Nothing wrong with a woman's perspective, but men's behavior was criminalized too. I would think it would add to the article to get some of their perspective.

Not including men's perspective, except as the businessmen who make money off the women and the rapists, makes the article sound like the reason for prostitution's negative reputation is men. Would a quote like the following fly?

> "Criminalization is a punishment for men who won't conform. We're just supposed to go live without sex and not complain about it, and be good men."

k__ 2 days ago 0 replies      
It's legal in Germany too.

My girlfriend grew up in a red light district.

She said it was mostly okay, the nights were a bit rough, because the clients would often argue loudly with the girls right before her window.

But the "madame" of a brothel around the corner would always look after the children going to school in the morning so they wouldn't be bothered.

the_mitsuhiko 2 days ago 0 replies      
As someone from a country where all form of prostitution is legal and regulated it did not really ever come up as something odd when I grew up. Prostitution is becoming less prominent in recent years because families move into places traditionally frequented by prostitutes but the industry is healthy as ever as far as I can see.
plainOldText 2 days ago 0 replies      
Speaking of various models of legislating sex, I found this TED talk quite informative: https://www.ted.com/talks/juno_mac_the_laws_that_sex_workers...
randyrand 2 days ago 1 reply      
off topic: This is why states rights is so important. Each state acts as an incubator for ideas. Its 50 experiments running all at once.
ilaksh 2 days ago 3 replies      
Do you guys think that the majority of sex workers really enjoy their work or prefer it? Do you think that this is the career they prefer? Do you think it may have some affect on their psychological well-being?

I believe prostitution is an indication of a lack economic and social well-being both in the case of buyers and sellers.

Within about one or two decades maximum there will be revolutionary advances in robotics including things like much better weight-to power ratio artificial muscles and increased bio-mimicry. This will lead to truly life-like sex robots. Along the same lines, the majority of "ordinary" jobs that humans can currently do without high-bandwidth connections to computers or control over them will be replaced with sophisticated AIs and robots or automation. The perspective on sex work and general objectification of women will shift more towards one side then.

forkLding 2 days ago 7 replies      
I wonder if government-operated brothels could be a better solution, this way there is a constant monitor on sex workers and human trafficking while providing extra income to the government and being able to protect the sex worker/citizens its meant to protect.

Not sure what would be the obvious cons. Happy to know if anyone points any out.

Corrado 1 day ago 0 replies      
I just can't understand how people think that you can regulate sex between consenting adults. Did we not learn anything from prohibition? Trying to make something that almost everyone wants illegal is just not possible. Hell, we can't even enforce a speed limit on the highways.
stretchwithme 1 day ago 0 replies      
I think the best way to handle prostitution and recreational drugs is to let each neighborhood decide what is allowed. If the majority of residents and majority of property owners are okay with a particular activity, it should be legal there.

There are always going to be problems allowing these activities and problems with banning them. If you let each neighborhood decide, at least you're moving the problem to areas that are ready and willing to deal with the problems. They can tax it, regulate it, provide security, drug treatment etc. That means the neighborhood has to form some kind of legal entity, of course.

And you're giving the customers reasons to go where they are welcome. By doing it on a neighborhood level, it becomes much easier to comply. It's easier to drive 15 blocks than to go to the next county.

And I think enforcement where it is banned can just be done with fines, giving people a good reason to take their business to where it is welcome. There's no reason to put people in prison.

MachineMan 2 days ago 0 replies      
A fitting phenomenon for the "Red Island" :)
lngnmn 2 days ago 2 replies      
Look no farther than some South Asian societies. Countries with no madness around prostitution are better off socially.

It is not an accident that Thailand is a Buddhist country - people are tolerant to whatever is not harming others living beings.

Too much pressure on sexual issues is a sign of primitive authoritarian societies (who are cooksure that they know better what is right and what is wrong than biology and evolution).

muse900 2 days ago 0 replies      
prostitution is probably the oldest profession.
cuckcuckspruce 2 days ago 1 reply      
nvahalik 2 days ago 1 reply      
Honest question... for those of you on board with this: how do you reconcile the complaints about the objectification of women as sex objects (and how that harms women) with this?
bitwize 2 days ago 1 reply      
Legalizing prostitution increases human trafficking:


The correct approach is to decriminalize providing, and crack down hard on buyers.

How to Pick Your Life Partner (2014) waitbutwhy.com
350 points by Tomte  1 day ago   204 comments top 20
geff82 1 day ago 3 replies      
You should also read part 2 of the article.

After some relationships from 18-28 that all ended unhappy, I decided to screen my next partner in a very rational way. Of course I wanted romance and some attractiveness out of my next girlfriend, but I primarily wanted a friend that I could talk to when we were 95 and sitting in wheelchairs next to each other, unable to do much else. Destiny called and the girl I then met and found worthwhile to date further became my wife and mother of two children.

Why I reference the second part of the article: we just talked about it and totally agreed that our marriage was more or less how it should be (as described in part 2) and felt not at all touched by part 1.

While we are for sure people that are different, we usually find new stuff we absolutely want to do together. Last year we did boat- and radio licenses (5 so far...), this year we began pistol shooting in a sports club, we travel to many places, have tons of family meetings (I think being in a supportive family helps a lot, too)... yes we also have the occasional fight, but there has been exactly no occurence where it had not been completely resolved before sleeping.

So being rational about your partner does not mean to stop the fun, stop the romance, but means you know exactly what you get into and care for the happiness for you own life.

Remember: a divorce is often the complete destruction of your life dreams. You can destroy yourself and/or your partner. Your partner REALLY gives the single life he/she has in your hands and hopes, like you do, that everything will go right. Don't be a fool when given such a treasure.

mikebenfield 1 day ago 10 replies      
1. I have always hated the notion that having a happy marriage is about filtering through many candidates to find the one perfect partner. Although I do think it's reasonable to try to learn early on what you're looking for, this idea that we should all be dramatically expanding our dating pool to find the perfect person is abominable.

Look at a city with a massive population of eligible young people, like say NYC.. and then listen to what people in NYC say about the dating scene there. It's not pretty. Having such a huge pool of potential mates discourages people from committing. In other words, it makes it harder to find a long term mate, and it makes you less satisfied with whoever you're seeing.

2. Biology isn't the only reason a person might want to get married before 40. There's something to be said for sharing your early adulthood with a person. Meeting your life partner at 37 will be a very different experience from meeting your life partner at 22. Not necessarily a bad experience, but not what everyone is looking for.

twic 1 day ago 1 reply      
The article attempts to presents a framework for making this decision rationally, but ultimately, it's the heart that chooses between FreeBSD and OpenBSD.
roceasta 1 day ago 2 replies      
I like the way this article flips the commonplace 'What are the criteria for selecting my perfect mate?' to 'What are my motivations for getting married?'.

However, it needs to be said that if and when children arrive on the scene, everything changes. So really, in my opinion, the predominant issue is 'Do I want to have children and how do I want them to be raised?'

lordnacho 1 day ago 1 reply      
What I always tell people who are dating, and only half joking, is you need to find someone you can be bored with.

When you're young and just done with education, starting work, things are changing quickly. You're getting a lot of new experiences and meeting new people. It can be very exciting.

Unfortunately, it's unlikely to continue like that. You can't always be travelling, changing jobs, or even go out on the town. This is primarily if you're going to have kids. They need a routine, which means you need a routine.

So the challenge is to do the exciting dating thing while looking for someone who will be happy with something much more monotonous. I met at least a couple of women who seemed likeable but just didn't act like they were going to be happy on the long straight.

fizixer 1 day ago 14 replies      
Marriage, monogamy, and (romantic) love are conservative notions that are close to being (or should be) marginalized in my opinion.

Romantic love, especially, is an extremely superficial and hollow idea exalted to high heavens by the group-think of the masses. I gave up religion, and over the years realized that love is the second, way more sneaky and no less damaging, delusion that humanity indulges in. It has to go the way of the religion.

I hope most of you would agree that at least 70%-90% of being in love has something to do with the appearance of your person-of-interest (most of it visual). No matter how you cut it, no matter how much of an intellectual you are and how much you prefer to connect deeply with a person, in the end, given two individuals, you fall in love with the one that is more visually attractive. What a coincidence!

And what is visual attraction? In other words why do we find some people more beautiful than others? Because we have a biologically ingrained form of discrimination based on looks. It's lookism plain and simple.

Note that I do not buy the silly notion that 'beauty is in the eye of the beholder' and that 'I don't find that person attractive but I'm sure someone else will'. This is somewhat backed by data (e.g., the OKCupid survey), but I have developed this belief on my own. Humans of all races have a built-in instinct to find some visuals more appealing and attractive than others. It is sad but true. To take it to an extreme, if we find some visuals disgusting (e.g., spilled guts, slimy things) we don't all of a sudden go "I'm disgusted by slimy spilled guts of animals, but that's just me, I'm sure there are plenty of people out there who have the exact opposite reaction, that they love to look at slimy spilled guts".

What I'm trying to say is that standards of visual attraction, while not physically objective, are largely biology objective, and yet, it's a bad thing, because it's a symptom of biological imperfection. It's a genetic defect in our biology and 99.99% of humans are afflicted by it.

I personally think social individualism is a concept that needs to gain traction in modern society. It is the 'atheism of love'.

Eduardo3rd 1 day ago 0 replies      
I very much enjoyed Part 2 of this article. It puts forward a much more complete framework for success than part one, which mostly just talks about the flaws in many standard approaches.

The notion that you are going to find the perfect person for the rest of your life at any given moment in time is ridiculous - you are going to change, they are going to change, your relationship is going to change both of you. The sense of friendship, being at home with one another, and the mindset that a marriage takes work have all been instrumental in my own marriage so far.

My wife and I met in our early 20s. The things we wanted out of life - both individually and as a couple - have changed significantly over the past decade. It is likely that if either of us had known where we would end up we might not have chosen our past selves to be together with our future selves. However, we've been a part of every individual and collective change and have worked to build a strong relationship through the changes. There is something amazing about walking through adulthood together and staying committed through all of the ups and downs.

tristor 1 day ago 2 replies      
This was a very insightful read for me, where I introspected and identified a lot of things I had done wrong myself in relationships. That said, it was also kind of depressing. It seems that one of the greatest unsolved challenges of life is efficient matchmaking, and despite all the technology poured into it we are nowhere close to having a good solution.

Swipe-apps (latest generation of matchmaking) are completely shallow and almost entirely based on appearance and no other factors which leads them to be used for either hookups or entertainment.

Profile-based apps lead to box-checking behaviors, incentivizing people to look for "scan-tron partners" as the article calls its.

Personalized matchmaking services are expensive, time-consuming, and are limited to the pool of people willing to invest that amount of time and spend that amount of money, which is not that many folks.

This is one problem area I'd love to see some new startups address in a serious manner.

blueyes 1 day ago 0 replies      
Finding the right life partner requires a series of filters. For me, those filters were: 1) quality of conversation; 2) ability to share work; 3) trust and reliability; 4) physical attraction. Not necessarily in that order, but long-term, the conversation and ability to collaborate with someone on the joint project of a household or family are way up there in what to look for. You definitely talk and work with each other a lot more than you moon and snuggle, so you should find someone you like to talk and work with if you want to like your life. It's really hard, though, because when you're attracted to someone physically or just desperate for sex it's a lot easier to fool yourself into thinking that their conversation is really interesting, when in fact you're just high on pheromones.
xupybd 1 day ago 3 replies      
Sounds great if you have lots of options. Some of us are getting older and haven't made it past anyone else's filter. At some point your will to just try and make it work regardless of the match.
Sleeep 1 day ago 0 replies      
>everyone looking for a life partner should be doing a lot of online dating, speed dating, and other systems created to broaden the candidate pool in an intelligent way.

Anyone ever heard of the paradox of choice?

All of the research I've read hints at relationships that start with a dating website being being much more fragile than relationships that have met more traditional ways and much more likely to end with a bad outcome.



One may argue this is good that we are ending "bad" or "not perfect" relationships... the other side of the coin is online dating may not be leading us astray from actually meeting someone we are compatible with and is just a distraction or worse, it may be making us bad at relationships once we met someone.

nomass 1 day ago 0 replies      
It was surprisingly hard to figure out the problem with these two articles.I mean it all makes sense, I couldn't stop nodding. And dont get me started with the picture from the stairs where the fighting couple stands below the lamentable single person. I mean: yes! And if that is not enough, our reading efforts are rewarded with a handy checklist for choosing the right partner.

I know the author might not happy about this, but I have an odd feeling (ha, shoot me). And the intensions they where, as they always are , the intentions where good. Nobody was hurt, no potential was spoiled, no time was lost. Everyone is happy. Sure there are conflicts and stumbling blocks, but with solid work and communication you disolve them into harmless bumps while sailing away on your Loveboat. Ready for the next chapture. Buying a house. Having kids. Growing older together. Die with a smile on your face surrounded by caring people.

If anything is overly romantic, than this. Completely ignoring the context you live in and reduce everything to a handfull of personal traits and skills is overly simplicistic (stupid) and naive, because:

- you cannot create love by communication, or transform friendship or nice feeling into love

- It is reasonable if you seek for true love and believe in fate (But only until you're 30)

- If you have been hurt, betrayed, used and left, take it as a sign to loath yourself less (and not others more)

- Life can be bitter. Love can vanish. Dreams can be shattered.

There is this song "Hello" from Adele, which is fairly popular. It describes somehow a state of love and connection after it is long gone. Something I'm not sure this author would even understand.

bluker 1 day ago 2 replies      
The 37% rule has been the most beneficial heuristic that I've applied to dating.


PrefixKitten 1 day ago 0 replies      
In order to pick someone you've got to have choices...
onuralp 1 day ago 0 replies      
Exploration versus Exploitation: Deciding When to Get Married - http://www.inference.vc/my_big_fat_data_driven_wedding_how_t...
djohnston 1 day ago 7 replies      
im in the honeymoon phase right now. sometimes i get sad thinking about the transience of this phase of the relationship. obviously i shouldn't fixate on that, but it's like this creeping twilight that i wish i could just stave off forever. anyone have tips on doing this?
graycat 1 day ago 0 replies      
What I saw in my own romantic relationships and those of others was a simple, blunt point: Do both of the people actually want to follow the first part of the standard marriage ceremony?

"We gather together to join this man and this woman with the bonds of holy matrimony."

Instead, one or the other didn't really want to "join" or "bond" and, instead, was just using, exploiting themarriage and spouse for some selfish aims.

For women, the main use was money -- she wanted to be supported. That was as far as she thought. Later one had one child, retired to a back bedroom, lived on chocolates, gained 150 pounds, and vegetated. Another had one daughter and later one by accident, retired to a back bedroom and lived on cigarettes and beer. One decided to become really good BFFs with Jim Beam and Jack Daniels. One really wanted just to do social climbing at the most expensive country club in town. Another wife wanted the spouse's wealthy family to fund a retail startup. When the startup failed, she went to college, then art school, then became a leading horse woman, all with no interest in her husband, and finally just moved out. She was very pretty, moved to NYC, apparently to try to make a big splash in media. Never heard from her again!

One husband liked sex with men as much or more than with his wife (at least two cases of that, with both of the wives very desirable); the wives didn't like that situation.

Another wife just wanted her husband to put her through law school at which time she planned to leave the marriage. Another wife just wanted her husband to put her through graduate school, at which time she planned to leave. Another wife wanted to use the marriage as a home base for various projects, around the world, to save the world.

I saw a lot of such problems: It was simple: One or both of the spouses failed right at the first sentence of the marriage ceremony -- they just didn't want to join or bond. Just didn't. To them the marriage ceremony was just a costume party with nothing serious and no connection with reality, and they wanted the marriage not to join or bond with their spouse but just to exploit them.

So, from this theme and these examples, what is in the OP is too complicated and misses the point: The point is, both spouses have to be serious all the way through the marriage vows, and failing right at the first sentence is really simple, much simpler than the OP, way too common, and a guaranteed disaster.

peterwwillis 1 day ago 4 replies      
> But good old society frowns upon that, and people are often still timid to say they met their spouse on a dating site. The respectable way to meet a life partner is by dumb luck, by bumping into them randomly or being introduced to them from within your little pool.

Well, yeah. This is how social networks work: similar people group together, people socialize within that group, and they match with people who they are similar to or find ideal. They don't need to search for a partner because their ideal partner is already somewhere within their social network.

People with poor social networking skills will not "bump into" potential partners because their network is poor. It's like expecting to find an investor for your startup if all you do is post to "Show HN". You might have to actually leave your forum, or computer.

And so, after eliminating all choices from one's existing in-person social network, people with bad social networking skills look for dates on the internet. This is an uncomfortable yet common-sense truth: much like marrying your cousin, there is a stigma for a reason.

There is currently no online service that can correctly predict if someone will enjoy your physical presence. You can check however many boxes you like and you will still end up being weird. This is perhaps one of the biggest reasons that there is a stigma against online dating: usually it does not work out. If online dating really worked you would only have to go on three dates before you met a perfect partner [that was also geographically and socially convenient to date]. But it turns out that online services are just another poor social network.


Aside from that probably-going-to-get-me-downvoted-to-hell-unpopular opinion: the author presents one view of how to pick your life partner, from one way of life. Depending on where you live, your personal values, your preferences, your goals, etc your ideal partner may look quite a bit different than the author describes.

Humans change over time. Your ideal partner right now may end up being a completely different person in 10 years. You can either accept this, or accept that your life partner is simply a "current partner" and that, similar to all other kinds of partnerships, it may not last forever.

The author is also conflating life partnership with marriage. Traditional marriage is a generally horrible custom that developed around the idea that one partner didn't have as much agency as another, and that the partnership is necessary so that the oppressed individual can retain some form of protection, usually in exchange for sexual or other reciprocation, and the relationship is almost always exclusive (for the oppressed partner).

You may want a life partner, but don't want to get married. This is increasingly the norm in modern western society (whereas before it might have landed you in jail or dead). You may want more than one partner, or your partner does. Non-monogamous relationships are becoming more common and better understood by the general public.

I'm not an expert, so I can't tell anyone how to pick a life partner. All I know is that to understand what you want in someone else, first you have to know what you like and don't like, for yourself as well as in others. Then you need to be able to express that coherently and immediately, without shame, regret, or miscommunication. Knowing how to do that will help you to search for and identify a potential life partner. But remember that it may not fall within the normal accepted standards of society, and you have to accept that, or you may indeed fall into the same trap that so many others have when they rely on a universal measuring stick to find the right fit. And above all, I think that if you can't find what you're looking for in your social network, you need to expand your social network - and not just via dating sites.

j45 1 day ago 0 replies      
I've learned to only get marriage or relationship advice from people who are in generally happy, healthy marriages and relationships.

Getting business advice from a bad business person wouldn't make sense.

vinum_sabbathi 1 day ago 3 replies      
im married and i read this

what a bunch of nonsense.

Deeplearning.ai: Announcing New Deep Learning Courses on Coursera medium.com
315 points by npalli  23 hours ago   85 comments top 21
seycombi 21 hours ago 3 replies      
For those who are unfamiliar with coursera or interested in just the videos (and NO certificate) you can enroll in "AUDIT" mode:

AUDIT MODE: http://image.ibb.co/iwm0xF/ng.png

The deep-learning course consist of 5 subcourses:






The deep-learning course is a different course than the prerequisite machine-learning course:


sumitgt 23 hours ago 3 replies      
I like this pricing model compared to udacity's new nanodegrees. This works better for people who want to rush through the course as quickly as possible. It essentially encourages faster completion.

You pay $49 per month and finish at your own pace. On Udacity you pay a fixed price of approximately $800 per semester with a trial period of 7 days. Since you are paying for the entire course upfront, you might realize after a few weeks that you don't like it. On Coursera, you can stop anytime and you've only paid $49 x number of months you tried.

m00x 21 hours ago 3 replies      
If you take a Coursera course, be prepared for it being exactly like a university class. About 45 minute of monologue per section, the odd quiz, then 2-3 exams.

From the 4 Coursera courses I've taken, I've gotten some good info, but I learned nothing practical. Udacity on the other hand, has ~5-10 min videos, frequent quizzes and practical projects.

altonzheng 20 hours ago 2 replies      
Any idea on how this compares to the deep learning course here: http://course.fast.ai/?

Very interested in taking a course, but there are so many offerings available. I have high level ML understanding from classes I took in college, but wanted to dive deeper into it.

bradleyjg 23 hours ago 7 replies      
I'm glad these courses use python. I had a lot of trouble with Ng's machine learning course because I was trying to learn MATLAB language, remember long disused linear algebra, and trying understand the substance of the lessons all at the same time.
binarymax 23 hours ago 0 replies      
So awesome. I just completed Andrew's excellent 2011 Machine Learning coursera course, and was looking where to go next. I am trying to go through the courses on fast.ai, but I don't enjoy them nearly as much compared to Andrew's teaching style and structure. Will be signing up for this!
dhawalhs 23 hours ago 2 replies      
The courses are not yet live. Meanwhile, here is a list of 23 Deep Learning online courses aggregated (aggregated by my company):https://www.class-central.com/report/deep-learning-online-co...

Also, link to Andrew Ng's original ML class: https://www.coursera.org/learn/machine-learning

bitL 23 hours ago 3 replies      
Wonderful! Thank you Andrew!

Now alongside Udacity, another set of classes with certificates - does anyone know if they hold any value to prospective employers? Last I've heard was that a recent PhD in ML was a must.

js09 22 hours ago 0 replies      
After having taken Udacity's ND, the only way I would consider taking this course is if they go in-depth with the theory so that I can clear interviews and work on projects from scratch. There's just too many resources for DL now. It's getting ridiculous. Had Udacity gone in better details with their ND I would glady have paid double for them.

Coursera does not having proper support systems like good quality forums/any mentorship/project reviews. Although, been a while since I took a Coursera specialization, would rate Udacity higher on that front even though their ND was too basic (I can't even talk about what I did in a project clearly because of so much lack of stuff).

Let's see how this specialization holds up.

Dowwie 20 hours ago 0 replies      
I hope that AI scientists will one day solve the problems with message forums. Message forums aren't designed to engage this many people. A popular MOOC gets tens of thousands of daily posts from students.

Maybe Andrew will make a practical class project out of trying to improve mass-collaboration using his MOOC as the test subject?

BrianMingus 22 hours ago 2 replies      
Latently (SUS17) also provides a more self-directed path to learning deep learning focused exclusively on implementing research papers and conducting original research: https://github.com/Latently/DeepLearningCertificate
mleonard 22 hours ago 1 reply      
Will the content for the courses be released all at once? I'd like to complete the 5 courses in approx 1 month.
nnd 22 hours ago 1 reply      
So it appears, that you can still take this course for free, and you only get a "digital certificate" of dubious value if you purchase the course. Am I missing something here?
stablemap 19 hours ago 0 replies      
Many weeks end with an interview that might be interesting independent of the course -- the first is a long one with Hinton.


tmaly 22 hours ago 3 replies      
What would be a good prerequisite for these courses?
jonbarker 21 hours ago 0 replies      
Transition to python is a great improvement! Took his previous course but the MATLAB part of it was no fun and virtually guaranteed that my work would be hard to implement in any real job setting. His advocacy of Octave probably predated quite a few of the python libraries that have since arrived.
wodenokoto 23 hours ago 1 reply      
Is Andrew Ng still with Baidu? Thought it was curious that it uses tensorflow.
nyxtom 23 hours ago 0 replies      
This is really exciting to hear. Andrew is absolutely a fantastic teacher!
alistproducer2 21 hours ago 0 replies      
I took his course up until the back propagation lesson and then lost interest. Credit to him as a teacher though because the basic of ML are still with me. I just didn't have any interest in going very deep with it.
SoMisanthrope 15 hours ago 1 reply      
Feels like marketing, not really a news story. Anyone else feel the same?
The Google manifesto isnt sexist or anti-diversity, its science theglobeandmail.com
333 points by 20100thibault  1 hour ago   213 comments top 34
Ajedi32 1 hour ago 6 replies      
Not much new here. This article is essentially just re-affirming all the scientific statements that were already made in the original memo, backed by links to scientific studies.

Only difference being that the author of this article has a PhD in sexual neuroscience (so people might have a harder time accusing her of not knowing what she's talking about) and is female (so some people might have a harder time of accusing her of sexism).

snowwrestler 1 hour ago 7 replies      
The article hides a common but incorrect assumption. Look at this paragraph:

> As mentioned in the memo, gendered interests are predicted by exposure to prenatal testosterone higher levels are associated with a preference for mechanically interesting things and occupations in adulthood. Lower levels are associated with a preference for people-oriented activities and occupations. This is why STEM (science, technology, engineering and mathematics) fields tend to be dominated by men.

The assumption here is that employment in STEM industries fundamentally and solely involves "mechanically interesting things".

The reality is that tech companies are composed of people and make products for people. Google themselves have found through their own research that the best managers are defined by their people skills, not their technical skills. So why aren't the management layers of tech companies composed of mostly women?

Strong technology is important for success, but so is leadership, market fit, team dynamics, understanding the customer, etc. The hardest question in tech companies is not "how" to build, but "what" to build. This is essentially a people-oriented problem, since customers are people.

EDIT: this tweet puts it succinctly:

> WEIRD how none of these guys ever argue that because our ladybrains are better at communication and teamwork we should be paid more


piokoch 47 minutes ago 3 replies      
I am always surprised that well educated people who are definitely not "creationists" but consider evolution as a way human kind developed are ready to ignore evolution when it comes to gender.

Clearly man and woman are different physically and mentally as for millenias they played different roles. Why "gender people" keep ignoring that and are claiming that sex is not something inborn and is a "cultural" phenomena is hard to understand.

For me gender studies are just new incarnation of Lysenkoism. Lysenko strongly belived (and thousands of soviet scientist) that weeds could spontaneously evolve into food grains because is should cooperate with communistic party.

Those who were against that obvious stupidity and claimed that genetics is the way to understand plant evolution were fired or put to jail or executed.

Similarly absurdal ideas were brought by soviet lingustics - if any one wants to have good fun, there is no better reading then Stalins's "Linguistics".

megous 57 minutes ago 5 replies      
What I find interesting about all this is how many people take personal offense from statistics. (and conclusions drawn from statistics)

The author seems to put an effort into explaining statistical distribution and what it means and what not. He's explicit that statistical observations can't be used to judge particular individuals. Draws a graph of overlapping distributions to drive the point home even more.

I'm not sure why would anyone get offended by statistical observation. It's not personal by definition.

dm319 41 minutes ago 0 replies      
> But sexism isnt the result of knowing facts; its the result of what people choose to do with them.

We know that men are taller than women. I can see you agreeing, but actually this statement is ambiguous, because these two are not the same thing:

 A man is taller than a women On average, men are taller than women
Sexism is taking a random male and a random female, and claiming that despite all the facts presented to you, the male is taller than the female. It doesn't matter that in a specific case a female is taller than a male.

The same can be applied to any group and their respective stereotype. The *ism happens when we fail to assess an individual on the data given to us, preferring to fall back on mentally-lazy stereotypes/generalisations even when what we can see says something different.

 A single study, published in 2015, did claim that male and female brains existed along a mosaic and that it isnt possible to differentiate them by sex, but this has been refuted by four yes, four academic studies since. This includes a study that analyzed the exact same brain data from the original study and found that the sex of a given brain could be correctly identified with 69-per-cent to 77-per-cent accuracy.
Well I'd argue that isn't great accuracy as 50% is what you'd expect from chance (though I haven't read those references). In fact, I might expect a similar accuracy from a machine-learning technique to predict sex based on your height.

I haven't touched on the causes of population differences. With height, I don't think anyone thinks it's anything other than genetic (by way of testosterone levels). For interests and skills, the proportion that is caused by testosterone versus culture/environment is still unclear.

If we believe there is still a cultural effect, then I think positive discrimination is justified to counter this.

As an anecdote, we were wondering why our four-year old son suddenly lost interest in 'Frozen'. He told us this week that a girl had told him at nursery that 'Frozen' wasn't for boys. Cultural stereotype reinforcement is alive and well, and starts early!

Delmania 13 minutes ago 1 reply      
From below (I wanted to make sure this didn't get hidden under a downvoted comment):

>I fail to understand how a memo calling for MORE diversity can get headlined as "anti-diversity memo" on all big media outlets. Do journalists even do independent research anymore or are they just regurgitating whatever reuters send their way without scrutiny?

The author is referring to psychological diversity, in other words, Google should be more receptive to diverse viewpoints. This is both true and not true. Yes, we should listen to others and understand, but that does not mean we should accept and value everyone's viewpoints. To invoke Goodwin's Law, perhaps we should be more sympathetic to the viewpoints of Nazis? How about white supremacists?

There are viewpoints that do harm people within society, and this is one of them. Strip this down, this is the basic "woman's nature" argument that was used for years in the past to keep women barefoot and in the kitchen. The underlying claim that women are bad at tech is ridiculous. As mentioned below, the early programmers and data entry workers were women as it was considered "office work". I'll also throw out names like Grace Hopper and Ada Lovelace. Read a site like Godel's Lost Letter, and Lipton always points out women who have made contributions to the field. I even recall an article about a house wife who researched new fractals. Women have been engaged in science, technology, engineering, and math (and medicine) since the beginning. They were male dominated because people held the viewpoint the author does, which is essentially, "It's not a woman' place". Bullshit, plain and simple. This memo does not call for more diversity. It may cite scientific research (yes, men and women are different physically and psychologically), but it calls for the same status quo that initiatives like the ones the author lambasts are trying to overcome. Are they perfect? No. but they are a step in the right direction. We need to understand these difference and adapt to them not use those differences as a way to exclude.

SCAQTony 13 minutes ago 0 replies      
I think science is a bit hyperbolic a term for this fiasco. More like mere statistics at best and we know how statistics can work out.

Calling psychology or psychiatry a science is a generous proclamation. None of the above practitioners, including a neuroscientist, neurologist, or a cognitive scientist, can explain completely why someone can look at a shot of whiskey, know it is an expensive whiskey, know how to balance the shot to their lips, then prepare for the burn, and then swallow it while hoping they will get "lucky" tonight but feel lousy tomorrow all at the same time.

We're not there yet when it comes to classifying humanity into "phylums" or categories via science.

dvfjsdhgfv 1 hour ago 2 replies      
To people who are flagging this article: why don't you read it? It's important, and the author is competent.
kartan 41 minutes ago 1 reply      
Women work in the Berlin Post Office with calculators, 1928:http://www.alamy.com/stock-photo-women-work-in-the-berlin-po...

The managers, a more people-oriented activity, are all men. But the people working with actual calculators are women. And it was not just this office, this was happening everywhere. Working with a calculator was a woman's job.

More: http://www.history.com/news/human-computers-women-at-nasa

There is a lot of factors to why STEM is dominated by men. Testosterone may be one, for real, but it is not the only one. And it doesn't justify such a big difference in numbers.

I don't know if the engineer wrote something awful or not, but this article is just a justification for the difference as if nothing can be done. And that is not true.

root_axis 49 minutes ago 0 replies      
> Scientific studies have confirmed sex differences in the brain that lead to differences in our interests and behaviour.

This is obvious and not the point of contention. The crux of the other side's disagreement is in the assumption that differences in brain chemistry attributable to sex necessarily account for all or the majority of the differences we observe in career distributions. I think the insane reaction to this memo is unfortunate because the author does appear to make an earnest effort to discuss this topic, but the memo's defenders are not doing the argument any favors by arguing against the weakest version of the opposing argument.

ebola1717 1 minute ago 0 replies      
If all this science were the end all be all, the low level engineers would be men, and men with their evolutionary handicap for empathy would be none of the managers, leaders, designers, product managers. That's how it is right?
alexholehouse 37 minutes ago 1 reply      
As mentioned in the memo, gendered interests are predicted by exposure to prenatal testosterone higher levels are associated with a preference for mechanically interesting things and occupations in adulthood. Lower levels are associated with a preference for people-oriented activities and occupations. This is why STEM (science, technology, engineering and mathematics) fields tend to be dominated by men.

And why we see fields like law being dominated by women, right?

EDIT: I, and I suspect most other scientists wouldn't disagree that there are [edit - had this as aren't previously, woops!] physiological differences between men and women, but as I read the memo, that was not what was being argued. What was being argued was that those differences were the reason for the gender imbalance in tech (i.e. women are predisposition to be less interested/capable in STEM fields), in other words, the effect size associated with biological sex is larger (and indeed must be significantly larger) than any/all combined societal/'nurture' effects.

program_whiz 32 minutes ago 2 replies      
I have a question to further this debate.

Would people consider it sexist to administer a completely automated test of technical and personality questions which was used by an unbiased program to hire only the best qualified candidates?

What would you say if the results were essentially the same as the status quo?

By the Article's Author's account, she believes that we would probably maintain the status quo with such a test, because she thinks people are self-selecting out of STEM. Others seem to think that there is some other barrier to entering -- would a test like this fix the issue, or is there something else going on?

typicalbender 57 minutes ago 1 reply      
As an additional perspective, here's an interview from James' perspective[1]. The interviewer is clearly fairly bias and holds the same viewpoint which is unfortunate but I think hearing James' perspective on the purpose of the document is interesting.

[1] https://www.youtube.com/watch?v=agU-mHFcXdw

jamesrcole 1 hour ago 5 replies      
What disturbs me most about this incident is how many people's first instinct, when faced with views they strongly disagree with, is to try and suppress those views and the person speaking them. Not to engage with the argument or to try and argue against it, just to shut the other person down. It's a totalitarian kind of response.
apeace 26 minutes ago 0 replies      
No matter how you debate the validity of the science, I hope people pay attention to one suggestion the 'manifesto' author made: that his company should do more pair programming.

Pairing has always struck me as a great way to get programmers communicating better. Without addressing any other points in the manifesto, I think he's correct that encouraging pairing would be a good way to make development environments more collaborative.

ainiriand 1 hour ago 1 reply      
Firing this guy was just a PR move in order to keep things under control. The memo was accurate and I know that Google is going to learn from it even if the guy has been sacrified.
Someone 27 minutes ago 1 reply      
I think it is unfair to call this "the Google manifesto".

If I state here that it is scientifically expected to see more foos than bars on HN, and then get banned because of it, that wouldn't become HN's manifesto, either.

I fear that, a few years from now, people will say it represented Google's official standpoints (which it doesn't) or those of a significant portion of its employees (which may or may not be the case, from what I know)

"Google employee's manifesto" or "James Damore's manifesto" are better names.

gkya 10 minutes ago 0 replies      
This stuff is going all-medieval, with witch hunts and pseudo-rogues. People are going back to emotionally responding to facts and following who shouts louder. Fixing gender issues is one thing, suppressing facts yo dislike is another.

We wouldn't have a percent of liberties and developments also in the anti-patriarchal quest without science and rationalism.

burner11 37 minutes ago 4 replies      
Why is it that nobody wants to fix the gender gap in nursing?
markdog12 8 minutes ago 0 replies      
Was horrified to see this flagged. Why would anybody flag a scientific viewpoint from a credible source with citations? Refute and discuss it, sure, but why flag it?
tptacek 33 minutes ago 2 replies      
This defense of the "manifesto" is flawed just like the others. It picks out a small subset of the claims made in the document, discards the context and all the other claims, and then harangues us for having a problem with "science".

We can talk straightforwardly about what makes the document problematic: whatever the validity of the "scientific" claims it makes about gender differences, there is no support (and likely no validity) to the connections it then makes to software development work. Despite that unjustified leap, the document goes on to suggest strongly that women working at Google are less qualified than men. There is no science Debrah Soh can cite to back up that assertion, however much she might want to.

Anyone can wrap an incendiary statement up in a pile of banal sentiment and ambiguous appeals to social science. When challenged, refocus the debate on the truisms and the footnotes and pretend you didn't write the nasty stuff you hid in the middle. And, as we can see, plenty of very smart people will fall for the trick.

Gender equity has been improving in the United States for several generations. As that has occurred, female participation in STEM fields (and in the professions, like medicine and law) has expanded dramatically. Many science fields are now approaching parity. Most have more than twice as much participation as computer science. That includes the field of mathematics, which is closely related to computer science and is certainly more intellectually challenging than "computer science" as practiced in the industry.

Among all STEM fields, computer science is distinguished for losing the participation of women over the last 10 years.

Unless the women of 1950 are somehow biologically different from those of 2017, the author's theory will somehow have to address the fact that her argument would have predicted the fields or law, medicine, biochem, mathematics, astronomy, statistics, accounting, and actuary would all be bereft of women over the 20th century --- obviously, the opposite occurred, despite the sexual revolution that was immediately to come.

The author of this article discusses a correlation between increasing gender equity and decreased STEM participation that does not appear in the evidence. There's a reason she does that: if you don't stipulate that correlation, the argument against gender bias in computer science has to confront another damning fact, which is that gender disparity in the field isn't global. Unless women in Asia are somehow biologically different than those of the US, her argument needs some way to address the fact that women make up the majority of STEM majors in many of those cultures.

Reading this article and then this thread, I find that there's really only two aspects of it that HN finds persuasive: the headline's appeal to "science", and the footnote observing that the author is a female scientist. That's not enough. Everything in between those things is wildly off.

In discussions about gender parity in CS, the word "preference" is a coded appeal to the Just World Hypothesis. There is a yawning chasm between neuroscience findings about "agreeableness" and "stress tolerance" and suitability for any particular kind of white-collar symbol-manipulation work. Ms. Soh must intuitively understand that, but mentions it not once in her piece, instead pretending that observations about the kinds of toys children play with allow us to reflect participation statistics directly into real preferences about work. Shenanigans.

alkonaut 1 hour ago 8 replies      
Why can't the memo be both sexist and science? Arguing that there are differences between genders that make one gender unsuitable for a task can be as scientific as you want, it's still sexist.

It's no different from doing the same but replacing gender with race - it's racist no matter how carefully and scientifically you worded it.

It's almost as if the person writing this article believes that if the manifesto is factually/scientifically correct then it must somehow not be sexist. As if sexism consisted of lies? I don't get it.

jpmattia 54 minutes ago 2 replies      
I'm surprised greater attention is not focused on the fact the "science" is in the midst of a reproducibility crisis. This is a big issue in the hard sciences; I can only imagine what it's like in the squishy science of gender behavior.

Tying your reputation to such a soft foundation is just inviting trouble.

dcre 12 minutes ago 0 replies      
This could have been one sentence long. "He's right! (I have a PhD.)" No content.
kaffeemitsahne 39 minutes ago 0 replies      
Funny that she calls it "the Google manifesto", as if it were endorsed by Google.
tlogan 20 minutes ago 1 reply      
The sad thing is that people do not even understand why the memo is bad.

The problem: You cannot invoke science to tell somebody: I'm better than you. Period. That causes wars.

slitaz 1 hour ago 1 reply      
So I read this article and do not see any merit or value.

The article mixes up genetics with what people end up doing in life.

toptal 1 hour ago 0 replies      
This is actually a great scientific read.
EJTH 1 hour ago 1 reply      
I fail to understand how a memo calling for MORE diversity can get headlined as "anti-diversity memo" on all big media outlets. Do journalists even do independent research anymore or are they just regurgitating whatever reuters send their way without scrutiny?

Debate me instead of downvoting please. Downvoting is like forfeiting a discussion and shows that you question your own narrative.

UK-AL 1 hour ago 3 replies      
Please don't flag the article if you simply disagree.

The key to changing minds is understanding the other side. Not shutting them out of the conversation.

gormo2 1 hour ago 2 replies      
kuschku 1 hour ago 1 reply      
This article would be a lot more worthy of a discussion if it actually discussed the actual issues.

How much is nature vs. nurture? (many of the behaviours the manifesto attributed to genetics are actually purely environmental)

Should that even matter? Shouldn't hiring processes be purely meritocratic?

The fact that the author of the memo argued for affirmative action for hiring more conservatives shows that he doesn't have anything against discrimination, as long as he profits. If he wanted a science-backed solution, he'd have supported motions to remove the topic of gender, race, political affiliation entirely from the hiring process. (Anonymous job applications can help with that).

Many of the sources were also quite misleading, or links to blogs instead of papers. The memo wouldn't pass peer review at any journal, and can, frankly said, not be called science.

As we also just had a big discussion about this topic on HN, this article is misleading and from an outlet that previously has published hit pieces on diversity policies[1], I flagged it - this discussion belongs into the recent megathread, not as its own onto the frontpage again.

[1] https://www.theglobeandmail.com/opinion/will-trump-make-amer...

smrtinsert 57 minutes ago 2 replies      
Cognitive differences observed in GENERAL != bad engineers for ALL.

The memo was a rambling incoherent mess.

More people riding bikes makes cycling safer for everyone, major new study finds cyclingweekly.com
222 points by okket  3 days ago   174 comments top 15
awjr 3 days ago 4 replies      
Denmark just worked out it stopped 55,000 sick days and where able to persuade 34% of car drivers that tried an eBike to switch to an eBike. https://cyclingindustry.news/danish-study-outlines-economic-...

Cycling can reduce risk of death by 41%, death by cancer 45% and death from heart disease by 46% http://www.bbc.co.uk/news/health-39641122

A single 10ft cycle track can carry the same number of people as 7 lanes of car traffic.

Netherlands is currently at 10% obesity and going down to 8.5% by 2030. The UK is at 27% and predicted to go up to 35%.

So cycling really does appear be to that miracle solution for urban transport but you absolutely cannot 'share space'. Absolutely key to segregate. A good end to end network is only as good as its weakest link and too many times, town planners give in to politicians and local residents and deliver stuff that really is does not provide inclusive ages 8-80 cycle routes.

gregdoesit 3 days ago 1 reply      
Come to Amsterdam to see first hand proof of this study. Virtually no one wears helmets... because you don't have to. Infrastructure is designed to be bikers first. Drivers are constantly aware of bikes and also receive training to avoid the most common accidents (hitting a biker when opening the door - see the Dutch Reach).

The biggest threat to a biker in this city is another (often less experienced) biker, pedestrians not looking around or trying to cross tram tracks parallel. However, it did take almost 40 years to get here though, changes starting in the 1970s - I do hope many cities will follow a similar route.

moonka 3 days ago 2 replies      
I believe it. After picking up cycling a couple years ago, I've noticed I am a lot more aware of bikes on the road, and drive with more care. I've also noticed my family driving more carefully around bikes as I've mentioned close calls I've had. I imagine it also leads to more willingness to invest in cycling infrastructure, which tends to make things safer as well.
u801e 3 days ago 0 replies      
I think a lot of the problems that cyclists have when riding amongst motor vehicles have to do with existing laws (such as riding as far right as practicable when going less than the normal speed of traffic).

In reality, most traffic lanes are not wide enough for a cyclist and a car to travel side by side with sufficient clearance between the two vehicles. A cyclist is about 2.5 to 3 feet wide and many states have laws requiring at least feet of distance between them and the cyclist. The cyclist will also ride about 2 to 3 feet from the right edge of the lane. A car is a little less than 6 feet wide on average.

If you add those distances up, you end up with a total of 14 to 15 feet. Even on interstate highways, lanes may only 12 feet wide. On surface streets, they may only be 10 feet wide. Because of this, you end up with the situation where cars will "lane split" while passing a cyclist and frequently misjudge how far they are from the cyclist when passing them. Also, the lane splitting car cannot leave enough room for cars in the adjacent lane because the lanes do not have sufficient width to accommodate a vehicle and half of another vehicle.

Another problem is that cyclists riding to the right aren't as visible to traffic as opposed to those who are "taking the lane". They're more vulnerable to collisions such as "right-hooks", traffic entering from side streets and opposing left turning traffic who didn't see them.

The laws should be changed to say that cyclists are like slow moving vehicles who are entitled to the full use of the lane they're riding in and passing traffic must move completely into the adjacent lane, when safe to do so, to pass the cyclist and only return to their original lane of travel after they have sufficient clearance. The law should also allow vehicles to pass cyclists on a double-yellow when its safe to do so with the same conditions as above.

This will legally sanction the "take the lane" type of riding and make cyclists more visible to the rest of traffic.

Theodores 3 days ago 2 replies      
More people riding also makes it normal. 15 years ago, working for a cycling wholesale distributor I was the only one that cycled to work. Yet I felt I had to explain why I cycled just so people didn't think I was a banned drink driver.

In 2017 I am glad to say that all the bike parking is taken at 8.55 in the morning and there are bike to work posters in the kitchen.

There is nothing new in this study. Perceptions have changed though and I thank all who have participated in this.

ilaksh 3 days ago 4 replies      
You mean less dangerous. To me, San Diego and Ft. Worth are average areas. In these places bicycles are relatively rare compared to cars. I am always surprised when I see them.

Cars weigh thousands of pounds. Consider the safety precautions for cars with air bags, crumple zones, and crash testing. There is no protection from cars for someone on a bicycle.

They try to say riding on the sidewalk with a bike is more dangerous because cars do not look out for bicycles at driveways and such.

But personally in most areas I don't feel safe biking on the street, and would rather stay on the sidewalk like I did when I was a kid. I would just have to pause to check before crossing driveways and intersections.

I believe that there will eventually be something like a smart safe city where 2000+ pounds vehicles do not freely mix with pedestrians, cyclists, and baby strollers. There should be physical barriers or totally separate walkways.

Think about it. They can't even sell a car without an airbag anymore. Yet what chance do you have from a physics standpoint to be truly safe when the other vehicle exceeds your mass by 2000+ pounds?

stordoff 3 days ago 0 replies      
Not to dismiss the study (because other studies have certainly found counter-intuitive results before), but I've always thought this would be an obvious result. If other road users are used to dealing with cyclists, and expecting them to be there at all, it's hard to see how that would make cycling less safe.
astrostl 2 days ago 0 replies      
I love the concept, but no way would I try to pull it off in 99% of the USA. I know a lot of road biking, bike to work, etc. enthusiasts and every damned one of 'em has some horrific hit-and-run story :-/
anonu 2 days ago 0 replies      
I ride in NYC and have for almost a decade now. The city has taken major leaps forward at becoming more bicycle friendly, thanks in part to Citi Bike proliferation and the addition of more protected bike paths. I think there's still a long way to go - especially in terms of education of riders around proper biking etiquette, especially towards pedestrians
projektir 3 days ago 4 replies      
I'm still deeply uncomfortable about riding a bike next to / in front of a car compared to the sidewalk, and I'm not sure how to overpower that fear...
newy 3 days ago 0 replies      
I've always believed this. More bikers on the road creates more awareness from drivers, and also more investment by cities into bike infrastructure, including dedicated and protected bike lanes.

More bikes available also means more bikers, something we're working on at Spin for US cities. Get in touch (email in profile) if this seems like a problem space interesting to you :)

trevyn 3 days ago 0 replies      
Or, generalized, more people doing X makes X safer for everyone. :)
discombobulate 3 days ago 0 replies      
Bike network effects!
zeep 3 days ago 0 replies      
if everyone was riding a bicycle, bicycle riders would be safe indeed...
Alex3917 3 days ago 5 replies      
This doesn't show that biking has gotten safer. It may have, but later adopters of biking are also less likely to engage in risky behavior.

Similarly, as the percentage of folks who use heroin increases, the relative rate of overdoses declines. But that doesn't mean that heroin itself has become any more or less safe.

A tall chimney to facilitate heat exchange in the atmosphere superchimney.org
285 points by chr1  4 days ago   188 comments top 37
lend000 4 days ago 1 reply      
For those skeptical of the science, note that there are large caves that exhibit this property, such as:https://en.wikipedia.org/wiki/P%C4%B1narg%C3%B6z%C3%BC_Cave#...

It seems like if the inside of the chimney column had a spiral shape, similar to a screw socket, the upward air pressure might alleviate some of the stress and make the column more structurally feasible. Granted, it would also dissipate some of the energy as heat.

vmarsy 4 days ago 2 replies      
This reminds me of Solar updraft tower prototypes [1], which concentrates heat at the bottom of the chimney instead of expecting cold air at the top of the chimney.

There's still a delta of temperature between the top and the bottom, but instead of

 T_bottom_chimney = T_hot_ambient_air_bottom T_top_chimney = T_cold_ambient_air_top
it is :

 T_bottom_chimney = T_much_hotter_than_ambient_air_top T_top_chimney = T_ambient_air_top
Everytime I start reading things like this, I wish I had a home with similar 'magic', like Solar chimneys[2] and other techniques[3],

[1] https://en.wikipedia.org/wiki/Solar_updraft_tower

[2] https://en.wikipedia.org/wiki/Solar_chimney

[3] https://en.wikipedia.org/wiki/Ground-coupled_heat_exchanger

saalweachter 4 days ago 1 reply      
So my question is "why wouldn't this just rip in half from the forces it's under?"

The upward force for the 20m chimney is calculated at ~600 tons. 600 tons is not a lot for a building to support in the downward direction, but quite a bit for a fabric tube to support, in tension. It's the rope/(space) elevator problem - you need a super material to handle that much force, don't you?

But aside from worrying that our wacky inflatable tube of death will rip free from its tethers and tumble freely in the wind, killing thousands, I actually really like this idea, as geo-engineering. It is a process that can be stopped and started relatively cheaply, unlike a lot of other proposals. If the tube has unforseen effects, it could be deflated and reeled in, unlike eg throwing particulate into the upper atmosphere.

the_rosentotter 4 days ago 2 replies      
So five kilometers of flailing inflatable tube man.

I get that the upwards wind force can sustain the fabric structure, but it is hard to imagine that it could also carry a bunch of huge turbines, as well as the cabling required to carry the generated electricity. Not to mention safety concerns. Does this seem unrealistic to anyone else?

Also, would it be possible to do a proof-of-concept using an existing man made structure like Burj Khalifa class skyscrapers? Presumably it would be easier to deploy a tube off the top of one of these than to build one from scratch.

smoyer 4 days ago 0 replies      
If the column of air is really moving at 300 MPH, they'd better diffuse that at the bottom so people (and things) aren't sucked into the chimney (it would suck to be ejected at the top without a parachute but you could probably sell a ride to the top to the wing-suiters).
mikeash 4 days ago 4 replies      
"The inside and outside air will be rising up. However, the air outside will be cooling adiabatically, so its temperature will be dropping. The air inside will be not affected by adiabatic cooling and will maintain its energy, so it will be warmer and less dense than outside air."

Is this a joke, or a crazy person? Air in the tube will expand and cool just like air outside the tube does.

robocat 4 days ago 1 reply      
datadata 3 days ago 1 reply      
Engineering issues aside, to evaluate if a super chimney would be energetically viable you need to understand the concept of "Convective available potential energy" or CAPE--https://en.wikipedia.org/wiki/Convective_available_potential.... CAPE has dimensions of energy/mass and a describes how much energy is released by raising a mass of air to some higher elevation.

CAPE is used to forecast storm development, as updrafts can more likely spontaneously form when there is more energy released by the updraft. CAPE values can also be zero or negative, in which case there would be no available energy to sustain an updraft. From my understanding, CAPE is the only factor that would determine if a super chimney could work at a given time.

I have not found a good resource on global CAPE patterns including daily patterns, but it seems very likely that there is any fixed location and fixed elevation that always has a positive CAPE value. It would be an absolute requirement to find such a location for this project to work.

You should also be able to calculate a crude bound on the maximum updraft velocity simply as a conversion of potential energy to kinetic energy. Wikipedia says that exceptionally high CAPE values proceeding extreme thunderstorms are around 5kj/kg, which would accelerate a mass from rest to 100m/s (220mph). Of course this is an extreme value, typical values are more like 1kj/kg, which correspond to a velocity of 44m/s (100mph).

Here is a really good paper on CAPE and atmospheric convection heat engines: http://journals.ametsoc.org/doi/pdf/10.1175/1520-0469%281996...

yohann305 4 days ago 1 reply      
Anyone here could run a super tall chimney software simulation ?

I'm super interested in seeing someone confirm or debunk this.Anyone else interested, upvote please

jcrawfordor 4 days ago 1 reply      
The short story "Shortstack" by Walt Richmond and Leigh Richmond depicts this idea and was published in Analog in '64. Likely coincidental, but amusing to see '60s science fiction apparently made flesh.
humanfromearth 4 days ago 2 replies      
For the 5km chimney it needs to hold on 500km/h winds. For scale a category 5 hurricane is 250 km/h. Is it even possible to have that kind of structure with existing materials?
gtt 4 days ago 1 reply      
I've tried to simulate 1km chimney in Comsol, but I cannot make it converge to a solution. If anyone is interested, the model is here https://mega.nz/#!jFgBxI6J!jdxloYFwcuk_YyGcIMlOmJTKcPbxyD2B4... (may be I'm doing something wrong with simulation parameters, help would be very much appreciated!)
randyrand 4 days ago 1 reply      
This will also function as a hell of a bird vacuum.
foota 4 days ago 1 reply      
Here's my analysis from maybe wrong principles. If you have slightly more dense air beneath slightly less dense air, the air will experience a net force upwards. If this net force is stronger than gravity, then the air will experience upwards acceleration. This will continue as long as there is a difference strong enough. At the top of the tube, there is no more force since the density will be the same since the air will spread out after exiting. (If it's not already at the same density after going through the tube).

Looking at it this way this seems sound to me, am I wrong?

ChuckMcM 4 days ago 3 replies      
Presumably you just lay a pipe that goes up the side of Everest and free power!

I wonder if the author asked the question "Why don't we have tornadoes all the time?"

If they had, that would have lead them to the physics of tornadoes. In my case it was the physics of so called 'dust devils' in the desert which are much smaller phenomena but based on the same ideas. Warm air rising through cooler air.

You might ask, but why don't we have them all the time? And the answer is that as air goes up, it spreads out, and as it spreads out it becomes less dense, and the lack of density is perceived as a colder 'temperature' even though the air molecules still have more kinetic energy and are thus 'hotter'.

In the video the tube is supposed to constrain the air (which it will) and the warmer air will rise inside of it, but without an energy source the warm air rises until its 'weight' is equivalent to the un-risen air underneath it, at which point it stops rising and the system is stable. If you were to cool off the bottom the air would start sinking again.

This has been experienced time and again by inexperienced makers of fires in their fireplaces. If you don't put enough energy into the air to make it rise, it comes back down the chimney and fills your living space with smoke. A fireplace is a remarkable little machine, where the fire heats the air, which pulls in more air as the air above rises, which puts more oxygen into the fire and increases its energy output etc. But without the fire burning in the fireplace the air stops moving.

Tornadoes benefit from a mass of really cold air sitting on top of warm air. This does two things, one the cold air above pushes down on the warm air to pressurize it, and two when a "hole" begins forming in the cold air mass it operates like an inverted tub drain and the warm air starts draining out of the tub. The energy source for a tornado is the temperature differential that is set up by the result of moisture condensing out of the air and super cooling the air around it.

Similarly a hurricane is powered by the temperature differential between the ocean and the air above it.

All three systems (fireplace, tornado, hurricane) share a common theme, there has to be a source of energy for them to operate. Without it, the air reaches equilibrium and just sits there. No magic allowed.

That said, if instead you built a tunnel, then you could connect two different air masses and extract energy from two different pressure differentials. The most interesting ideas have a tunnel under the Rockies or under the Sierras between the Mojave desert on one side and the milder (and moister) climate on the other. To the delta you can get from that is linear with respect to distance and/or a geographic feature that can inhibit the natural balancing of the air masses (like a range of tall mountains).

Sadly neither super chimneys nor lighter than air vacuum balloons are workable ideas.

Edit: It occurs to me that if you could make the chimney high enough you could put the top in the underside of the jetstream, then you could suck air up using the venturi effect.

Tarrosion 4 days ago 1 reply      
So many signals suggesting this is a wild physics-defying idea that could never work, e.g. how many websites claiming 'this one neat trick solves global warming' really hold the key to solving global warming?

I hope that's not the case and by this time next decade we're all laughing about that century and a half where we put so much carbon in the atmosphere wow wasn't that a hoot...

Realistically, I am sympathetic to the idea that geoengineering, massive structures and engineering projects enabled by modern materials, etc. deserve more thought.

scythe 4 days ago 0 replies      
These geoengineering proposals should be understood as mitigation strategies to be implemented after we have reduced carbon emissions, since even then we still have a problem. But this one seems very dubious.

>Speaking in terms of thermodynamic, we can say that chimney prevents adiabatic cooling of a rising parcel of air. Normally, when hot air freely rises in atmosphere, it expands as it gets higher and pushes the surrounding air. That causes surrounding air to heat and rising air to cool. That process continues until equilibrium is reached. At that point air stops its ascending. Unlike freely rising parcel of air, the air in the chimney is restricted in its horizontal expansion and thus, it is not free rising. When air rises in the chimney it also expands but only into upper direction. It compresses the layer of air above it, heats it up and loses its own heat. At the same time air below does the same thing. And thats how it goes all the way until the chimney exit: layers of air are being pushed and push themselves. That results in maintaining the same amount of heat in every layer of air, and that is how the chimney works.

This explanation ignores gravity. Air above you exerts more pressure on you than air below you, albeit by a tiny amount. But when the only thing moving is air in a 5-kilometer chimney, you can't get something for nothing. For intuition, just imagine the chimney is full of water. The water at the bottom is obviously under more pressure than the water at the top. The chimney faces the same consideration, but the ideal gas law applies.

Furthermore, the equation used in the "Calculations" section:

>q = dh2 /4 [ (2 g (po - pr) h ) / ( (l pr / dh) + pr ) ]

is sourced from this website:


which rather obviously works from the assumption that the chimney is placed inside of a heated house.

This all seems to be a sort of Sokal effect in climate science, I'm afraid. The paper 'SubiculumCode cited does not really analyze the thermodynamics used for the chimney and points more to the unrealistic dimensions (1 kilometer diameter and 10 kilometers high).

shoefly 4 days ago 0 replies      
Whatever we do, it's important that we learn how to control the "ingredients" of our atmosphere. There are so many things that could go wrong with our atmosphere and result in mass extinction. Global warming, ice ages, massive volcanic eruptions, etc. If we can find a way to quickly filter out the crap and rebuild our atmosphere... well, this technology could be used for protecting our Earth and future pursuits in space.
desireco42 4 days ago 0 replies      
I think this, however flawed some of the explanation of the effect might be, is something we can try and experiment with fairly easily. Either it can be done, or not. And I believe it can.

Now, we can't let Musk do all the cool things, maybe someone else could step in and fund a project to explore application of updraft towers.

mbfg 3 days ago 0 replies      
If you search 'solar tower' on youtube, you will see all kinds of videos of existing installations of things that are similar, albeit most are not as tall, nor are they flexible. But the basic concept appears to be the same. Quite a few of them are from many years ago. So it would seem the idea works to some extent, and perhaps the the idea of a much taller, and flexible variant is the crucial difference that will make a big difference.
mbfg 4 days ago 0 replies      
Given this, if successful, is creating storms at the top, won't it continually and repeatedly be hit by lightning? And given that it is some kind of fabric, cause havoc?
pdonis 4 days ago 1 reply      
We already have something that does the same thing as this claims to (facilitate heat transfer from the surface to the upper atmosphere): it's called the hydrologic cycle.
dghughes 4 days ago 0 replies      
Wouldn't such a chimney take off like a Chinese lantern?
fastball 4 days ago 1 reply      
Would the radiant heat from the ground be enough to continue this effect at the same rate during the nighttime?
SubiculumCode 4 days ago 0 replies      
Watched the video. Sounds miraculous :) Anyone here knowledgeable of atmospheric thermodynamics?
toddh 2 days ago 0 replies      
Would it be possible to make these into skyscrapers to house people and businesses? That would handle the financing part of it.
animex 4 days ago 1 reply      
This model must be able to be simulated somehow! To Minecraft!
kpil 4 days ago 1 reply      
Is it really a good idea to move more water vapor, a potent greenhouse gas, significantly higher up in the atmosphere?
codecamper 4 days ago 0 replies      
Hate to be a downer, but out of 157 comments so far, nobody has mentioned ocean acidification. If there was a way to build these chimneys... we could then go on burning fossil fuels & so then the oceans would become more acidic, possibly leading to the inability of krill to form exoskeletons, removing a one of the main oceanic bottom of the food chain food sources.
unabridged 4 days ago 1 reply      
Things like this are the reason I don't think global warming will ever be a problem. Even if this example turns out to be a pipe dream, we will eventually figure out a way to lower the temperature or remove co2 from the atmosphere.
stephengillie 4 days ago 2 replies      
Would this be an efficient source of air pressure for the Hyperloop?
chroem- 4 days ago 3 replies      
Whoever made this assumes that the chimney wall would be a perfect insulator, which absolutely cannot be the case if it's supposed to be a thin cloth or film barrier. The air would cool to the same temperature as the surrounding atmosphere.

This will not work.

foota 4 days ago 1 reply      
Any idea how tall one of these would need to be in theory to support itself? I think that would be a cool sight to see and a great way to prove feasibility.
pmoriarty 4 days ago 4 replies      
How much will one of these cost?
hossbeast 4 days ago 1 reply      
Unreadable on mobile
stefantalpalaru 4 days ago 2 replies      
If a constant air flow is needed to keep the chimney upright, what will happen during the night, when the desert cools down?
ryanobjc 4 days ago 1 reply      
A spelling error makes it hard for me to take the proposal seriously.

For something as important as this, the details count. A loose approach to spelling is disturbing.

Kids Pass Just Reminded Us How Hard Responsible Disclosure Is troyhunt.com
280 points by ohjeez  3 days ago   89 comments top 15
ScottBurson 3 days ago 11 replies      
I have trouble understanding this mindset. It's like, if you were walking away from your car in a parking lot, and someone said "Hey! You've left your car unlocked!", and you yelled at them angrily "Stop looking at my car!!!". It makes no sense at all, and yet it's practically the universal response from people who don't know what they're doing.

People occasionally suggest that software engineers should be professionally licensed. I have a different proposal: I think that people who want to manage a business involving software development should have to get trained and licensed.

ETA: while my proposal is somewhat facetious when considered about all software development, perhaps it's not completely inconceivable that we could require businesses collecting any personal information from users to be licensed and audited. We already have PCI-DSS compliance rules for businesses using credit cards; this would be analogous, though it would have to be enforced by the government, as credit cards wouldn't necessarily be involved.

sam_goody 3 days ago 1 reply      
Heh! Nothing new in the behavior.

Mr. Feynman famously found you could lift the combo off a safe [with the a-bomb's secrets] when it was empty. When he alerted the Colonel not to leave his safe open, the response was to:

send a note around to everyone in the plant which said, During his last visit, was Mr. Feyman at any time in your office, near your office, or a walking through your office? Some people answered yes; others said no. The ones who said yes got another note: Please change the combination of your safe. That was his solution. _I_ was the danger!

swang 3 days ago 2 replies      
BBC posted a followup...

1. Kidspass spokeswoman said that it was their off-hours crew that blocked Alex and Troy. They were unblocked 10 hours later.

2. They will institute a vulnerability policy as a result of this.


avaer 3 days ago 3 replies      
Putting text on a page isn't hard stuff. The hard stuff is teaching computer security to an organization that mistakes responsible disclosure for a hack attempt, and thinks a Twitter block will protect them.

It's probably also hard to know what a good security audit looks like, unless you grasp basic security in the first place.

I have no idea what the solution is.

coldcode 3 days ago 0 replies      
After so many decades in this industry nothing surprises me at all. Security is usually an afterthought that barely warrants spending more that a token amount. I once did a contract at a public university and found the app that every department used to verify with the state that money was appropriately spent used incrementing id's in the url and used GET to handle the delete button. I wound up fixing it for them on the way out (after weeks of telling me it wasn't a concern). A simple command line script would have deleted the entire database leaving the university with no budget for the upcoming year. Another place I worked kept production passwords in the code repository; when I complained they told me they passed their audits every year so it didn't matter. HIPAA company in the US no less.
confounded 3 days ago 2 replies      
Very few companies that use technology are technology companies.

Is there no open-source standard for authentication and user-data management? Do companies really need to roll their own each time?

reitanqild 3 days ago 2 replies      
Related - but only to the blocking:

A friend of mine who works with one of the really expensive consulting companies witnessed someone lashing out on twitter about how bad such and such people where.

So he answered along the lines of: I grew up in such and such home, my experience is totally different and I'll be happy to buy you lunch.

Answer: blocked.

Blocking is a power thing for some people. IIRC it used to be a thing in the old Usenet and of course it existed before that in other forms.

bvv 3 days ago 0 replies      
This seems like an area where a trusted organization (perhaps the EFF?) could do a lot of good by creating a "for dummies" webpage where the vulnerability disclosure process is explained in layman's terms (i.e. with suitable car analogies...) from a website owner's perspective. Those who discover a vulnerability in a company's IT infrastructure can then submit a link to this page with their reports.
S_A_P 3 days ago 1 reply      
All I can imagine that is happening there is panic. Defensive behavior such as this indicates either they don't really know how to fix this quickly, or they just don't care.
rmellow 2 days ago 0 replies      
Sometimes we forget the entrepreneurs behind these services can be technologically illiterate. When they realize they have a problem they don't understand, they get scared, and can easily get confrontational and try to dodge any liability (e.g. by getting the police involved).

How can we teach these entrepreneurs to act? Perhaps by creating an accessible and gently worded guide on how to act; an FAQ from a reputable organization that you can link to every time you disclose a vulnerability? IEEE, EFF I'm looking at both of you.

tarr11 3 days ago 1 reply      
What is a good way to implement responsible disclosure for single developer / side projects?

Eg, when you don't have the resources to pay for bug bounties etc.

jamiethompson 2 days ago 0 replies      
Having used Kids pass. It's a pointless product anyway. Snake oil. All the "deals" are just links to PDFs, many of which have no barcodes and the ones that do are just generic barcodes that don't relate to a particular Kids Pass account. Also more often than not, when you redeem the vouchers they aren't scanned.

There's nothing to stop you downloading all the PDFs you'll ever need and then ending your free trial. Other than the fact that that's technically fraud.

z3t4 3 days ago 2 replies      
Writing "They have a serious vuln" on twitter is not responsible. Try to hack those who have bounties, please leave the others alone, or at least contact them privately when you find a vuln. Give them a chance to fix it, and if you want to be helpful also tell them what the issue is.
notyourday 3 days ago 1 reply      
The solution to this is simple. Disclose everything. Have these companies destroyed. Have everyone who works for them fired and become unhirable. Have their houses foreclosed on because they cannot afford to pay the mortgages or rent.

That's the only way to ensure that the security is taken seriously.

DanBC 3 days ago 1 reply      
One of the benefits of the oppressive regime in the UK is the proliferation of regulators.

The Information Commissioner is the regulator for this kind of thing.


They do take action on this kind of thing.

Despite S.E.C. Warning, Wave of Initial Coin Offerings Grows nytimes.com
226 points by JumpCrisscross  21 hours ago   187 comments top 15
alistproducer2 19 hours ago 5 replies      
I was involved with one of the bigger ICO right at the beginning of the ICO craze. I had pretty regular conversations with the project founder and was in talks to become the dev evangelist. The project had raised ~$5 million in ETH (this is when ETH was trading around $80). Despite the fact that this project had raised a crazy amount of money the whole project was being conceptualized and coded by a single 23 old guy who was promising features that much larger projects with teams of the best people in the field were yet to solve. Not to mention the project was blatantly lying about it's PoW algorithm.

When I tried to point out that the project should use some of that money to organize the software engineering aspect the founder got pissed off with me accusing me of trying to run the project. They had a second round and raised even more money (in addition to the insane capgains on their first round).

I check the github from time to time and it appears they've got at least 2 other devs involved and may actually bring a blockchain to market but this doesn't hide that fact that the project needed nowhere near the amount of money they raised. Hell they could've used a couple hundred grand from the first round and literally just paid a team of people to build the blockchain for them. did I get off of a gravy train? Yes. But there are more things to life than money.

nipponese 20 hours ago 1 reply      
The problem is projects like these: http://bitqyck.me/ A classic hard-sell 80s pump and dump scam reborn in the ETH era.Bitqyck came to my attention through a family friend with very limited understanding of cryptocurrency, but just enough financial literacy to be in an investment group being fed these scams by people they trust as technical leadership. I imagine, ultimately, these people will demand protection from having to vet ICOs themselves, and twenty years later there will be an awesome movie about this time starring Leo DiCaprio.
utnick 15 hours ago 1 reply      
If you want to really see what the environment is like now, read the bitcointalk ico announcement threads. It's absolutely insane. Very little of that stuff makes its way to hn.

People are raising money with 10 page vague 'white papers' with ideas that make no sense or are completely unworkable on the blockchain.

There is very little skepticism or research with these icos . Most of the commenters are trying to help the project and receive tokens in return, do tasks to qualify for free tokens ( bounties ) , or promote the ico so their token value increases

If there was a good way to short individual ico tokens it might bring some sanity

the_stc 19 hours ago 1 reply      
The SEC guidance galvanized us on not offering tokens. Instead, we are going to offer shares directly in the company, even if they are mostly issued via an ERC20 contract. If we are going to get heat from the SEC, might as well go all out and issue proper shares.

Plus tokens don't even make sense most of the time. You can't buy a Tesla car with TSLA and you do not get dividends/trading with a car. And no one outside kids at Chuck E Cheese want to use venue-specific currency.

ringaroundthetx 20 hours ago 3 replies      
Even the SEC realizes that not all token sales are securities. Some people in the industry simply aren't getting it, they are creating a straw man which the agency isn't even arguing for.

> The agency said that it would focus on coins that should be categorized as securities.

So there are all these people on the periphery just HOPING for their prophecy of a heavy handed government breaking the cryptocurrency rush.

They read the headlines.

They create the headlines.

They purposefully don't read the SEC report or consider any information to the contrary of what they are expecting.

Despite S.E.C. WARNING? This is an article about everyone getting smarter and restructuring their offering where necessary, and continuing to move forward.

Even the WARNING in the SEC's DAO report two weeks ago acknowledged that not all token offerings were securities. But detractors and regulated entities with a legitimate reason to be skittish, used it as the WELP SHOW'S OVER GUYS argument which is completely unfounded.

> Nick Morgan, formerly a lawyer in the S.E.C.s enforcement division, said that the security label was likely to apply to any coin that an investor buys with the expectation that it will increase in value as a result of the efforts of the entrepreneurs who created it.

The SEC still answers to the courts, and this consolidation of capital will allow these new organizations to take it to the courts efficiently. The Howey test from the 1940s did not consider this kind of asset to ever exist. It isn't an end all be all, it is a test.

The SEC is not mandated by Congress to undermine interstate commerce, it is created to provide confidence. So far, they've been playing it smart.

anovikov 20 hours ago 2 replies      
Most ICOs usually exclude Americans. Because these scamsters know it's a bad idea to get Uncle Sam angry.
blhack 16 hours ago 1 reply      
To put it into perspective, the tezos ICO recently raised a little more than $200,000,000 worth of BTC and ETH.

Here's their twitter: https://twitter.com/tez0s?lang=en

Here's their subreddit: https://reddit.com/r/tezos

Here's their website: https://tezos.com

It doesn't look like they have really updated anybody on what they're doing with $200,000,000 since their ICO closed.

SeanDav 5 hours ago 0 replies      
There is a great bit of anecdotal advice for situations like this:

"When your hairdresser starts discussing it, it is time to get out..."


vanattab 21 hours ago 17 replies      
I am thinking a short position in bitcoin is in my near future. I am not saying bitcoin is doomed to fail but I am pretty confident this shit coin crazy is going to hit the fan sooner or later and it's going to drag the "legitimate" crypto currencies down.
gaetanrickter 21 hours ago 1 reply      
Add to this "Beyond Bitcoin: Overstock Lets Customers Pay With More Than 40 Alt Coins" http://fortune.com/2017/08/08/overstock-digital-currency/
dustingetz 15 hours ago 2 replies      
Aren't all these people just going to be arrested?

It's not illegal today, but it will be soon enough, and when you piss off powerful people they find a way to shut you down, one way or another.

whytaka 16 hours ago 2 replies      
How exactly are ICOs performed?

1) Set up mining software.

2) Start running it on your own machines and mine enough tokens for ICO.

3) Sell tokens. Profit. ??

Are investors at some point invited to run the mining chain on their own machines? Surely they don't just trust that the company raising money will do manage the accounting on their own.

mindfulplay 7 hours ago 0 replies      
I started with quarters years ago before it was cool.
api 20 hours ago 2 replies      
Is there an honest non-scummy way to raise capital through this mechanism for something without an inherent scarcity built into the system and that doesn't carry nasty future litigation risks (or at least doesn't carry them more than a SAFE or a convertible note)?

My impression so far is no but I could be wrong.

d3vnet 21 hours ago 3 replies      
So how about making investors explicitly state that they are not US persons?
       cached 9 August 2017 15:11:01 GMT