hacker news with inline top comments    .. more ..    16 Mar 2017 News
home   ask   best   2 years ago   
Guetzli: A New Open-Source JPEG Encoder googleblog.com
63 points by ashishgandhi  1 hour ago   14 comments top 8
vladdanilov 16 minutes ago 0 replies      
I'm working on a similar thing (http://getoptimage.com). While Guetzli is still visually better and a bit smaller in file size, it's terribly slow and requires a lot of memory. But it's a great experiment. So much knowledge has been put into it.

I believe using a full blown FFT and complex IQA metrics is too much. I have great results with custom quantization matrices, Mozjpeg trellis quantization, and a modification of PSNR-HVS-M, and there's still a lot of room for improvement.

SloopJon 0 minutes ago 0 replies      
The Github README says, "Guetzli generates only sequential (nonprogressive) JPEGs due to faster decompression speeds they offer." What's the current thinking on progressive JPEGs? Although I haven't noticed them recently, I don't know whether they're still widely used.
mattpavelle 30 minutes ago 1 reply      
I'll run some of my own experiments on this today, but I'm initially concerned about color muting.

Specifically looking at the cat's eye example, in the bottom of the pupil area there's a bit of green (reflection?) in the lower pupil. In the original it is #293623 (green) - in the libjpeg it is #2E3230 (still green, slightly muted). But in the Guetzil encoded image it is #362C35 - still slightly green but quite close to grey.

In my experience people love to see colors "pop" in photos (and photography is where JPEG excels) - hopefully this is just an outlier and the majority of compressions with this tool don't lose color like this.

i80and 47 minutes ago 0 replies      
Some comparison with the mozjpeg encoder here: https://github.com/google/guetzli/issues/10


> We didn't do a full human rater study between guetzli and mozjpeg, but a few samples indicated that mozjpeg is closer to libjpeg than guetzli in human viewing.

leetbulb 9 minutes ago 0 replies      
I use ImageOptim (https://imageoptim.com) for small tasks. For larger tasks, https://kraken.io is nuts.
dmitrygr 49 minutes ago 1 reply      
Cool, but neither the article nor the paper (https://arxiv.org/pdf/1703.04416.pdf) mention just how much slower it is.
ktta 26 minutes ago 1 reply      
I wonder how Dropbox's Lepton[1] compresses JPEGs encoded using Guetzli. Since they already pack more info/byte would there be noticeable compression?

Someone out there must have tried this.


corybrown 34 minutes ago 3 replies      
Very cool. I'm not an expert, but does JPEG generally have a ton of flexibility in compression? Why so much difference in sizes?
A million-dollar engineering problem segment.com
86 points by gwintrob  1 hour ago   29 comments top 11
gravypod 0 minutes ago 0 replies      
I've always been set aback by how much AWS servers cost. Maybe I'm just too cheap but you can go to very reputable hosting companies and get things at fractions of the price.

For example, if I need 16GB of memory and 4 cores here are my options:

 * AWS (t2.xlarge) $137.62 * OVH (EG-16) $79 * So You Start/OVH (SYS-IP-1S) $42 * Kimsufi/OVH (KS-3B) $29 * Hetzner (EX41) 46.41 [Lowest cost model is 32GB of ram] * Joe's Data Center (No model name) $45 [Duel socketed L5420 which is 8 cores]
That's a crazy price difference and, unlike frmo what I understand about AWS, you don't need to pay for bandwith for most of these companies or they have some crazy high limit. IIRC it's common for 20TB to be the standard (FREE) bandwith limit. You're also on real hardware, not a VM.

Unfortunately not all of these are perfect systems. There are issues with them. Some have slower CPUs, some have slower disks, some are in other countries but you can just pick the ones that suit your need for whatever you have to run. You can also use VMs that are far cheaper then these dedicated systems. If your workload is under 50% duty cycle on a CPU and under 4GB of memory you don't need a dedicated server. Buy a VM from one of these companies:

 * RamNode (8GB SVZS) $28 [Fully SSD storage, very fast network] * OVH (VPS CLOUD RAM 2) $22.39 [12GB(!) of ram and 2 vCores] * Linode (Linode 4GB) $20
These are all VPSs but they will get the job done and are cheap enough that you can build something great on a budget.

This is just from a few minutes of remembering when I had to buy my own hardware. I'm sure this isn't an exhaustive list. You can usually find information on the web hosting talk forums and on a site called low end box. I don't have links on hand but they're worth a read.

Negitivefrags 41 minutes ago 3 replies      
A friend of mine was annoyed that a small service he liked was shutting down.

He contacted the developer who said that they were shutting it down because the server costs were higher than the money they were making.

They were spending 5k a month on AWS crap and claimed it was impossible to get any lower.

He helped them consolidate everything onto a single rented dedicated server costing 400 a month. Now the service is profitable, and will stay up.

It runs way faster on the single server. It also has required less maintenance after the move too.

This kind of shit is everywhere. At this point simply not using AWS is a competitive advantage.

kornish 14 minutes ago 1 reply      
The Dynamo incident highlights an important lesson when using consistently hashed distributed data stores: make sure the actual distribution of hash keys mirrors the expected distribution. (though to their credit, someone writing an automated test using a hard-coded key was beyond their control).

Incidents like this are generally why rate limits exist, which they don't currently have [0], but perhaps they'll consider a burst limiter in place to dissuade automated tests but not organic human load spikes.

Unfortunately there doesn't seem to be an easy way to fix the per-user ID write bottleneck, short of adding a rate limit to the API which would push backpressure from Dynamo to the Segment API consumer. Round-robin partitioning of values would fix the write bottleneck, but has heavy read costs because you have to query all partitions. They undoubtedly performed such analysis and found that it didn't fit their desired tradeoffs :)

Great post, very informative. Thanks for sharing! Also, love the slight irony of loading AWS log data into an AWS product (Redshift) to find cost centers.

[0]: https://segment.com/docs/sources/server/http/#rate-limits

teh 1 hour ago 6 replies      
I've been joking with friends that my next job will be AWS efficiency guru. I've somewhat optimized our own use, but I think I could use similar, simple rules to get 20% out of a 500k / month budget.

Give me what I save you in 2 months and I'll have a good business :)

jamiesonbecker 17 minutes ago 0 replies      
Great writeup. The "user_id" one really hit home for me. @ Userify (ssh key management for on-prem and cloud) we currently have hotspots where some companies integrate the userify shim and put 'change_me' (or similar) in their API ID fields. Apparently, sometimes they don't always update it before putting into production... so we get lots and lots of "Change Me" attempted logins! It's not just one company, but dozens.

Fortunately, we cache everything (including failures) with Redis, so the actual cost is tiny at most, but if you are not caching failures as well as successes, this can result in unexpected and really hard to track down cost spikes. (disclaimer: AWS cert SA, AWS partner)

Segment's trick to detect and record when throttling, and using that as a template for "bad keys" (which presumably are manually validated as well) seems like a great idea as well, but I'd suggest first caching even failure calls on logins if possible, as that probably would have mitigated the need to ever hit dynamo.

PS the name 'project benjamin' for the cost cutting efforts.. pure genius.

qaq 13 minutes ago 0 replies      
Here's amazing way to optimize your AWS bill don't use it. Compared to dedicated you overpaid millions than you spent non-trivial developer time/money to get the number down but you are still overpaying. At 5K/month AWS might make sense (although debatable) at your level of spend it's a really bad idea. At this level an ops team of 2 1 on-site 1 remote (diff time zone) would give you way more flexibility and a ver low bill.
fourstar 1 hour ago 1 reply      
I wish I could invest in Segment. Lot of smart people over there.
dastbe 55 minutes ago 0 replies      
Reading through this, I would change "Dynamo is Amazons hosted version of Cassandra" to "DynamoDB is Amazon's hosted key-value store, similar to Cassandra". The former (to me) sounds like you're saying they vend a managed Cassandra.
joshua_wold 11 minutes ago 0 replies      
Can you lower our bill now? :)
officelineback 1 hour ago 1 reply      
Is this the same thing as segment.io?
nik736 1 hour ago 2 replies      
Have you guys considered going bare metal or a hybrid approach? With such immense spendings (even when saving the $1m/yr) it would probably be a lot cheaper.
Weapon physicist declassifies rescued nuclear test films llnl.gov
251 points by mxfh  5 hours ago   126 comments top 23
soneca 3 hours ago 6 replies      
Coincidentally, I am just 15 minutes away from finishing listening the 350 minutes podcast episode of Dan Carlin's Hard History: The Destroyer of Worlds(1), which is all about how the advent of nuclear weapons changed everything regarding international politics and the war itself.

The podcast covers the political and military landscape and events between 1945 and 1962, culminating at the Cuban Missile Crisis, focusing on the American side of the Cold War.

If you know Dan Carlin's work, and take a hint from the episode's length, you know that it is a profound analysis, thoroughly studied from a variety of sources and with a real, intelligent effort to understand all sides, all parties, all individuals, and all context of the events. A real historian work.

Of particular interest is the glimpse on what was passing through all those brilliant scientists minds throughout and after the development of the tech. This is concentrated in the first half of the podcast if you are interested.

(1): http://www.dancarlin.com/hardcore-history-59-the-destroyer-o...

EDIT: just finished it, I think the final words of the episode are worth sharing:

"As I said, human beings are 70+ years into an on-going experiment to see if they can adapt or evolve to handle their weapon's technology. So far, so good. But let's remember that we are a long way from the edge of this tightrope. These might not even be the most powerful weapons we are creative enough to invent. And the systems we have in place still have human beings involved, which is one of those variables that makes it tough to think that we can go another 100, 30 or so years, and have the human story end with the words '...and we lived happily, ever after.'" Dan Carlin

M_Grey 5 hours ago 5 replies      
"You can smell vinegar when you open the cans, which is one of the byproducts of the decomposition process of these films," Spriggs said. "We know that these films are on the brink of decomposing to the point where they'll become useless. The data that we're collecting now must be preserved in a digital form because no matter how well you treat the films, no matter how well you preserve or store them, they will decompose. They're made out of organic material, and organic material decomposes. So this is it. We got to this project just in time to save the data."

That's... incredible. Then it sounds like they used a DNN or something similar to analyze the rate of fireball expansion in the films to refine their earlier estimates of yield.

What a beautifully subtle use of tech.

rangibaby 4 hours ago 5 replies      
I'm not surprised these were kept classified for so long. No one could watch these and not be moved. That beautiful beach with a mushroom cloud on the horizon. The birds fleeing. What a juxtaposition! Nuclear weapons are humanity's shame. That we used our best minds and science to build these things.

"I should have become a watchmaker" - Einstein

cr0sh 2 hours ago 0 replies      
Is it me, or does it piss anybody else off when stuff like this happens, especially with something of this magnitude?

I mean - as terrible and stupid as it still is - it is one thing to lose the original video footage of the first moon landing (GAH!).

But here we have data, about the most destructive weapons ever created, that just sits gathering dust, decomposing, becoming useless. Meanwhile, the computer models, being developed and designed by people who likely weren't even alive at the time of these tests, are relying on data and assumptions that don't match up with the data in those tests!

What does that mean? I don't know - but it doesn't sound good on the surface - not good at all. Would these weapons work if needed (let's hope that is never the case!)? Would they fail? Are they failing now?

And why wasn't this stuff digitized a long time ago? Why do we not seem to care about this crap?

Lastly - we are in an era that are so far away from that time period of testing. It bothers me and frightens me that so many people - most who either weren't alive at the time, or were children with no access - these people have no idea about the destructive power of these weapons, having not experienced first-hand the nuclear tests that did occur. These men and women who were "there" are dying or dead already. And those replacing them seem to be getting bolder about wanting to use these weapons, without having the gut affirmation of what they would unleash. I have an opinion - which may be unfounded and false - that one of things that have kept us (humanity) from nuclear annihilation thru war has been the fact that there are still people who remember those tests. Those who remember - at a gut level - that the power of these weapons, from witnessing (or having other close-knowledge of other witnesses) these tests - have stuck with them in a way that can only be akin to that of say, seeing the earth from the moon, or from space.

It's a primal thing - the knowledge from seeing these weapons and their capabilities - a single one which can destroy an entire city - and knowing we and "them" target many to many to cities and other places...

Yet - here we are, people wanting to use them. And the data that could help make these weapons "safer" rotting away. I'm not sure if that is a good or bad thing overall...

Fuck these weapons.

matthewmcg 2 hours ago 1 reply      
This is important work. I would also recommend any of the Peter Kuran documentaries (Trinity and Beyond, Nukes in Space, Hollywood's Top Secret Film Studio).

What's really interesting about these is that there is a visible frame count on some of the videos and that you get to see a few frames before the detonation. E.g. the Androscoggin test opens on frame "-8" with the first flash visible at frame 0. You can see this by pausing the youtube videos and advancing frame by frame with the "." key.

There are a whole bunch of really interesting but rarely-seen effects that occur in those first one or two frames.

Here's an example from a different source of the "Teller Light" that immediately follows the initial release of gamma radiation from the initial nuclear fission in the bomb core: http://www.americancrisis.us/images/2013_08_30_013709_1_tell...

There's a neat description of this in a paper from 1958 here:

"The glow is known begin immediately (well within a shake) after gamma irradiation of the air, and has an apparent brightness of the order of that of the sun."


It would be really interesting to see the first few high speed frames of a detonating multi-stage weapon. Presumably this would reveal details about the sequence of energy flow from the exploding fission stage to the compressed fusion stages. Indeed, there were diagnostic pipes like this fitted to some of the early H-bomb test devices, e.g. http://nuclearweaponarchive.org/Usa/Tests/IvyMikeDevB1600c20...

Sadly, all of the multi-stage tests I have seen in public sources begin a few frames after detonation, or, like these, they are shot from far enough away that you just see a single diffuse glow.

jacquesm 4 hours ago 1 reply      
"It's just unbelievable how much energy's released," Spriggs said. "We hope that we would never have to use a nuclear weapon ever again. I think that if we capture the history of this and show what the force of these weapons are and how much devastation they can wreak, then maybe people will be reluctant to use them."

That's exactly why they likely will be used. So it serves as a deterrent right up to the point where it serves as an advertisement.

Interesting bit of un-intended poetry in there: The nitrate film used to store the images is itself rather explosive, so here is one explosive used to record the effects of another:


DonHopkins 2 hours ago 0 replies      
Tom Jennings is going to love this. He collected these device test images from wxvax7.esa.lanl.gov:


"In the late 1990's LANL (Los Alamos National Laboratory) had a public web server running on what later turned out to be a classified machine. I don't think any security breach occurred, but one day the machine simply disappeared. I wrote to the webmaster (back when you could do such a thing) who told me an audit discovered this alarming state and it was shut down.

Luckily I'm a packrat and occasionally use 'wget' to snarf down the contents of a website. This is one of those times. The images aren't anything exclusive, just a good set directly from the horse's mouth..."

Tom has a thing for cold war nuclear and computing machinery. He also created FidoNet and Homocore.



mixermf 4 hours ago 2 replies      
The closing quote: "The legacy I'd like to leave behind is basically a set of benchmark data that can be used by future weapon physicists."
epalmer 3 hours ago 0 replies      
My dad was 10 miles away from the explosion at Bikini Atoll. He was on a ship. I wish I had spoken to him more about it. He just did not want to talk about his work on the Manhattan Project or as a civilian at Bikini Atoll. I did find out he measured radiation levels on the Atol 24 hours after the explosion. He did not get cancer and lived a healthy life to 82 years old. Many of the people he was with did get cancer at a relatively young age.
AndrewKemendo 2 hours ago 0 replies      
I realize that the article indicates that these old films were just lying around untouched for decades, but what you are missing is that this data is "declassified." That doesn't mean it wasn't analyzed between the 50's and today, or that it doesn't exist in other media forms in classified channels - which it does.

The US government does a lot of replication of effort because most agencies don't know what the other agencies are doing. The DOE/CIA/DOD/DOS NBC program is massive, and the simulations and modeling teams have all of the extracted RAW data that exists on past tests, including the ones shown here.

I think this is a great effort, but all the teeth gnashing about how all of this data would be otherwise lost if not for this effort is - at least in this case - misguided.

patorjk 5 hours ago 1 reply      
Should these films be marked as public domain? I noticed the license was "Standard YouTube License" for all of the ones I checked on the playlist.
j2kun 1 hour ago 0 replies      
I interned at LLNL and went to a few talks where they discussed small-scale physics first-principles about how the interior of a star works, and I was shocked to discover that these videos were basically all that they had to go off. (And NIF, though each firing of the laser is extremely expensive and contained to a tiny chamber)
linkmotif 4 hours ago 0 replies      
This is amazing work. Thanks for posting. Reminds me of the people who rescued the lunar orbiter tapes from a McDonalds.

I just don't understand why every little 3 minute video needs car commercial music in the background. (re: https://www.youtube.com/watch?v=pWpqGKUG5yY)

taude 3 hours ago 0 replies      
I highly recommend everyone watch the two part series on PBS on Uranium - Twisting the Dragons Tail [1].

I found it really interesting the amount of money we spent to develop the Atomic bomb, and I think about today, if we invested the same percentage of GDP budget and went all in with alternative energies...

[1] http://www.pbs.org/program/uranium-twisting-dragons-tail/

ghaff 4 hours ago 1 reply      
I don't know about these ones specifically but a lot of the nuclear test photography was done by Doc Edgerton and EG&G.


overcast 4 hours ago 1 reply      
This footage is always so scary, and beautiful at the same time. Amazing stuff. I can't imagine what it was like seeing the first test go off. The feeling those people must have had about the future.
ceejayoz 5 hours ago 2 replies      
This one's pretty remarkable: https://gfycat.com/WeirdPoisedFairybluebird
JustSomeNobody 3 hours ago 0 replies      
So, he is able to himself declassify them or he is able to get them declassified?
chrisper 3 hours ago 1 reply      
How did they keep these tests secret? Surely you could see these clouds from far away?
vernie 2 hours ago 0 replies      
Why do the YouTube videos have such low resolution?
random3 4 hours ago 0 replies      
Wow! Watch starting from 1:07 in slow motion (0.25) and as it goes you can see (I believe) the shockwave expanding starting at ~1:11 the ground gets gradually lighter colored and when it reaches the camera it shakes it.
hamburglar1 4 hours ago 0 replies      
I wanted more explosions. SAD
Announcing Rust 1.16 rust-lang.org
128 points by steveklabnik  1 hour ago   19 comments top 5
AsyncAwait 43 minutes ago 0 replies      
Probably the most significant addition in this release is the cargo check subcommand, which does all the type/safety checking part of rustc, but skips the LLVM part of generating the actual binary.

If you just want to make sure you code compiles, this is the command for you.

There's also a bunch of API stablilizations around Strings, Vectors etc, complete changelog: https://github.com/rust-lang/rust/blob/master/RELEASES.md#ve...

Congratulations to everyone involved on the release!

FlyingSnake 46 minutes ago 4 replies      
I might be a minority here, but can someone please update me on there state of Rust on iOS? I develop C++ cross platform iOS/Android apps and would love to use Rust if possible.

The last time I checked Rust didn't support .frameworks creation, didn't support Bitcode and calling the main thread from Rust wasn't easy.

I would love to hear if these issues have been resolved.

ComputerGuru 2 minutes ago 2 replies      
This releases fixes the #derived mess, right? Meaning we don't need to mess with weird build scripts to get serialization and json support?
hawkice 29 minutes ago 2 replies      
I like rust a lot, but even hello world web apps compile (and recompile) so slowly. I barely get a single step down my to do list before recompiles take more than a second, and it makes it functionally unusable compared to other tools. I know this sounds silly, but it's true. I don't want the check, I want to see the webpage changed after my code changes. I keep hearing about incremental compiles but when I used nightly and tried turning it on, I guess it didn't work because compile times barely changed.
shock 1 hour ago 1 reply      
I'm eagerly awaiting a release of RLS that is installable via rustup.
Launch HN: Effective Altruism Funds (YC W17 Nonprofit)
66 points by tmacaulay  1 hour ago   13 comments top 6
SEMW 1 hour ago 2 replies      
Congrats on the launch!

> Weve found that some of the most cost-effective charities can buy one year of perfect health (a QALY) for as little as $80

Which was that? IIRC GiveWell only claim $100/QALY for the AMF - have you found a charity you believe is 20% more efficient than that, or is this just a difference in how you measure cost per qaly?

[Edit] another question -- there's recently been criticism[0] of the poor quality of data and effectiveness research in animal welfare EA, compared with human welfare EA. Would the animal welfare fund be doing things like actively commissioning new research into intervention effectiveness? Or would it be more hands-off, just limited to picking charities based on the data that's out there at the moment?

[0] https://medium.com/@harrisonnathan/the-actual-number-is-almo...

andy_ppp 1 hour ago 2 replies      
This is really interesting and a great idea; however, I'd almost trust charities more if they listed how they had failed each year and where delivery and services went wrong and what mitigation has been put in place.

It would be good to have a system where charities were allowed to fail like bad companies as well, but it's difficult to be this honest.

jrysocarras 17 minutes ago 0 replies      
Caveat: I know little about this area, so my question may be rooted in naivete.

Q: Why are you operating as a nonprofit? I would be willing to pay a reasonable management fee to you in exchange for the assurance that my money will likely have a greater social impact. If you were able to charge management fees, wouldn't that give you a greater ability to grow and attract talent?

DodgyEggplant 28 minutes ago 0 replies      
+ 1000 For the animals welfare. The animals that can't blog, tweet, or complain. Wildlife preservation and care for animals is as important as other common non-profit and foundations goals, but often neglected
a_w 1 hour ago 1 reply      
Great idea and I wish you all the best.

Assuming this becomes very successful, what effect will this have on new non-profits if most people adopt this approach? Will it make it difficult for them to raise funds?

wcgortel 1 hour ago 1 reply      
Congratulations on your launch! Well done. A great idea that needed to happen. Looking forward to kicking the tires on it.
Police ask for whole city's Google searches, and a judge says yes citypages.com
339 points by johns  3 hours ago   186 comments top 30
unit91 2 hours ago 22 replies      
I'm trying to see what the big deal is here (relax, I'll explain).

Police are not asking for the entire search history of everyone in the town, then combing through it to see what they've searched for. Rather, police are asking Google who, in a narrow amount of time, searched for the name of a relatively unknown person who had $28K stolen from him.

Is this really that different from police asking for security camera footage from a convenience store after a robbery? In either case, obviously innocent bystanders will be quickly eliminated from the list of possible suspects. Any remaining suspect(s) will still need to be (1) charged, (2) have a judge allow the evidence in court, and (3) convicted on the basis of evidence beyond reasonable doubt by a jury of their peers.

Unfortunately, I think obvious invasions of privacy (e.g., PRISM) have made folks very jumpy about any electronic evidence collection, which I don't think is warranted.

EDIT: Thanks to guelo for pointing out that this was an unsuccessful attempt at wire fraud, so the victim (fortunately) retained the $28K. I missed that previously.

milesf 2 hours ago 8 replies      
Which is why I use ddg.gg (DuckDuckGo). They don't track you.

People are starting to wake up to the implications of a world without privacy. I carry around Snowden's quote on my phone and repeat it verbatim when people say they have nothing to hide:

"Arguing that you don't care about the right to privacy because you have nothing to hide is no different than saying you don't care about free speech because you have nothing to say".

Do not be discouraged. Just keep educating people one-by-one and we will win this fight. We have to, otherwise our ancestors who fought and died for freedom and liberty will have done so in vain.

danjoc 1 hour ago 0 replies      
The police aren't asking for a city's searches. They're asking for anyone, anywhere, who searched Google for 'Douglas ? ?' in a 5 week period. The article this one links to explains this much better.


Google isn't the only search engine on the internet, but the police claim they were unable to find the information in Yahoo or Bing results. They suspect it must have come from a Google search, because that's where they were able to find the same information. The judge felt this provided probable cause to permit the search on Google.

This article is bad. It appears to be written by a layperson who doesn't understand law or how the internet works. And it pours on the hyperbole to excite emotions.

Fake news.

ars 2 hours ago 1 reply      
This title is click-bait.

They did not ask for "whole city's Google searches", they asked for the identity of the "person who searched for a specific search string", which is not the same thing!

Suggested new title: "Edina police ask for identity of person who searched for specific name, and a judge says yes"

ascendantlogic 1 hour ago 0 replies      
I'm not too outraged over this one. They're asking who in a specific city googled a specific name during a specific timeframe. This is how you find out who your suspects are. I understand it's a razor thin line between "normal police work" and "dragnet surveillance" but this time I think it's the former.
k-mcgrady 1 hour ago 0 replies      
>> police want to know who has searched for a particular name [douglas] used as part of that fraud

This is a super click-bait headline. Sounds like Police got the search history of everyone in the city. The reality, which is quickly explained in the article, is nothing like that. They have a list of people in the city who searched for a pretty specific term and it sounds like Google does the digging and gives them a list of names.

Whether that's a problem or not is still up for debate but that headline is ridiculous.

tabeth 2 hours ago 4 replies      
Curious: if anyone thinks it is bad for Google to give the police information, do you think it's bad for Google to have the information in the first place?

Another way to think about this is: when (not if, but when) Google gives a government information in some time in the future, would that change your behavior now?

deckar01 1 hour ago 0 replies      
Here is the original source with more info on the actual crime they are investigating and the reason they are targeting Google.


Edit: I find it rather amusing that the cops found the image used in the fake passport on google images, but decided to request google searches for the name instead of access logs for the image on the actual server.

mgarfias 2 hours ago 0 replies      
I'm not sure living in the future is worth this.
joshmn 1 hour ago 1 reply      
Minnesotan here:

The inside joke about Edina is that it stands for "Every Day I Need Attention"; it's mostly old money, old-ways of doing things. For example: Up until a few years ago, they outlawed happy hour (as a state, we recently approved liquor sales on Sunday; odd, I know) and no, I'm not kidding.

Having said, this doesn't surprise me at all. Not that a judge approved it, but that Edina of all places requested it.

coolsdude2282 51 minutes ago 0 replies      
This may sound silly to you all but I have recently given up paying for internet at my house as an experiment due to the prevalence of free public wifi nearly everywhere. I'm about 9 months into having cancelled my internet/cable entirely and it is working out well for me. I have only ever been very mildly inconvenienced by it.

Things like this make me never want to go back in all honestly, I save quite a bit of money with basically no downside (unless you count buying very very slightly more coffee than I otherwise might). Since I don't use google logins, disable tracking/cookies, and also change my mac address semi-regularly this seems like an adequate solution without going full on Tor/NoScript all the time(which I would prefer but the inconvenience cost becomes too great).

If this is allowed to legally stand, I am going to have to think long and hard about ever subscribing to an ISP or creating a google account ever again.

m00dy 2 hours ago 1 reply      
If Google gives that information, I will stop using it right away
r721 1 hour ago 0 replies      
I wonder which is the case here:

1) Google will query some internal search logs

2) Google will query a bundle of search histories which we see at "My Activity" page

ohstopitu 1 hour ago 0 replies      
This was unclear from the article...but from what I understand the police want info on anyone who searched a specific term on google for a specific image from geo-location/IP in that city a week before the crime took place ?

1. What if the criminal didn't use Google?

2. What if the criminal did use Google but used a VPN or Tor or a Proxy?

KindOne 1 hour ago 0 replies      
I remember seeing something like this online. I think it was murderpedia.org ?

Details are a bit fuzzy and might be wrong. In the 90's someone murdered a person or two. Somehow the cops asked the local ISP for anyone making constant searches of the case. They traced it back to the murderer.

forkerenok 2 hours ago 1 reply      
I wonder what principles a judge in the U.S. justice system? Is it only constitution?Can people somehow adjust what makes a judge? (speaking of competency in privacy matters)
nailer 1 hour ago 0 replies      
Unvoted and flagged as misleading:


> Police ask for whole city's Google searches, and a judge says yes

Implying searches not related to the case are included.


> In specific, police want to know who has searched for a particular name used as part of that fraud.

gondo 59 minutes ago 0 replies      
theoretical question: what if google will say that they do not know how to get this specific information out of their data? how can anyone prove if its true without full access to google infrastructure? will the police confiscate all their servers?
tdfx 2 hours ago 0 replies      
It does not mention what Google's response was, or if they have complied with the order.
Winblows69 1 hour ago 0 replies      
I totally agree with switching to DDG and ceasing all Google activity.

However, this is fear-mongering, slippery-slope-fallacy-ridden sensationalism. They targeted a specific name. The judge isn't going much further than that.

I get it - the police are potentially violating innocent peoples' privacy.

But for some reason, what this judge did doesn't overly bother this Google-hater / privacy defender. Maybe it's because the article is written for the National Inquirer.

mirimir 1 hour ago 0 replies      
Well, one would think that they'd have been bright enough to use StartPage/DuckDuckGo, a VPN, Tor, or some combination of them.
shmerl 1 hour ago 0 replies      
So, unconstitutional decision by the judge, and cops are surely happy to run with it.
ruleabidinguser 1 hour ago 0 replies      
Good motivation to continue security work
danm07 1 hour ago 0 replies      
That's it. Incognito mode, permanently.
gist 50 minutes ago 0 replies      
Mac addresses? Under what scenario does any provider (ISP even) end up with your mac address? (Am I missing something here that I don't know about?)

Understand that your router has your mac address and in theory your ISP could end up with it by getting it there if they wanted to but don't know of cases where they legally actually do this or have any need for the info. True?

jjawssd 1 hour ago 0 replies      
The end-game of Google (and Facebook, etc.) is to feed on everything everything you think, do, and say so they can predict and influence your beliefs and what you purchase. Of course, law enforcement wants a piece of the cake as well to be able to control and predict the behavior of the population. Break free from the hive mind. Disconnect yourself from the spying apparatus.

And now for a more amusing take on the matter: https://www.youtube.com/watch?v=sUIcCyPOA30

frugalmail 1 hour ago 0 replies      
We should have a new Technology court, where qualified hybrid legal/whatevertech folks preside.
ocdtrekkie 2 hours ago 0 replies      
The best solution for avoiding government surveillance overreach continues to be for nobody to store the data. A service provider can't be forced to disclose information it doesn't have.

I recommend duckduckgo.com for those bothered by this: "We dont store your personal information. Ever. Our privacy policy is simple: we dont collect or share any of your personal information."

stagbeetle 2 hours ago 2 replies      
If I understood the article correctly, the title is misleading.

The cops were not granted an entire raw database of all google searches by person. Instead, they were granted "'any/all user or subscriber information' of anyone in Edina who'd looked up that name between December 1, 2016, and January 7" which is IMHO a much less serious breach of privacy.

You could retort that collecting these users' "name(s), address(es), telephone number(s), dates of birth, social security numbers, email addresses, payment information, account information, IP addresses, and MAC addresses" is a serious breach of privacy, but I believe those are all available to law enforcement (save the internet-specific info).

canadian_voter 2 hours ago 9 replies      
I think we need to get over the idea of privacy. It's not technologically, politically or legally feasible to expect that anything you do on the internet can or should be kept from the authorities.

Big data is such an incredible tool for crime prevention. Minority Report is not a dystopian vision: it's a blueprint for a more just society.

If every "private" email, video conference and search is available for inspection and cross reference, we can end anti-social behaviour as we know it and achieve the kind of peace and harmony that only come from full accountability.

Crime thrives in dark corners. A surveillance society is a searing searchlight to expose and eradicate the the rot.

The question is: what comes next? If you bought a pressure cooker on Amazon a month before the Boston bombing, do police get to know about it?"

Absolutely. That is valuable information for an investigate. That doesn't mean everyone who bought a pressure cooked should be locked up, but that information can be used in conjunction with other information to narrow down a list of suspects. Legitimate buyers have nothing to fear and terrorists get caught.

An Upgrade to SyntaxNet, New Models and a Parsing Competition googleblog.com
267 points by liviosoares  3 hours ago   57 comments top 11
dang 7 minutes ago 0 replies      
We changed the title from "Google open-sources Tensorflow-based framework for NLP", which appears misleading, given that it happened last May: https://news.ycombinator.com/item?id=11686029.

On HN the idea is to rewrite titles only to make them less misleading (or less baity). Please see https://news.ycombinator.com/newsguidelines.html.

jacquesm 2 hours ago 6 replies      
I've been fighting Tensorflow in the last couple of days to try an application on it, never before have I seen such a convoluted build process and a maze of dependencies. The best manual on getting tensorflow with CUDA support up and running is here:


But it is a little bit out of date when it comes to version numbers.

If you're going to try TensorBox (https://github.com/TensorBox/TensorBox) it will get a bit harder still because of conflicts and build issues with specific versions of TensorFlow.

There has to be an easier way to distribute a package.

That said, all this is super interesting and Google really moved the needle by opensourcing TensorFlow and other ML packages.

qeternity 2 hours ago 2 replies      
Given how many areas NLP can be applied to, I can only imagine all of these future internal project proposals where someone has to explain to some C-suite exec how they are going to revolutionize the business with Parsey McParseface. Or better yet, when they have to budget a big upgrade to the "DRAGNN based ParseySaurus". Fun times ahead.
jakekovoor 15 minutes ago 0 replies      
This is definitely a game changer!

It's a very interesting research carried out by Google's research team and I believe this will be especially beneficial for future speech translation algorithms that would bring us a whole new, fresh experience with the way we converse with Alexa, Google Home, Siri, and many more.

If you need to install TensorFlow onto your Windows 10 computer then here's a great guide which I have followed quiet a few times. :)


minimaxir 2 hours ago 4 replies      
See also: spaCy, which is an open-source NLP framework that has some integration with Keras as well: https://news.ycombinator.com/item?id=13874787

...and apparently will release a major version update today. Ouch.

tlow 55 minutes ago 2 replies      
For those of us who aren't developers but maybe more aptly called "hackers" (cause we hack stuff together even though we're operating out of our league, sometimes we get stuff to work). I am wondering, is there a even higher level guide to using Tensor Flow. I am currently growing Sweet Peas in my office in enclosed containers that automanage environment, nutrition and water. I have the capaability to log a lot of data from a lot of sensors, including images. I have _no idea_ how I would even get started using Tensor Flow, but it would be cool if I could run experiments on environmental conditions and find optimal conditions for this sweet pea cultivar. Maybe I'm talking nonsense. Let me ask a more basic question, how might one log and create data for use with Tensor Flow. How might Tensor Flow be applied to robotic botanical situations?
devy 2 hours ago 2 replies      
"Python 3 support is not available yet." [1]. It's only supported in Python 2.7, Why?

[1] https://github.com/tensorflow/models/tree/master/syntaxnet

webmaven 3 hours ago 0 replies      
Very interesting release.

The bit about guessing the part of speech, stem, etc. for previously unseen words should (I think) make it much more useful in contexts that succumb to neologizing, verbing nouns, nouning verbs, and so on (such as business writing, technical writing, academic papers, science fiction & fantasy, slang, etc.).

I wonder how well it would do at parsing something that seems deliberately impenetrable, like TimeCube rants, or postmodern literary criticism.

pbnjay 2 hours ago 0 replies      
This looks amazing. I'm especially curious how well it will work at identifying Gene/chemical nomenclature since it is fairly consistent like English spelling. For named entity recognition in biomedical text this could be really useful!
canada_dry 2 hours ago 4 replies      
Hoping this will quickly make into someone's home grown self-hosted version of Alexa.

Alexa, turn the lights on in the kitchen.

Alexa, turn on the kitchen light.

Alexa, light up the kitchen.

Should all accomplish the same task using this framework.

camdenlock 25 minutes ago 0 replies      
> and to allow neural-network architectures to be created dynamically during processing of a sentence or document.

Oh lord, is this the spark that lights the google skynet powder keg

Web Scraping: Bypassing 403 Forbidden, captchas, and more sangaline.com
267 points by foob  6 hours ago   102 comments top 15
chatmasta 2 hours ago 3 replies      
Note that 99% of time, if a web page is worth scraping, it probably has an accompanying mobile app. It's worth downloading the app and running mitmproxy/burp/charles on the traffic to see if it uses a private API. In my experience, it's much easier to scrape the private mobile API than a public website. This way you get nicely formatted JSON and often bypass rate limits.
nip 3 hours ago 1 reply      
Scrapy is indeed excellent. One feature that I really like is Scrapy Shell [1].

It allows to run and debug the scraping code without running the spider, right from the CLI.

I use it extensively to test that my selectors (both CSS and XPATH) are returning the proper data on a test URL.

[1] https://doc.scrapy.org/en/latest/topics/shell.html

Lxr 3 hours ago 2 replies      
I have done a lot of scraping in Python with requests and lxml and never really understood what scrapy offers beyond that. What are the main features that can't be easily implemented manually?
superasn 4 hours ago 1 reply      
The web scraping tool of my choice still has to be WWW::Mechanize for Perl.

P.S. I wrote a WWW::Mechanize::Query ext for it so that it supports css selectors etc if anyone is interested. It's on cpan.

janci 2 hours ago 1 reply      
I use Java with simple task queue and multiple worker threads (scrapy is only singlethreaded, although uses async I/O).Failed tasks are collected into second queue and restarted when needed.Used Jsoup[1] for parsing, proxychains and HAproxy + tor [2] for distributing across multiple IPs.

[1] https://jsoup.org/[2] https://github.com/mattes/rotating-proxy

thefifthsetpin 2 hours ago 1 reply      
Better solution: pay target-site.com to start building an API for you.


* You'll be working with them rather than against them.

* Your solution will be far more robust.

* It'll be way cheaper, supposing you account for the ongoing maintenance costs of your fragile scraper.

* You're eliminating the possibility that you'll have to deal with legal antagonism

* Good anti-scraper defenses are far more sophisticated than what the author dealt with. As a trivial example: he didn't even verify that he was following links that would be visible!


* Possible that target-site.com's owners will tell you to get lost, or they are simply unreachable.

* Your competitors will gain access to the API methods you funded, perhaps giving them insight into why that data is valuable.

Alternative better solution for small one-off data collection needs: contract a low-income person to just manually download the data you need with a normal web browser. Provide a JS bookmarklet to speed their process if the data set is a bit too big for that.

foxylion 2 hours ago 8 replies      
I'm curious what others use to scrape modern (javascript based) web applications.

The old web (html and links) work fine with tools like Scrapy, but for modern applications which rely on javascript this does no longer work.

For my last project I used a chrome plugin which controlled the browsers url locations and clicks. Results where transmitted to a backend server. New jobs (clicks, change urls) where retrieved from the server.

This worked fine but required some effort to implement. Is there an open source solution which is as helpful as Scrapy but solves the issues provided by modern javascript websites/applications?

With tools like Chrome headless this should now be possible, right?

jacquesm 4 hours ago 3 replies      
Note that in some places this constitutes breaking the law.
m00dy 4 hours ago 4 replies      
I use greasemonkey on firefox. Recently, I have written a crawler for a major accomondation listing website in Copenhagen. Guess what? I got a place to live right in the center in 2 weeks. I love SCRAPERS I love CRAWLERS.
dmn001 3 hours ago 0 replies      
The first part seems like a very long-winded way to say "don't use the default user agent".

The captcha was unusually simple to solve, in most cases the best strategy is to avoid seeing it in the first place.

mirimir 2 hours ago 0 replies      
Sometimes it's also necessary to spread requests over numerous IP addresses.
herbst 6 hours ago 1 reply      
I've used antigate for captchas and ether Tor or proxies for 403s before. Usually the browser header alone does not help for long.
bla2 5 hours ago 2 replies      
Nice overview! The "unfortunately-spelled threat_defence.php" just uses British spelling though.
ouid 2 hours ago 0 replies      
too bad it's named for ovine prions.
Exuma 4 hours ago 0 replies      
Great article!
Acing the technical interview aphyr.com
45 points by alecsx6  2 hours ago   3 comments top 2
dasil003 15 minutes ago 1 reply      
This is beautiful. It captures the spirit of wizardry and wonder that brought me to programming in the first place, juxtaposed against the reality of interviewing at "top" tech companies where the interviewers more often than not have only a fraction of my experience, and seem to have followed the type of curated success-seeking golden path that used to be reserved for finance.
pkd 3 minutes ago 0 replies      
Thank you. I wish to someday walk into an interview and pull out Clojure as my language of choice and write every bit of code they ask me to as a pure function - and then look into the interviewer's eyes and try to decipher if they had seen the eternal light of Lisp yet.
GNU Guile 2.2.0 gnu.org
232 points by amirouche  6 hours ago   57 comments top 14
pkd 5 hours ago 4 replies      
For me this is the most exciting part:

Complete Emacs-compatible Elisp implementation

 Thanks to the work of Robin Templeton, Guile's Elisp implementation is now fully Emacs-compatible, implementing all of Elisp's features and quirks in the same way as the editor we know and love.
This means we can finally have a proper GuileEmacs!

mwcampbell 6 hours ago 1 reply      
Also check out this personal blog post from Andy Wingo, the primary developer:


didibus 1 hour ago 1 reply      
Can anyone tell me if Guile is relevant? The list of example programs written in Guile is small. EmacsLisp and not Scheme seems to be the Gnu lisp of choice. The VM is not the fastest and not the most portable. Is there any driver behind it?
avar 5 hours ago 2 replies      
For those excited about the "Guile's Elisp implementation" in this release. The last major Guile release was 6 years ago, and much of this GuileEmacs work is still highly WIP and from my searching on emacs-devel seems to have stalled in 2015 for lack of volunteers.

Just because Guile implements Elisp the language doesn't mean there isn't a ton of work to be done on Emacs itself to swap out its native VM for Guile, and it seems nobody's keen on finishing up that work.

porker 5 hours ago 0 replies      
For others looking for what Guile is:"Guile is an implementation of the Scheme programming language."
lloydde 6 hours ago 1 reply      
The mailing list announcement is better both for the detailed content and not being served as difficult to read on mobile "justified" text: https://lists.gnu.org/archive/html/guile-devel/2017-03/msg00...
davexunit 4 hours ago 0 replies      
This release has been a long time coming and I'm happy that the day has finally arrived. There's a small patch of mine in this release (my first compiler hack ever) that optimizes comparison operations for floating point numbers. If anyone is interested in hacking on compilers, I highly recommend checking out Guile as one of the easier points of entry into the space. Andy Wingo, the author, even wrote up a blog post with plenty of project ideas to improve things: http://wingolog.org/archives/2016/02/04/guile-compiler-tasks
hisham_hm 5 hours ago 0 replies      
Congratulations to the Guile team for the release! They're great people, and the Lua community is happy for having shared a devroom at FOSDEM with them for two years in a row.
davexunit 4 hours ago 1 reply      
To try out Guile 2.2.0 easily from any GNU/Linux distro (from the full release notes):

 Bonus track! This release also contains a new experiment, a binary installation package for the x86_64 architecture. The GNU Guix project (https://guixsd.org/) has assembled a graph of package definitions (for example, GCC, glibc, Guile, and so on) and is able to build that graph in an entirely deterministic way starting from only a handful of trusted bootstrap binaries. Guix recently added a "guix pack" facility that can export build products from a Guix system, including all run-time dependencies. We have used the new "guix pack" to generate an experimental binary distribution for the Guile 2.2.0 release. If you are on an x86_64 system running GNU/Linux, begin by running the following commands: wget https://ftp.gnu.org/gnu/guile/guile-2.2.0-pack-x86_64-linux-gnu.tar.lz wget https://ftp.gnu.org/gnu/guile/guile-2.2.0-pack-x86_64-linux-gnu.tar.lz.sig gpg --verify guile-2.2.0-pack-x86_64-linux-gnu.tar.lz.sig If verification fails, then see above for instructions on how to import the appropriate GPG key. For reference, the pack's sha256sum is: c707b9cf6f97ecca3a4e3e704e62b83f95f1aec28ed1535f5d0a1d36af07a015 guile-2.2.0-pack-x86_64-linux-gnu.tar.lz Then in your root directory -- yes! -- do: cd / sudo tar xvf path/to/guile-2.2.0-pack-x86_64-linux-gnu.tar.lz This tarball will extract some paths into /gnu/store and also add a /opt/guile-2.2.0 symlink. To run Guile, just invoke: /opt/guile-2.2.0/bin/guile Voil!

stonewhite 6 hours ago 1 reply      
Anyone tried GuileEmacs with this version and noticed any differences?
bitwize 2 hours ago 0 replies      
Congrats to the Guile team. Remember -- Guile goes with everything.
nerdponx 5 hours ago 1 reply      
How does this stack up against all the other popular Lisp variants out there?
_pmf_ 6 hours ago 1 reply      
Any word about Windows support?
andrewvijay 6 hours ago 9 replies      
Why are those brackets there in the syntax? What's the need? It looks hard to read when the programs are bigger. Is there any super advantage to it?
The Legal Professions Favorite Attorney: Vinny wsj.com
19 points by okfine  1 hour ago   1 comment top
pkamb 35 minutes ago 0 replies      
Great name for a law-advice chatbot.
False information on the internet is hiding the truth about onions marketplace.org
86 points by howsilly  3 hours ago   40 comments top 12
aluminussoma 1 hour ago 5 replies      
I recently created a web page that provides - in my biased opinion - unique, relevant information on a niche topic that I'm interested in. It's been impossible for me to crack the top search results on this topic. Instead, the top 20+ results are all news articles repeating the same thing.

I never had reason to complain about Google's algorithm until I became a content creator. Now, I wonder what other great web pages I'm missing out on.

Gravityloss 1 hour ago 0 replies      
This is about way more than onions. We are always in the danger of decline and regression to the dark ages. The world of rumors, beliefs, marketing and magical thinking is always there, biding its time.
dkrich 1 hour ago 1 reply      
The problem, IMO, is introduced when Google tries to provide definitive answers for questions that don't have definitive answers. To be sure, there are lots of questions that have definitive answers that would correctly satisfy 99% of searchers (what is the speed of sound, what is the height of the Empire State Building, etc.).

Then there are questions that require explanation and don't have a binary correct/incorrect answer. Even with the question serving as the example for this article: "How long does it take to caramalize onions?" Well sure, the author cited in this article adamantly claims it takes much more than ten minutes. But does it? Maybe there are equally or more qualified people who say that under normal circumstances and a certain heat level it doesn't take more than ten minutes to caramalize an onion. Who is right? I don't think Google can parse the available information to provide a "correct" response, and shouldn't try to.

So to me a better solution would be to categorize queries based on whether they can be answered definitively and, if they cannot, don't attempt to.

sleepychu 1 hour ago 4 replies      
> Ryssdal: Now, we should say, that in the days, what, week or so since this post was published, Google has in fact changed the search results, right? So, you've changed two things: You've changed the New York Times and you've changed Google.

This is still a problem with this specific example today.[0]

[0] - http://imgur.com/a/2R5k5

diego_moita 28 minutes ago 0 replies      
So, it isn't very different from the Big Lie idea[0]: if you repeat a lie long enough it becomes truth.

It works for humans, it works for algorithms.

[0] https://en.wikipedia.org/wiki/Big_lie

sgs1370 2 hours ago 2 replies      
For great onion techniques see "Ruhlman's Twenty: 20 Techniques, 100 Recipes, A Cook's Manifesto" by Michael Ruhlman. Before this book I didn't know there was a difference between sweating them and caramelizing them.
scandox 1 hour ago 0 replies      
> Because bad information that people passionately believe has a way of rising to the top.

Great single line summation of the whole problem.

mnx 55 minutes ago 0 replies      
The page seems slow/dead, so here's a mirror:http://archive.is/0Jc7G
coldcode 1 hour ago 0 replies      
Onions have layers and so does the truth apparently. Determining actual truth from a lot of vaguely related information is quite difficult. I've always found that there are things you can enter a search string for but you can't describe it sufficiently to find the actual result, just things that appear related somehow.
woliveirajr 1 hour ago 5 replies      
And I'd like a step-by-step recipe on how to caramelize onions, which ingredients to use, etc... Like "the final recipe".
peterwwillis 1 hour ago 0 replies      
Is nobody else disturbed by the fact that nowhere is anyone specifying what kind of onions to use? Vidalia, Texas, Walla Walla? Bermuda? Cvennes??
mtgx 28 minutes ago 0 replies      
One could even go as far as saying that "Google is fake news."

Now, I wouldn't say that myself, but Google brought this upon themselves. I've criticized them in the past about their "top answer" solution, as well about the AMP-powered carousel with (corporate-owned) "media partners".

They're setting up a system where there's a high chance that the top answer is indeed false, but Google acts as if it isn't - and that's the real issue here. Google envisions a future where its AI assistant will soon only give you that "top answer" as the "right answer".

I think it's wrong of Google to do that, at least until our AIs become smarter than humans, and can actually discern the truth way better than humans can from "reading" thousands of related articles/papers.

Foucaults work on power matters now more than ever aeon.co
23 points by Petiver  2 hours ago   6 comments top 4
golemotron 1 minute ago 0 replies      
The thing that Foucault misses is that there are many ways of seeing the world. You can see it in terms of power, but then confirmation bias will lead you to a place of deep distrust and paranoia.
macawfish 42 minutes ago 0 replies      
Last night I had a dream that I was telling someone to check out Michel Foucault. I specifically remember spelling their name and telling them they should look him up.
camdenlock 15 minutes ago 0 replies      
It... it really doesn't, and this article makes that fact quite plain. Foucault's philosophies are nuggets of well-worn common sense extrapolated out into billowing collections of unnecessary text.

I wonder if anyone has done the kind deed of whittling his most oft-cited gaseous emanations down to their core common sense origins. That'd save a lot of time for those of us not interested in a career in academia.

lutusp 36 minutes ago 2 replies      
Foucault? The physicist Foucault [1] responsible for the Foucault Pendulum and many other advances? Or the philosopher Foucault [2] responsible for some unfalsifiable fluff of great interest to philosophy department tea parties -- conversations that typically return to Jacques Derrida [3], the "show-stopper" who thought there was no basis for shared, objective truths, and whose views are regularly discussed as though their decoded meaning doesn't contradict the possibility of their productive discussion.

It's a sign of the times that, in Web searches, Foucault the philosopher completely buries Foucault the physicist.

(I got snookered into reading this because "power" has a meaning in physics apart from its meaning in philosophy -- the former of which actually means something.)

[1] https://en.wikipedia.org/wiki/L%C3%A9on_Foucault

[2] https://en.wikipedia.org/wiki/Michel_Foucault

[3] https://en.wikipedia.org/wiki/Jacques_Derrida

Programs that have saved me 100+ hours by automating repetitive tasks sadacaraveo.com
31 points by dshacker  1 hour ago   13 comments top 6
AdmiralAsshat 4 minutes ago 0 replies      
I'd have to throw most of the standard UNIX utils in there: grep, awk, cut, sed, sort, uniq, and of course, vim.

Outside of the tech world, people seem to think that grabbing some columns out of a file and rearranging them or pasting them somewhere else is some kind of sorcery.

tombert 41 minutes ago 4 replies      
Man, I'm pretty convinced that Vim macros have single-handedly saved me years of my life. They're so incredibly useful that it's made Vim one of the first things that I install on any machine.
kilroy123 6 minutes ago 0 replies      
Are there other sites with these kind of automating tips and tricks? I would love to automate more in my life.
mooreds 33 minutes ago 0 replies      
* 'set -o vi' in bash lets me search my command history with the same keys as moving around vi

* git + sql scripts

* simplenote (notes across devices) Semi-automation of pieces of my job captured in these notes.

* zapier if I'm trying to connect two web apis.

Kenji 2 minutes ago 0 replies      
Whoa, when I click on this link, it automatically closes the tab it resides in!

At first I thought this was an elaborate meta joke since not reading articles would save me thousands of hours, but I think it is a bug. I am using the latest google chrome with a very restrictive uMatrix that blocks pretty much all cross-site requests that are not images, and Ad-Block.

rc_bhg 25 minutes ago 1 reply      
Writing bash scripts and backing them up to github saves me tons of time.
Humans of Simulated New York: an exploratory comprehensive model of city life arxiv.org
45 points by matthiasdv  3 hours ago   12 comments top 7
coldcode 1 hour ago 2 replies      
Seems similar to the various Simcity designs. I wish it was available in a form we could interact with. When designing simulations like this of real life, among other problems the initial conditions are hard to generate since real cities grow in historical contexts, not at some recent starting point. I assume the initial conditions probably result in a fairly unpredictable and chaotic end state.
thetwentyone 2 hours ago 0 replies      
Even though you can't draw any meaningful conclusions from this, I think this is a great effort. The results are weird in the sense that the results in the scenarios don't resemble our current outcomes. A couple of comments on that:

1) Models are hard to get right. You need to find the right balance between simplifications and reality. The results indicate something is not calibrated or functioning the same as our current state.

2) Models are intended to be simplifications. We draw meaningful results from very simple models (e.g. supply and demand) but without modeling the complex interactions it's hard to say what impact could a certain policy have on the system. Agent-based models are a way to do this and the computing power available is allowing us to experiment with this.

Re: #1. We may find that modeling and seeing outcomes similar to our own is very hard to get to. This may be due to model specification, or it could be that given the rules that govern our own interactions and intentions, our current state may be the one we just chanced upon.

This is a hard challenge to tackle but I'm glad that the authors have published this and shared their code. Don't expect to have prescriptive solutions from it anytime soon though.

urschrei 2 hours ago 1 reply      
It's not completely fair to judge the quality of this paper by the first few references (8 and 25, in particular), but it does make me extremely dubious. It's not really appropriate to cite Ed Glaeser in an urban modelling paper (or anywhere, really, other than as a cautionary example), and that interpretation of Bettencourt and West won't bear much scrutiny either. I won't address thecontributions (I work in a department that does a lot of urban modelling and ABM it's cited in the paper but I'm not a modelling / ABM person myself)
danbruc 2 hours ago 0 replies      
Interesting what turned out to be the best scenario.

The scenario with Low technology seems to be the most reasonable one with an initial decrease in quality of life that rebounds in the later run of the model. That seems to happen, as prices remain much lower than those observed in the Average and Positive scenarios, although not at zero as the Negative one

Where the scenario Low technology is defined as follows.

a massive solar flare from the sun disables all electronic equipment

a bioengineered super-nutritional food is available

disease has been totally eliminated

JackFr 1 hour ago 0 replies      
I suggest it is a near certainty that these New Yorkers are living in a simulation.
coolspot 1 hour ago 0 replies      
That is how it begins.
jmcmahon443 2 hours ago 1 reply      
This is breakthrough. I'm telling you, it is a game changer, and it was coded by women -- which is also significant.

Simulation of human life combined with reality capture and BIM software are going to change the game. So-much-so that the market price of commodities like copper and aluminum are going to become of GLOBAL interest.

This could not have come at a better time than when scientists are freaking out about the Great Barrier Reef being almost completely dead.

Roger Penrose's Gravitonic Brains (1994) cmu.edu
29 points by pmoriarty  3 hours ago   1 comment top
deepnet 8 minutes ago 0 replies      
Worth noting this review of Sir Penrose's theory of minds is by Dr Hans Moravec, an important roboticist and AI theorist, widely known for Moravec's Paradox :

"it is comparatively easy to make computers exhibit adult level performance on intelligence tests or playing checkers, and difficult or impossible to give them the skills of a one-year-old when it comes to perception and mobility."[1]

which began the shift in paradigms exemplified by behaviour based robotics.

 [1] snipped from http://www.eugenewei.com/blog/2014/10/13/moravecs-paradox-and-self-driving-cars

A Brief History of the Grand Unified Theory of Physics nautil.us
47 points by pmcpinto  3 hours ago   11 comments top
grabcocque 2 hours ago 8 replies      
Something I always felt 20 years ago when doing my physics degree, and now 20 years later when no measurable progress has been made down the road of unification feels even more appropriate:

General Relativity and Quantum Mechanics are two astonishingly successful theories. They don't really have any massive gaps waiting to be filled. The belief that there must be a way to unify them always seems to me to stem from the human desire for tidiness, not any sound epistemological reasoning about how physical laws behave.

We want the theories to be unifiable because we're human. Sadly the universe doesn't much seem to care what we want.

When You Should Use Lists in Haskell (Mostly Not) htwk-leipzig.de
22 points by luu  2 hours ago   10 comments top 2
nilved 1 hour ago 2 replies      
The first-class status of linked lists in Haskell really bothers me. In Idris there's no special syntax for lists; lists of `a` are denoted `List a` and `[]` is simply syntactic sugar for the `null` function of that module. This means `List` and other structures like vectors are on the same level.
mixedCase 1 hour ago 1 reply      

Site needs to upgrade its certs.

Scrolling on the web: A primer windows.com
241 points by thmslee  10 hours ago   115 comments top 19
afandian 9 hours ago 12 replies      
I wish people who made webpages trusted browsers to scroll. The number of sites that play with the scroll speed and consistency by hijacking scroll events makes me doubt whether the designers spare a passing thought for usability.

Scrolling is one of those things that (in my opinion) get calibrated in the brain for hand-eye co-ordination. Which is why (again IMHO) there are religious wars fought over, for example, Apple's trackpads. This stuff is important. It can be a jarring experience when scrolling suddenly doesn't work. Flip side is, I'm sure some people don't care.

There's nothing more annoying than a web designer saying "I know better than you" and re-implementing features. Because they're usually wrong.

EDIT: Would people accept it if each webpage re-adjusted your mouse pointer speed and acceleration? Is there any difference between one mouse function and another? Why do web designers think they have the right?

modeless 8 hours ago 4 replies      
Back in the day browsers competed over user friendly features like popup blockers, even though that technically breaks a web API. Today browsers are so terrified of breaking a web API that they have allowed the mobile web to become a cesspool. Browsing most sites has become intolerable, the more modern the worse. Scrolling freezes for seconds at a time, whole pages jump up and down constantly while you read, scroll-hijacking ads interrupt you, fixed position headers obscure half the page and appear/disappear at random, "mobile-friendly" sites are more often than not worse than desktop sites even on mobile (plus they break deep links), and I can't be the only one who's noticed that even the old popup blockers have stopped working!
onion2k 7 hours ago 2 replies      
This is a fantastic piece of technical writing. There's plenty of detail and pretty much no fluff. More of the same please Microsoft.
romaniv 2 hours ago 0 replies      
I'm not sure why anyone is using scrolling events for any standard websites at all. It's much easier to have a function on timer that detects the current viewport position and acts accordingly. The code is simpler, the performance is better, and it's way more future-proof.
stupidcar 7 hours ago 3 replies      
And yet there is still no way in CSS to, say, freeze a table header and make the body scrollable. Or freeze one column and make the rest scrollable.

For all its advances, the web platform is still incredibly primitive in some areas.

nuclx 8 hours ago 1 reply      
I'm happy, if I'm able to scroll at all. Hate the trend of dynamically loading content when scrolling down.
codedokode 3 hours ago 1 reply      
It looks like a poor design desicion made 20 years ago (that wheel event is synchronous and must be run before scrolling) leads to all major browsers code becoming unnecessary complicated. I think it would be better to discard that design and make a new event system that would be used with new pages (and don't waste developers' resources to optimize old pages).

Maintaining compatibility is good, but at some point we need to stop and make a new design.

By the way here are the things I'd like to get fixed too:

- make a distinction between same-domain and cross-domain requests (use another HTTP method for cross-domain POST request) so all sites get XSRF vulnerability fixed

- make cross-domain requests anonymous by default

- make cookies unaccessible to Javascript by default

- stop JS loading from blocking the page so we can put script tags in the header where it makes more sense

- make as much events asynchronous as possible

- fix keyboard events, key codes and mouse buttons numbers

- make it impossible to change window.opener.location and windows.parent.location

- add CSS rules to set width for a column in a table, not for cells

- add error reporting so if some resource (like a script) fails to load the user gets a message that the page is broken

anotheryou 7 hours ago 1 reply      
What I really want for spacebar scrolling is this in working: https://greasyfork.org/en/scripts/20937-red-read-line

An indicator on where the bottom of the screen scrolled to, so I don't need slow animations or incremental scrolls to find where I left off reading.

(The linked script is not working on all pages and sometimes blocking klicks, I just threw it together to try if the Idea works. I think the principle works really really well.)

makecheck 3 hours ago 0 replies      
The only thing worse than having a broken scrolling experience on mobile is imagining just how much extra data was wasted to download that useless script so that a crappy mobile experience could be had, and how much battery was wasted running the unnecessary script so that a crappy mobile experience could be had. There is complete priority inversion on the web and I am so sick of it; my device should have ample controls to obey ME and not the whims of some random site.
m_fayer 5 hours ago 0 replies      
This is the kind of "accessible take on fundamentals that you might not have even realised were fundamentals" documentation that I'd like to see more of. It took me a few years of web development to even start thinking about just how, exactly, the concurrency model of the web functions.
jstimpfle 6 hours ago 2 replies      
Bold claim (not a professional webdesigner): Many problems with performance and usability would go away if people made more websites where the document is 100% width and height and contained boxes are individually scrolled. ("Like frames").

Basically, the way it's typically done when using desktop widget toolkits.

But most websites are still made so that there is only one document-wide scrollbar.

joshiefishbein 4 hours ago 0 replies      
The constant finger-pointing to the shortcomings of other browsers was getting particularly annoying especially when IE and Edge are historically the lowest performing, late-to-the-game browsers.

Nice article, but comically ironic.

syphilis2 3 hours ago 1 reply      
I feel that the mechanics of scrolling need to be reworked. I have to do a number of tweaks to get free scrolling to work as I'd like, as opposed to the default line-by-line type scrolling. Even line-by-line scrolling has issues, such as not being aligned with the lines of text or block elements on a web page. Scrolling ought to have different control options as well, say ^Scroll scrolls through paragraphs or headings.
combatentropy 1 hour ago 0 replies      
I prefer to scroll with the spacebar. Therefore fixed headers are the enemy (like in this story!).
RyanMcGreal 7 hours ago 1 reply      
Theres a curious anomaly, though: if you try to scroll using touch screen scrolling, the page happily moves up and down, even while JavaScript is blocking nearly everything else on the page. This also works for touch pad scrolling, mouse wheel scrolling, and click-and-drag scrolling (depending on your browser).

Mouse wheel scrolling still works on Chrome but not on Firefox.

barrystaes 6 hours ago 3 replies      
I would think that "NOP" code like that example would be optimized out:

setInterval(() => { var start = Date.now(); while (Date.now() - start < 500) {/* wheeeee! */}}, 1000);

_pmf_ 5 hours ago 0 replies      
My favorite is dedicated mobile sites that do animations on scrolling.
teknopaul 8 hours ago 0 replies      
page is not readable in Opera mini. Lolz.
juiced 5 hours ago 0 replies      
Offtopic, but how come, when I see this website is on windows.com, that I expect it to load slowly?
Voltaire and the Buddha publicdomainreview.org
7 points by Vigier  1 hour ago   1 comment top
pmoriarty 30 minutes ago 0 replies      
That website is full of absolutely fascinating articles. I want to post them all to HN, and maybe I will. Thanks!
DeepStack: Expert-Level Articial Intelligence In Heads-Up No-Limit Poker fermatslibrary.com
87 points by apetresc  6 hours ago   19 comments top 5
LeanderK 3 hours ago 2 replies      
How can one create visualisations like Figure 3 (assuming it's latex)[0]? I have to write my Bachelor thesis soon and need to improve my latex-skills. I only know to to use the math-mode and write text.

[0] the paper: https://arxiv.org/pdf/1701.01724.pdf

andr3w321 4 hours ago 1 reply      
Anyone know where to find the pseudo code? It references item 10 in the References and Notes which states See Supplementary Materials.
maaaats 4 hours ago 0 replies      
Pretty helpful with the annotation along the way, explaining mbb/g and what it means, for instance.
PaulHoule 4 hours ago 1 reply      
One thing I haven't seem come up in discussions or media coverage about this is that the heads up game is entirely a different thing than playing with 5 other players.
IMTDb 5 hours ago 3 replies      
This seems to be a major threat to online casinos.Is there any study on how they plan on fighting the AI's that will likely be flooding the market soon ?
How 1,600 People Went Missing from U.S. Public Lands outsideonline.com
51 points by jgrahamc  4 hours ago   31 comments top 14
jonnycoder 1 hour ago 0 replies      
"he lists such recurring characteristics as dogs unable to track scents, the time (late afternoon is a popular window to vanish), and that many victims are found with clothing and footwear removed. Bodies are also discovered in previously searched areas with odd frequency, sometimes right along the trail. Childrenand remainsare occasionally found improbable distances from the point last seen, in improbable terrain."

I've been hunting for 7 years and my first thoughts are that:

-Found without clothing and footwear can indicate hypothermia

-Improbable distances and terrain is not a surprise. I've felt mild panic set in when I wasn't where I was suppose to be in terms of meetup spots and felt the need to rush uphill or avoid using common sense in the moment. I've also scaled some scary terrain (I'm no rock climber or mountain goat) and I've heard stories from guys I hunt with who have done some stupid scary stuff when it comes to terrain

-The other comments read like a bad movie that tries to scare us into believing the forest is the most dangerous place in the world other than your own sidewalk, where a stranger will kidnap your kids at the most opportune moment. Most of my coworkers throughout the years all think I'm crazy when I tell them I go hiking and hunting in the forest, as they believe I'll be eaten by bears. By the way, I saw a baby black bear about 15 yards from me last season, cutest thing in the world, and so was his mother as I was walking away :)

cwbrandsma 2 hours ago 1 reply      
I live out west. It isn't that hard to start going in a direction and not see anyone for days.

My brother volunteers for Search and Rescue (Search and Recover most of the time). If someone dies in the desert, because of the sage brush and other grasses, you can easily walk right past a body and not know it (assuming there is no smell)

scarhill 1 hour ago 0 replies      
It's easy for even a big search to miss someone, even when that person is alive. Geraldine Largay got lost on the Appalachian Trail in Maine, survived for 26 days, and died less than 2 miles from the trail, yet a large search didn't find her, and her remains weren't found for more than two years after her disappearance:


barking 1 hour ago 2 replies      
All that searching and his body found only 1.7 miles from the ranch defies belief. Also the fact that he was found in the direction in which he had set out. It really sounds like the search wasn't systematically done at all.
jessaustin 2 hours ago 2 replies      
...Interior tried to build its own database to track law-enforcement actions.... The result, the Incident Management Analysis and Reporting System, is a $50 million Database to Nowhere last year, only 14 percent of the several hundred reportable incidents were entered into it. The system is so flawed that Fish and Wildlife has said no thanks and refuses to use it.

I know, this is normal, and one shouldn't get upset. It just feels like $50M is enough resources to accomplish quite a lot, if it were properly allocated. It feels like people should be in prison for this.

mutagen 1 hour ago 0 replies      
If you enjoyed this article, there's a rather deep rabbit-hole documenting the search for a German family that went missing in Death Valley about 20 years ago.


DanBC 1 hour ago 0 replies      
See also the Pima County Missing Migrants project. When a corpse is found in a desert a group of people try to identify who it used to be and inform relatives.



verytrivial 1 hour ago 1 reply      
Reminds me of http://www.otherhand.org/home-page/search-and-rescue/the-hun... another very compelling read.
davidw 1 hour ago 0 replies      
Completely tangential, but "faith, hope and charity" are also the 'names' of Oregon's Three Sisters mountains, more commonly referred to as South, Middle and North.
MK999 1 hour ago 0 replies      
I always intended to read "Missing 411" after hearing the author speak on the radio, but haven't made time for it.
nepotism2016 57 minutes ago 0 replies      
Sounds like first season of True Detective
Zigurd 2 hours ago 0 replies      
>Joes body was 1.7 miles as the crow flies from the ranch. Searchers had been close. In November 2015, Keller and David Van Berkum had come within several hundred yards. I regret not searching there on the 25th of July, Keller told me. Thats where I wish Id started. What part of here would take a life? Its not the meadow on top; its the cliff.

In populated areas you can skid off the road into a pond and not get found for years, if ever. When you hike even heavily travelled trails, all kinds of wildlife is just outside your range of perception along the trail. It's easy to stumble upon a bear's den, for example. Missing skiers can be 10 meters into the woods, off a marked trail, in a hole at the base of a tree.

Searching for a cold corpse in a fairly wild area is a low-odds roll of the dice, not a determination that a body isn't there. Unless you have something like a mobile phone or body heat that can be sensed at a distance, you have, at best, a well-trained dog's nose at ground level, limited by where the dog and handler can reach. Human senses have much lower odds of success.

swayvil 2 hours ago 5 replies      
Strange stuff happens in the wilderness. Strange beasts sighted. People disappear.

Some say that it's because what we call "reality" is, to a significant degree, conjured into being by our collective attention. So the edge of the collective (civilization or whatever) is the edge of reality.

mkstowegnv 2 hours ago 0 replies      
There is nothing surprising, technological or policy relevant in this article. Although better than a previous similar HN story (see https://news.ycombinator.com/item?id=12674976), it is just a run of the mill tabloidish Outside Magazine article
The eigenvector of Why we moved from language X to language Y erikbern.com
362 points by platz  14 hours ago   164 comments top 28
The_suffocated 11 hours ago 11 replies      
The research methodology in this blog post is fundamentally flawed. The author only counts how many people move from X to Y, but he doesn't count how many of them do not move at all. The whole diagonal of his (sample) transition matrix are actually missing values, but he treats them as zeroes. This greatly distorts the equilibrium distribution. As a result, he misinterprets each equilibrium probability as the "future popularity" of a language as well, when it at best only represents the future popularity of a language among those who constantly switch their languages.
Maro 3 hours ago 1 reply      
I wish more people would read 'Hack and HHVM', written by Owen Yamauchi, formerly a member of Facebooks core Hack and HHVM teams.


The hidden lesson for me was that rewriting the code in <new-language> is not the only option. Another option is to slowly improve the language/runtime itself until you've essentially switched it out underneath the application, which is what happened at Facebook. Meanwhile keep refactoring the code to take advantage. (Granted, this is isn't an option for a small company.)

I work at Facebook and sometimes write www code. When I was interviewing and thought about writing PHP code, I didn't get a warm and cozy feeling, being reminded of terrible PHP code I've seen (and written myself) in the 2000s as a "webdev". Thanks in part to the advances described in the book, the codebase is definitely not like that; it's easily the best large scale codebase I've ever seen (I've seen 3-4).

My thoughts in blog form (written 3 months after I joined):


jeremyjh 8 hours ago 1 reply      
I don't think the Google queries measure what the author thinks it does.

I noticed it lists 14 results from Haskell to Erlang, which I was skeptical of. When I google "move from Haskell to Erlang" or "switch from Haskell to Erlang" I do find results (such as quora questions, versus questions,lecture notes) but none of those results are the type of article we're looking for.

If they really want to do this, I think they also need to validate that some of those keywords are in the title of each page.

kilon 33 minutes ago 0 replies      
I always thought that there is no such thing as best programming language in the world. After all they only recycle the same recipe , again and again. Then I found out Smalltalk and realized how wrong I was.

There is nothing that comes close that can compete with the massive success that Smalltalk has been. Its blows my mind how it can be so much better than anything else out there including the usual suspects (Lisp, haskell, blah blah).

But in the end its not about the language , its about the libraries. Hence why Python remains my No1 choice.

In the end however even Smalltalk is terrible outdated. The state of software is in abysmal condition trapped in its own futile efforts of maintaining backward compatibility, KISS and do not reinvent the wheel.

In sort software is doing its best to keep innovation at a minimum and as such pretty much everything sucks big time and is still stuck in stone age.

I once considered becoming a professional coder working in a company doing the usual thing, I am glad I was wise enough not to choose that path. I would have killed myself right now with all this nonsense that makes zero logical sense.

But my hope is in AI, the sooner we get rid of coders, the better. Fingers crossed that is sooner than later. Bring on our robotic overlords.

Saying that I know a lot of people that really love coding and respect it as an art and science, so there is definitely hope.

ArneBab 2 hours ago 1 reply      
This looks pretty interesting. Most striking to me is that Go is taking from other 'target' languages. You can see the 5x5 block of the other strongest target languages giving to Go, but not taking from it. To make what the Eigenvector says explicit:

Top 5 giving to Go directly: C, Python, Java, Ruby, Scala

Top 5 giving to C: C#, R, Java', C++, Fortran

Top 5 giving to Python: C', Perl, Java', C#, C++

Top 5 giving to Java: C', C++, PHP, Python', C#

Top 5 giving to Ruby: Python', PHP, Perl, Java', (C++ only 215)

Top 5 giving to Scala: Java', Ruby, (Python', C#, PHP only 100, 17, 16)

': language also in top 5 givers to Go.

The other top languages take from each other (there is migration in both directions), but currently Go mostly takes here. However it does lose people to Rust which is actually the strongest go-to language from Go. And C++ does not give to go.

This might point to a discrepancy between Go marketing and reality (efficiency and replacing C++).

It would be great if you could repeat this exercise next year to see how things changed.

(besides: the script is nice and concise!)

jlrubin 12 hours ago 1 reply      
I appreciate that the author wanted to implement their own eigen vector/value method, but really they should use:

 numpy.linalg.eig(x)[0] numpy.linalg.eigvals(x)[0]
Numerical stability can be hard to get right...

fishnchips 5 hours ago 4 replies      
Am I reading this incorrectly, or there is more movement from Swift to Objective-C than the other way around? Do I sense a methodological error?
verytrivial 4 hours ago 0 replies      
I find it amusing that for Javascript frameworks the approximate end-state is perpetual oscillation between React and Vue.
rjbwork 7 hours ago 3 replies      
Highly skeptical of so many people migrating from C# to C, or python to Matlab, to give a couple of examples. This seems like a highly flawed methodology from many perspectives, as pointed out in comments.
tray5 13 hours ago 4 replies      
Well the fact that C is going so strong guarantees we'll be dealing with easily preventable bugs for the next 100 years
hasklel 5 hours ago 0 replies      
10000 vocal webdevs make a blogpost about moving from Node/Python/Ruby to Go because their app is slow as shit and the JVM isn't trendy enough for them. Also I wonder if I'm reading this correctly but are there actually people moving from Cassandra/DynamoDB to Mongo??
jaimex2 12 hours ago 2 replies      
Anyone else routinely roll their eyes at "why we moved from x to y" blogs?

They are always just "We wanted to do this in a particular way. So we fought the framework till we decided to move to another that does things the way we thought they should be done. Now things are much better but we will fail to mention down the line all the new compromises we have to deal with"

w8rbt 6 hours ago 7 replies      
I love golang, but I hate trying to search HN articles for the word go... I need a find go but not ago search button in FireFox ;)

grep go | grep -v ago

jstewartmobile 11 hours ago 1 reply      
I really appreciate his method of breaking it down per-niche.

When it comes down to it, all languages are DSLs. Even LISP/Scheme are DSLs for making DSLs (like Butterick's "Beautiful Racket" earlier today).

Presenting them as per-niche directed graphs is probably less likely to steer newbies (and sadly, not-so-newbies) into another round of "let's redo everything in X!"

santaclaus 13 hours ago 1 reply      
Who is the one person who rewrote their matlab homework in php?
chiefalchemist 2 hours ago 0 replies      
This is certainly interesting. Great for happy hour bullshitting, but too flawed to take seriously.

I'm not dismissing it. Just wanting to define proper context. Else some twithole will start a shit storm over null.

urs2102 14 hours ago 4 replies      
This is super interesting. It's also interesting to see the converse - who isn't moving anywhere. Go, Elixir, Dart, and Clojure all seem pretty happy!
hamilyon2 11 hours ago 0 replies      
Only relatively small project can afford a rewrite. So, this is statistic among projects that can affod a swith.And as far as I can see, this is eigenvector of trend. First derivative of actual state of things. More informative of the state of fashion today
brightball 5 hours ago 2 replies      
The Python/Ruby axis are interesting. You've got over twice as many Python to Erlang posts out there...and a fraction of the Python to Elixir's. Ruby has the opposite. A bunch of Ruby to Elixir and very few Ruby to Erlangs.

I wonder why that is?

Yuioup 12 hours ago 2 replies      
The author is surprised that angular is holding up. I've been learning angular2 the past few weeks after having never used a single page application framework before and I'm loving every second of it. I'm never going back to ASP.NET MVC except to use it as an API.
pjmlp 10 hours ago 2 replies      
I don't see any database that we actually care about.

SQL Server, Oracle, DB2, Informix.

z3t4 6 hours ago 0 replies      
It would be interesting to do the same thing but with Bing and see if C# and .net comes up first ... ;)
PDoyle 7 hours ago 1 reply      
> Surprisingly, (to me, at least) Go is the big winner here. Theres a ton of search results for people moving from X to Go.

You mean ... Google search results?

I'm not trying to suggest that Google's search engine is intentionally biased toward Google projects, but I think it's reasonable to assume that their own projects wouldn't fall into whatever unintentional blind spots their search engine may have.

peterwwillis 12 hours ago 0 replies      
It should be noted that the premise dictates this eigenvector is limited in scope. It seems to apply mostly to people who both wish to create products (usually for some commercial venture) and have a habit of saying things like "Well this looks hard. Let's try reinventing this wheel with different tools and see what happens."
wlllmunn 9 hours ago 1 reply      
I find it very hard to believe that no-one is moving to node.
iopq 9 hours ago 0 replies      
I half-expected Unicode to have an emoticon for this https://assets-cdn.github.com/images/icons/emoji/trollface.p...

But alas, this post had to use an image.

azinman2 2 hours ago 0 replies      
Oi. Another person who thinks the number of search results returned is a real number that means something.... the fact that it gives even plausible results is impressive as the number is made up by googles servers.
Square Service Outage issquareup.com
45 points by m_coder  43 minutes ago   13 comments top 8
dvcc 25 minutes ago 1 reply      
Being down for an hour as a payment processor is crazy. Going off some old figures [0], and assuming 0 offline transactions (and a bunch of other assumptions too), I think it is around ~$3,500,000 in unprocessed transactions?

Must be stressful trying to bring it back online.

[0] https://techcrunch.com/2014/01/13/putting-squares-5b-valuati...

jrobn 1 minute ago 0 replies      
per issquareup.com "Were still experiencing issues; however, we are seeing initial positive improvements in response to the steps we have taken to remove load from the affected service"

Could this be a DoS of some kind?

jrobn 7 minutes ago 0 replies      
We use Square as our point of sales system at our spa. We are biting our nails now since most of our sales are $75+ and people don't generally carry around that kind of cash anymore. Our iPad also suddenly got signed out of the POS app. Luckily my phone was signed in so I put it in airplane mode to kick it into OFFLINE mode.

You can't sign into the square dashboard either so access to square appointments on the browser is a no go.

askafriend 8 minutes ago 0 replies      
I just went to a coffee shop that I go to regularly and was confused when they said they're cash only for today. This explains why.

On that note, I also saw multiple people leave to go to a different coffee shop because they didn't have cash on them.

huangc10 11 minutes ago 1 reply      
Is the actual failure with logging in and creating transactions or with the checkout or is everything down? This seems like it'll be a pretty big blow especially with lunch soon in the west coast.

At least good old hard cash still works.

joez 21 minutes ago 3 replies      
How bad is this?

Seems like they have offline mode. Do their customers know how to use this? What's the chance for increased fraudulent swipes?

jvehent 13 minutes ago 0 replies      
If your service has higher SLA requirements than your providers contractually committed to, you're doing something wrong.
myowncrapulence 31 minutes ago 0 replies      
Been an hour.. wow. Is this a ddos on their auth services?
Archaeologists find early democratic societies in the Americas sciencemag.org
39 points by Thevet  4 hours ago   9 comments top 5
ravenstine 1 hour ago 2 replies      
On a similar topic, for those unfamiliar with the Iroquois Confederacy, the wikipedia article itself is an interesting read:https://en.wikipedia.org/wiki/Iroquois

In essence, they were a sort of "united states" before the United States we know today. Although the wikipedia article provides sparse evidence for this, it's theorized that the 6 Nations influenced the founders in shaping the early [western] American government.

So many fascinating aspects of American history and pre-Columbian have been overlooked, sadly, because we have tended to view the natives as either savages or "noble savages".

twblalock 1 hour ago 0 replies      
The contention about democracy in the title of the article is undermined by the disagreements between archaeologists that are cited in the article itself.

I know that the authors of the articles don't always get to pick the titles, but come on.

GuB-42 2 hours ago 0 replies      
I have a feeling this topic will degenerate quickly...
ch4s3 1 hour ago 0 replies      
This is an interesting read, and I particularly thought it was illuminating with respect to the Indus Valley Civilisation.
douche 1 hour ago 2 replies      
Every time one of these archaeology articles comes up, I'm reminded about what thin reeds most of these theories are based upon. It's the blind men and the elephant, except the pieces available for inspection are even smaller, and the conclusions more sweeping.
University of Tokyo Graduate School Entrance Exam Mathematics (2016) [pdf] u-tokyo.ac.jp
148 points by v3gas  10 hours ago   152 comments top 21
ninjin 6 hours ago 2 replies      
Goodness me, this brings back memories having taken the entrance exam back in 2009. I am glad that they have removed the "We do not guarantee the accuracy of the English translation of the original Japanese version of the exam" clause. It should be noted that this is only one "session" of a series of exams that you take in a single day. If I remember correctly there were two computer science sessions as well, followed by a presentation of your thesis in front of the faculty once your exams had been graded. After this you had to wait for a few weeks for the final result.

It should be noted that I don't find these exams to be particularly difficult, the main difficult lies in the fact that the number of potential topics are drawn from the whole University of Tokyo undergraduate curriculum, so if you studied as an undergraduate at a different school or country -- like me -- you are at a significant disadvantage. What I did was to look at ten years or so of exams to get some statistics on what was likely to be on the exam, then locked myself in my room for the whole of January to cover portions that I had not studied before -- like network protocol specifics. In the end, it worked out for me, but I got somewhat lucky in that there were two statistics questions that I could breeze through to cover for my weaker calculus skills given my old school's focus on discrete mathematics.

thearn4 7 hours ago 5 replies      
Some number theory, linear algebra, calculus of variations, combinatorics, etc. From the applied math side of things, it isn't terribly deep. It's actually right about what I would expect from an applied math student who was good at the undergraduate level and ready to study at a competitive university.

For general admissions for any discipline though, I'll agre it's probably a bit rough for non applied math grads. At least it doesn't include any analysis, abstract algebra (groups, rings, fields, etc.), or topology. Which would be murder for non-math majors.

ldjb 7 hours ago 4 replies      
Note that the linked paper is a mathematics exam that applies to the entire faculty (not just CS). In addition to that exam, one must also take a CS-specific exam. It seems that the Summer and Winter exams are rather different:

August 2015: http://www.i.u-tokyo.ac.jp/edu/course/cs/pdf/2016computer-s....

February 2016: http://www.i.u-tokyo.ac.jp/edu/course/cs/pdf/2016computer-w....

Edit: Looking at the exam guide [0], it seems that it's only if you want to sit the exam in the Summer that you must also take the separate maths exam; the Winter CS paper already includes maths questions so there is no separate maths exam.

[0] http://www.i.u-tokyo.ac.jp/edu/course/cs/pdf/H29csguide_e.pd...

demonshalo 7 hours ago 10 replies      
Why is it that I find these questions always posed in a way that is MEANT TO make me NOT understand them? Granted, math is not my topic. However, I am interested and want to learn.

I don't mean to be a dick, but I just simply find it frustrating that the goal of most mathematicians is to pose some question/conjecture in as few sentences as humanly possible. It really grinds my gears how interesting problems are always abstracted behind confusing vertical specific language. Even when reading a Wikipedia article about some cool problem, you get hit in the face with some abstraction that makes most problems seem far more complicated than they actually are.

This is probably not a view shared by many here, but if math problems were to be communicated in more natural ways, far more people would be interested in the sciences.

Just my 2 cents!

Ps. I understand that this is an M.SC. exam. I am talking about math in general, not this pdf!

MariuszGalus 5 hours ago 1 reply      
Question 1. Linear Algebra Level

Question 2. Calculus II Level

Question 3. Probability & Statistics Level

You learn how to solve all of this in Undergraduate American Engineering courses.

ajeet_dhaliwal 5 hours ago 0 replies      
I believe I could have done all of these questions 12-16 years ago. Shockingly I can't do any of them anymore without refreshing myself for what they are even asking in some cases.
tasey 6 hours ago 2 replies      
Wow, people saying that it is "really simple". I feel dumb.
m23khan 4 hours ago 0 replies      
Assuming it is directed towards admission into a Graduate Mathematics / Engineering Program, it looks pretty average to me.

For a student who is in-form (e.g. a fresh graduate or a mature student who has taken 2-3 advanced calculus and algebra+stats courses as refresher), they should be able to perform.

Daishiman 1 hour ago 0 replies      
For a grad school entrance exam this isn't too difficult. It's first and second-year maths stuff for a decent CS degree that doesn't skip out on the basics.
rotskoff 4 hours ago 1 reply      
No algebra at all---seems very odd because much of modern mathematics relies heavily on group and field theory.
hokkos 3 hours ago 2 replies      
What exam is it relative to that : https://en.wikipedia.org/wiki/Education_in_Japan#School_grad... ?What are the age of the students ?
0xfaded 4 hours ago 1 reply      
I'm a CS grad who is fairly strong with maths but never really did learn to write a proper proof. All of these questions I would have been able to "solve", but the proofs would have been very casual. What would be a valid answer it 2.3 where it basically asks you to integrate an expression of F after substituting y'=dx/dy?
kuon 5 hours ago 1 reply      
I would have been able to do this with no effort 20 years ago, but nowadays I would be totally incapable of solving even one of the problem. Damn I feel the years.
x1798DE 7 hours ago 3 replies      
Interesting that this is entirely in English. Is this the English language version for foreign students, or are Japanese masters students expected to be fluent in English?
dysoco 3 hours ago 2 replies      
Does anyone have links to Undergraduate entrance exams from other Universities?

I am currently taking one in my country and would like to compare the level with mine.

ajarmst 6 hours ago 1 reply      
Hah! I gave 3.1 - 3.4 as an assignment to second-year CompSci students literally three weeks ago. Although my version was to determine the values experimentally for n<=8 bits and then estimate values for n up to 64. Bonus marks awarded for a mathematical proof of the pattern (which isn't really all that bad if you apply probability to patterns of two bits).
zulrah 6 hours ago 1 reply      
And I was angry at my uni,that my cs curriculum contained almost no maths. After reading this paper I realized that I am glad I didn't have to study this. Some calculus would have been useful though, as I am in a machine learning field now
_-__--- 5 hours ago 2 replies      
I get stuck after the first few subproblems on all three. Does anyone have a PDF of the solutions? Are the solutions ever released?

I'm a little rusty on my linear algebra.

wayn3 6 hours ago 3 replies      
This is really really simple btw, for a grad school entrance exam. Anyone who has successfully studied mathematics for a year should be able to score 100% on this exam.
tokyoooo 5 hours ago 0 replies      
In problem 2, the Euler Lagrange equation imply that we are looking for the function y(x) such as the surface is minimal, so the solution is a cylinder, ie y(x)=constant=c=2.
youareanelitist 2 hours ago 0 replies      
The elitist cunts that are in this chat are the reason why science will never matter to the world.

But hey, at least you are better than everyone else right?

The Internet Is Saving Culture, Not Killing It nytimes.com
79 points by ayanai  6 hours ago   70 comments top 11
fluxic 3 hours ago 5 replies      
Call me a pessimist, but as a content creator this piece came off as extremely out of touch. Yes: NYT is booming in subscriptions, and Patreon has carved out a nice little niche for supporting podcasters and YouTubers. But it's nowhere near the level of support artists have had for the last hundred years.

Before, if you were a writer, you could write for the neighbourhood paper or magazine and carve out a decent living (see: Hemingway, DFW, etc.) Now, unless you're a staff writer for the NYT or maybe 10 other 1%er sites, there's no way to make a living off subscription sites. They have killed local markets that have supported writers. Book sales are down. Newspaper sales are down. Literary magazine sales are non-existent. Anecdatum: I wrote a series of Medium posts this month with a Patreon link at the bottom: 70k hits, $0 donations. It's hard for me to imagine reaching 70k people with my writing in 1910 and still being a pauper.

Two things: this isn't necessarily all doom and gloom. There are probably more artists living than at any point in history. And the internet helps us reach more people than ever before.

However, bragging that the world's most popular newspaper is adding subscriptions and that another couple of creators are making $10,000 on Patreon does not a Renaissance make. Patreon's PR team is trying to spin something thatat least for writersisn't happening. The needle is not moving.

padobson 5 hours ago 6 replies      
You only have to spend a little time consuming digital media to understand that products that rely on advertisements for income suck. As soon as subscription services started offering ad-free versions of their product, I started buying them.

I'm not saying it's the best way forward, but would it kill Twitter and Facebook and Snap to experiment with some of these models? It's working for content creators, why should social media platforms be any different?

At the very least, it makes sense to offer paid tiers of access for their APIs, which I get some of their "enterprise" clients are already doing, but certainly a great API is monetizable and worth paying for.

throwawaycopy 3 hours ago 1 reply      
Ever-lower barriers for creators?

There have never been any barriers for creators. Folk art has always been a part of human societies.

There were barriers for mass dissemination of artistic works, but not the process of creation.

Mass media put the local artists out of business. Before the advent of radio and recorded music there were many more musicians and bands. Now everyone has to compete on a global level.

I would argue that in this new global digital attention economy that there is even more competition and even less room for local and alternative voices.

Live music profits are at an all time high but it is all concentrated at the top. The long tail is a fallacy.

tannhaeuser 3 hours ago 1 reply      
I'm not even sure we'll be able to read content created in this epoch at all two hundred years from now. While the NY Times article is mostly HTML (with lots and lots of script injected, but still), sites such as medium.com and many others created in this decade are just a single big JavaScript blob with auto-generated CSS.
relics443 5 hours ago 2 replies      
I feel like 200 years from now, our times will be viewed as something akin to the industrial revolution. Recency bias makes us feel as if we're more important in history, but 200 years is a long time. I'm sure there'll be another "revolution" or two in that time. And the people of that time will probably consider it to be the most pivotal revolution yet.

History folks. It repeats itself.

nickgrosvenor 30 minutes ago 0 replies      
I made music as a hobby for a decade and without the internet no one would have heard any of my music. With the internet, my songs have been played hundreds of thousands of times.

So I'd agree with this. The internet is responsible for my bands success.

garysieling 2 hours ago 0 replies      
At the beginning they talk about places that have lost income, but the internet has also preserved a lot of content that was only accessible if you visited a place that owned a copy (library, museum, etc).

For instance, UCLA has a great old lecture series from the '60s they've been digitizing:https://www.findlectures.com/?p=1&collection1=UCLA%20Archive...

There are a ton of other places doing this. Once these types of projects are funded once, they can be available pretty much forever, because they are drop in the bucket for youtube, which seems tremendously cost effective.

eduren 4 hours ago 2 replies      
The way I've come to view culture is that it is a byproduct of communication. Stick three people in a room and have them talk to each other, eventually they come to share a culture (a small one, but nonetheless significant to them)[1]. If it's indeed a byproduct, it follows that the amount of culture generated is directly proportional to the bandwidth/speed of that communication.

In that interpretation, the internet can only be an accelerating force; driving society forward into greater levels of social consciousness.

[1]: A relevant XKCD comes to mind https://xkcd.com/915/

M_Grey 4 hours ago 1 reply      
"Wenn ich Kultur hre ... entsichere ich meine Browning!"

Terrible play, questionable subject matter and worse fans, but still... a good point. There's nothing wrong with culture as a concept to live through, but as a point of discussion it's akin to a discussion of religion or dreams.

ideonexus 4 hours ago 2 replies      
tldr; We've all become patrons of the arts through micro-transactions.

Ten years ago, I donated regularly to charities that would enable micro-funding for people living in third-world countries. A $10 donation would buy some chickens for a woman in South America, who could turn a profit on the eggs and eventually pay the loan back. I thought it was a fantastic idea.

Today, I find I'm funding everyone's projects. I pay a $2/month for a serialized zombie radio drama I listen to on my morning run. $4/month contributes to one of my favorite weekly science podcasts. $4/month to Amazon gets me unlimited streaming music on my Alexa. I think nothing of paying $3 for a phone app I might play once or twice before getting distracted by the next shining new thing.

Culturally, even without this micro-funding, we are swamped in culture. My DeviantArt feed overwhelms me with more amazing art than I have time to appreciate. The same is true of books, shows and films both professionally-made and independent. As a result we are drowning in culture, but dividing into micro-cultures as individual. Twenty years ago, everyone had the common cultural experiences of watching the same shows and movies. Today, we no longer can take those common-cultural references for granted. I'm not saying that's a bad thing, just an observation.

muninn_ 5 hours ago 1 reply      
I largely agree. The only thing that will remain constant is that the news today is absolutely terrible since it's trying to become entertainment and not what it should be: news.
Insert D-Wave Post Here scottaaronson.com
89 points by seycombi  6 hours ago   24 comments top 8
doctorpangloss 2 hours ago 3 replies      
D-Wave is the original Theranos.

- They make bold claims that many people plainly say don't make sense, before seeing any evidence.

- They use hype to buy time to do scientific research.

- The successes of the D-Wave machine have the same characteristic p-hacking profile of Theranos's Edison machine. That is, try 200 problems, and find 1 that appears to work, which could just be due to chance.

- Apologists talk about "there are a lot of smart people working" there or blame "marketing," but who haven't seen the product and don't know any more than anyone else.

The main differences are (1) the lack of an enigmatic founder. (2) D-Wave publishes, and then in later analysis their claims don't hold up.

grabcocque 3 hours ago 5 replies      
One thing I'm never quite sure of in these discussions as to what extent D-Wave are a bunch of shysters and to what extent they're actually honest believers that they've built a quantum computer and that their repeated savagings are nothing more than the reasonable expectations of sceptical science being done.

Basically do they really believe they've built a quantum computer or do they actually know they haven't and have decided to pretend?

deepnotderp 1 hour ago 0 replies      
One of D-wave's founders is now trying to patent teleoperation of robots and pass it off as "novel": http://spectrum.ieee.org/automaton/robotics/artificial-intel...

Also, tossing in some machine learning which I presume they're also trying to patent (because it's not as if UC Berkeley and Google have done that 15 times a year since 2010)....

All this makes me think they're not legit.

mdekkers 3 hours ago 0 replies      
...that went over my head depressingly quickly....
hcs 3 hours ago 1 reply      
Perhaps better to link to the blog post than the blog index:http://www.scottaaronson.com/blog/?p=3192
mtgx 4 hours ago 0 replies      
tl;dr Still no evidence that D-Wave's new "thousands of times faster" computer is any faster than a classical computer, even for annealing tasks. And we'll see if we've reached quantum supremacy when/if Google and IBM reveal their 50-qubit quantum computers in the next few years.
monochromatic 3 hours ago 1 reply      
Sad to see a blogger I otherwise like spouting the "Trump is a Russian puppet" propaganda.
sigmaprimus 2 hours ago 0 replies      
This sure sounds like sour grapes to me, arguing that a faster, or true quantum computer will be built in a few years that will out preform Dwave's current system is pretty weak sauce. After reading this article I can't help but see the similarities between this article and the recent attack posts against uber that seem to be the flavor of the day. DWave has built and sold several functioning systems and this is what most likely puts a bullseye on them for people to take cheap shots..just like the Uber haters. Regardless of whether or not DWave has built a true quantum computer, or if the next conventional computer will be faster, they are doing it now, they have found the more effective algorithms and they are pushing the envelope. It's always easier to criticize and whine than stick your neck out and try something new and innovative and articles like this one show that even if you do succeed in those endeavors the haters are going to hate. The one part of the article that I did appreciate is the mention of Geordie Rose who was one of DWave's founders and the fact that an ex wrestler founded such a cutting edge company may speak volumes to the need for strong willed people in unicorn type companies to fend off then nay sayers before they even get off the ground.
       cached 16 March 2017 19:02:02 GMT