hacker news with inline top comments    .. more ..    11 Dec 2014 News
home   ask   best   4 years ago   
1
Google News to shut down in Spain
165 points by apsec112  2 hours ago   81 comments top 18
1
rsync 2 hours ago 7 replies      
Things are going very badly for a global internet if we are discussing a web site "shutting down in spain".

Of course we all know what that means and it seems very sensible in 2014, but remember - if someone had told you in 1998 that a certain website would not be operating in country X, you would have laughed and explained to them (like a child) that the Internet was a single global network and that if one had Internet access at all they would have access to the site in question.

All of that simplicity and innocence has slipped away.

2
gasull 3 minutes ago 0 replies      
Even worse, this might affect Facebook, Twitter and Spanish Reddit-like site Mename (very popular there):

http://boingboing.net/2014/07/28/spain-pushes-for-google-tax...

Just like we have the term 'patent troll', maybe we need the term 'tax troll' for some Governments.

3
jonathansizz 2 hours ago 3 replies      
A pertinent quote from the Guardian article on this story:

Germany passed a similar law to Spains and Google removed newspapers from Google News in response but in October publishers reached an agreement with the company after traffic to their websites plummeted.

4
onetimeusename 1 hour ago 1 reply      
I am reading through the official law and essentially it is saying that the Spanish government finds this necessary to reinforce intellectual property protections, the ip here being the news/stories. The thing that isn't clear from the law was whether anyone had actually complained about what Google was doing or whether Google was actually found violating any ip laws in place(it doesn't seem so). The whole thing is 40 pages so I probably won't read it all. Can anyone clarify if there had been some sort of issue here?

edit: Something that sticks out is that the law dictates how any agreement involving ip is to be done even if previous agreements are in place in order to cover costs "equitably". Yet I can't see how Google isn't already beneficial. The wording suggests Google would be causing damages since damages can be included in payments under this law.

5
franciscop 2 hours ago 1 reply      
Things are getting pretty freaky around here in Spain. We've got some horrible (freedom-wise) laws passed recently and we're all angry about them. They've been trying to for several years, and now that people is tired of fighting these stupid laws back they can pass them. This basically means that Google would need to pay to index newspapers.
6
AznHisoka 2 hours ago 2 replies      
Why can't the just display the headline and no snippet from the article text? Or does that count as a snippet? But wait... what about the actual organic search results? Do they have to remove Spanish news articles from that too?
7
youssifa 40 minutes ago 0 replies      
May not necessarily be a popular opinion on here, but I'll state it anyway:

Google may not directly profit from Google News, but they still manage to extract value from it. We exist in a top-heavy paradigm where giant servers profiteer off the work of everybody else, capturing a disproportionate percentage of the total value created.

It's unfortunate but not unexpected that Google's response is this snarky blog post. But I wish people wouldn't pretend this is somehow a giant government tipping the scales against "openness".

I see this more as an institution in charge of making sure our collective greed not getting the better of us trying to distribute wealth to those who create it proportionate to the value being created.

Is it a futile attempt, likely unaware of its own vision? Sure. I just wish Silicon Valley would get its head out of the sand and realize that the current paradigm isn't necessarily sustainable for anyone -- whether you're the one sitting atop Mt. Server enjoying crazy network effects or the person contributing value for peanuts (if you're lucky).

8
jpatokal 2 hours ago 0 replies      
TL;DR: This new legislation requires every Spanish publication to charge services like Google News for showing even the smallest snippet from their publications, whether they want to or not. As Google News itself makes no money (we do not show any advertising on the site) this new approach is simply not sustainable.
9
etanol 1 hour ago 0 replies      
This is one example of what happens when the population gives absolute power to a single political party: lobbying paradise. We spaniards have the politicians we deserve.
10
sebicas 2 hours ago 0 replies      
I am wondering if they will also remove newspapers from search results. Indexed search results of newspapers may also contain "News snippets"
11
logicalman 1 hour ago 1 reply      
Can they allow users in Spain to access Google News from a non-Spanish TLD but still serve those results in the Spanish language?
12
sounds 1 hour ago 1 reply      
When you say "A lot of Europeans," can you cite a source, like a poll or even an internet forum, where like-minded Europeans express this view?

I'm not challenging your assertion, I'd just like to see it in action and measure it for myself.

Edit: in case you edit or delete your post, this is what I'm asking about:

  Does still anyone believe that 9/11 was a terrorist  attack. A lot of Europeans think it was the US  government itself.

13
FileNimbus 1 hour ago 0 replies      
CatchCrisis.com
14
caiob 2 hours ago 3 replies      
Since when Google makes money by putting ads on their sites?
15
abennobashi 1 hour ago 1 reply      
Good for Google. Greedy, thieving socialist governments have no place in the free world. This new Spanish law stinks of "Anti Dog eat Dog" legislation. Downvote me to hell, but Who is John Galt?
16
gcb0 2 hours ago 3 replies      
"we don't make money because we do not advertise on that site" is a huge fallacy.

This sounds like a bluff because if that gets traction in the US then their search business will collapse. Imagine $0.01 for every search that shows a snipet of wikipedia to the wiki foundation?

17
notjackma 1 hour ago 2 replies      
"As Google News itself makes no money (we do not show any advertising on the site) this new approach is simply not sustainable. "

Rubbish!

Google's Human Resources department doesn't make any money either, yet it exists, along with other loss-making divisions.

Google as a whole is a very profitable company, and it could easily pay publishers over the long run. It just doesn't want to.

18
superuser2 23 minutes ago 7 replies      
EDIT: Google does not actually publish article text anymore. I'm pretty sure it used to. But given its current operations, yeah, this law doesn't make any sense. I withdraw my comment, but will leave it so people can see what the replies are about.

Journalism is a profession. Professionals are not free. We used to pay journalists by making newspapers the primary ad-delivery platforms (i.e. what Google is now). That's gone.

HN likes to pretend that Twitter/the blogosphere/"citizen journalists" will adequately replace professional journalism. In practice, this is not true. The only places doing high-quality, long-form local reporting are the city papers. Even they can barely afford to do it anymore. So they are looking for ways to stay afloat.

Buzzfeed is viable because "news" actually drives enough ad impressions to cover its costs. Real news doesn't. That pretty much leaves paywalls. You can say, "I absolutely refuse to pay for text because information wants to be free," but then you would no longer be able to be informed about what goes on in your city.

In theory. Google News republishes the full text of basically every article of basically every newspaper. If your hometown paper raises a paywall, you could continue to enjoy your newspaper's output right up until the moment it closes its doors. Google News needs to be stopped because it bypasses the only mechanism newspapers have left to be able to pay for real journalism. If you don't give them that option, then the only "news" left will be Buzzfeed-alikes or paid for by corporate/political interests instead of customers who (claim to) want professional reporting.

Fragmentation of the internet is bad, yes, but what should have happened is a global shutdown of Google News.

2
Donate to the Internet Archive
104 points by jakeogh  2 hours ago   20 comments top 10
1
chroma 33 minutes ago 2 replies      
I've donated to the Internet Archive and I'm a big fan of Jason Scott, but the Internet Archive is not an archive. Any site on it can go down without warning, thanks to the fact that they apply current robots.txt rules to past archives. Once a domain squatter or regretful admin forbids archivebot (or crawlers in general), archive.org's copy goes down.

This has ruined many supposedly permanent links. The infamous "She's a Flight Risk" blog from a decade ago is down.[1] My first website is missing. Even public domain stuff like NASA's report on nuclear propulsion is gone.[2]

With just a small rule change (obey robots.txt at the time of crawling), they could eliminate the risk of a page disappearing. Instead, we're stuck with a slower version of the link rot we're used to. It doesn't stop me from supporting them, but it's incredibly frustrating.

1. https://web.archive.org/web/*/http://www.aflightrisk.blogspo...

2. http://web.archive.org/web/20121029225832/http://ntrs.nasa.g...

2
striking 1 hour ago 1 reply      
They need a whole $75/person instant as compared to Wikipedia's $3/person instant and they're not resorting to shouty, loud boxes that open modals on mobile platforms.

I love you, Internet Archive.

3
benbreen 1 hour ago 0 replies      
Archive.org has been an absolute godsend for historians and others who use rare books. Case in point: when I was doing PhD research in Lisbon three years ago, I had to search several rare book shops and ended up paying 80 euros for a very rare 19th century Portuguese book I needed for my research. Here it is on archive.org in multiple editions, all text-searchable: https://archive.org/search.php?query=duarte%20ribeiro%20de%2...
4
benjaminRRR 6 minutes ago 0 replies      
This is something I am happy to donate for. These guys have been plugging away quietly for a long time to give us a record that would otherwise literally disappear into thin air.
5
allworknoplay 38 minutes ago 0 replies      
I hate to sounds bitter, but I've been a donator for ~8 years, and the one time I asked for help (really: filed a bug report re: an archive they said they had but which didn't acually resolve & which I needed access to) they did not get back to me, ever, despite numerous support requests and additional donations.

I'm not sure whether that means they need more funding or whether they're simply unresponsive, but it definitely didn't help me solve my problems with a trademark troll.

6
dmethvin 1 hour ago 1 reply      
Just in case you'd like to look at their financials before donating, they are at faqs.org [1]. I couldn't find the official IRS Form 990 online.

[1] http://www.faqs.org/tax-exempt/CA/Internet-Archive.html#anal...

7
pkaye 51 minutes ago 0 replies      
I just donated... I use them all the time to look up old computer magazines from the 90s.
8
jws 1 hour ago 0 replies      
Sounds reasonable. I did just have to use them as "backup of last resort" to recover a web site. A little scraping turned "sorry, its gone", into "web site is back online".
9
smeyer 2 hours ago 1 reply      
Here's a link to the donation page (rather than to the main page, which happens to have a donation banner now): https://archive.org/donate/index.php .
10
equivocates 1 hour ago 1 reply      
Are you kidding me, I'm trying to take down my internet history, not keep it around.
3
MDBM High-speed database
56 points by threepointone  2 hours ago   9 comments top 4
1
justin66 1 hour ago 1 reply      
This looks interesting. At this stage of the game a more meaningful benchmark might involve LMDB, Wiredtiger, and, yes, LevelDB.
2
coreymgilmore 41 minutes ago 1 reply      
Thoughts on using this as a cache instead of memcache or redis? Yes, it does not have nearly as many features or functions but when raw performance is needed I could see this working (given an api for using this via Node.JS, PHP, etc.).
3
philliphaydon 1 hour ago 2 replies      
Do people get annoyed by all the JavaScript frameworks and Databases coming out in regards to adoption from a company point of view? I mean every other day a new database comes out and claims to be better in one way or another than something else and then its like "fuck I picked X when now there's Y"

It seems over the last year technology has been growing more rapidly than any other period.

Fun times but so hard to keep track of everything!

4
extralam 1 hour ago 0 replies      
interesting. follow
4
Yahoo and Mozilla Form Strategic Partnership
16 points by xkiwi  46 minutes ago   1 comment top
1
pvnick 28 minutes ago 0 replies      
Considering both Mozilla and Yahoo [1] value privacy rights so much this can only be a good thing.
5
Firefox.html: Rebuilding Firefox UI in HTML
459 points by ash  10 hours ago   98 comments top 22
1
paulrouget 9 hours ago 5 replies      
I've built that. Just to be clear: this is a personal project, not a mozilla project (even though I'm a mozilla employee).

Also - many people find it silly to use HTML instead of the operating system toolkit library. But it's not HTML or native. It can be both. In this case, the HTML code define the layout, and we can draw native widgets inside (look at the <input type=submit> tag in HTML, it's a native widget). For example, if you run the current build on Mac, you'll see that new tabs use Yosemite "vibrancy" effect. Native look and native performance, can, in theory, be achieved.

2
icefox 9 hours ago 6 replies      
The browser shipped with Blackberry 10 was written in HTML and was a real joy to develop.

Edit: some more public details

The default browser on the BlackBerry 10 platform was a completely new browser application. The chrome was written in HTML, CSS and JavaScript. Being able to develop the chrome on your desktop browser or being able to run inspector remotely and using your desktop was very handy. The core was a command line application called webplatform that "launched" a url that was a "webapp". The webapp had API's exposed to it such as creating WebView's in or out of process (yup blackberry has had multi-process tabs for a while now...). One joy was being able to pull up the Javascript console for the browser WebView and dynamically calling exposed c++ API's in any WebView in any process to test out features or diagnose problems.

It started out as a quick little proof of concept I tossed together over a day and the upside was large enough to invest time into. One of the reasons for making the main browser in html was that as a platform we wanted web application to succeed. Eating our own dogfood we made sure webapps could handle the job. The API's that were needed were there, memory usage were low, startup time was fast etc. And if you search for reviews of the blackberry 10 browser you will find that the end result was a success.

Edit 2: Much more in depth information can be found on this video which was a presentation given at Blackberry Jam by several of my colleagues. Skip to the 23 minute mark to see some actual code of what a webapp browser would look like.

https://www.youtube.com/watch?v=bZ8vxhTezvs

3
kibwen 9 hours ago 0 replies      
I'm very excited at the potential of the browser chrome being implemented entirely via standard web technologies. And given that Servo is never going to implement XUL, it would save a lot of effort that would otherwise be spent implementing a bespoke, minimally-functional UI (which has been tentatively named Crow, if the MST3K reference wasn't already obvious).

See also this other thread linked from the discussion in the OP, "Moratorium on new XUL features": https://lists.mozilla.org/pipermail/dev-platform/2014-Octobe...

4
bkeroack 9 hours ago 8 replies      
I realize this is a personal project, but this is deeply amusing considering that:

1. Way back when (circa 1998-2001?), the Mozilla project started as a radical redesign of the next gen Netscape browser. One of the core principles of the architecture was that the browser itself would render the UI elements (using an HTML-like tech call XUL).

2. The Firefox browser (then called "Phoenix") was a reaction against the above, which was thought to make the browser too heavyweight and slow. Originally Phoenix was the Gecko engine in a native UI window without all the XUL overhead (and without all the other components of the Mozilla Suite like the email client).

Now we're seeing the reverse trend, 15 odd years later.

EDIT: It turns out I misremembered about Phoenix dropping XUL completely--rather they dropped XPFE for a "new light-weight XUL toolkit", along with dropping all the non-browser components of the Mozilla suite.

5
taf2 7 hours ago 2 replies      
This is great. My experience with XML for UI went something like this.

1. build an XML language for abstracting C++ UI MCF or gtk

2. realizing the need for HTML content - embed mozilla/xulrunner

3. realzing xulrunner /mozilla already provide a cross platform toolkit use XUL to build the user interface with xpcomm wrappers

4. realize that HTML is better for user interface, and only write HTML/javascript with xpcomm wrappers.

5. switch to web development completely and avoid desktop apps all together :)

6. from time to time check in on the state of desktop/app development and see if they've finally figured out html is better for interfaces than any interface builder

6
Morgawr 9 hours ago 6 replies      
Color me surprised, but when we get to the point where we run HTML to build a browser that is supposed to be the tool used to render and display such HTML... haven't we gone too far? Is a browser necessary? What is rendering that HTML if not a browser?

I mean, I get it, a browser is used to browse (duh) the internet, not necessarily to render HTML, but at this point we really need to ask ourselves "why are we doing this, again?".

EDIT: Still impressive though, nice proof of concept!

EDIT 2: As much as I hate the "Why am I getting downvoted?" shenaningans that people usually pull, I want to clarify the intent behind this post. I'm not flaming or hating on the project, it's a cool idea. I was trying to spark some insightful conversation on "where do we want to go from here?" and "do we really need a browser if we have gone this far?". If you want to downvote me, why not just reply to this post and have a nice interesting conversation instead?

7
matthewmacleod 7 hours ago 0 replies      
This is a fantastic idea and I'm really glad to see experimentation in this area. It's already great to look at how things like Webkit's dev tools are implemented in HTML and see how that idea might be extended.

At it's core, a browser is an HTML/JS rendering engine with some chrome to allow users to manage what pages they're looking at and in most cases, that chrome is actually pretty minimal. It seems like a natural evolution to play with the idea of implementing that chrome in the natural UI language of the browser engine too. Yeah, the tools aren't there yet but experiments like this will give us some scope to play around with what might be possible and identify the pain points that must be overcome to make it a reality.

Great stuff.

8
nacs 8 hours ago 0 replies      
Reminds me of Breach[1] / Thrust[2] which basically lets you do the UI plus some low level stuff via Node/JS.

There is a 'browser in a gist' [3] using Breach which is a good example of its use.

[1]: http://breach.cc/

[2]: https://github.com/breach/thrust I believe this is the new core of Breach by the same people)

[3]: https://gist.github.com/morganrallen/f07f59802884bcdcad4a

9
browserxul 6 hours ago 0 replies      
Copy this into firefox's URL bar for a fun treat:

    chrome://browser/content/browser.xul

10
agumonkey 9 hours ago 1 reply      
I'm always pleased when I use chrome devtools on chrome devtools. So I can't welcome this enough. Have fun.
11
simonw 7 hours ago 1 reply      
Would it be possible to implement XUL using Web Components and port the existing XUL interface to HTML that way?
12
misuba 9 hours ago 1 reply      
Is there a connection between this and the old Chromeless project? (It doesn't say there is, so I suppose my real question is, why not?
13
espadrine 8 hours ago 1 reply      
Can HTMLRunner be used as the basis of something like node-webkit[1]? What's its strength / weaknesses compared to that?

[1]: https://github.com/rogerwang/node-webkit

14
agumonkey 6 hours ago 0 replies      
15
grumblestumble 8 hours ago 0 replies      
I'm all for it if it means we finally get ::-webkit-scrollbar equivalency in Firefox...
16
hawski 8 hours ago 0 replies      
This sounds like a good idea. But change should happen gradually - no to parallel versions.Like author of zeroconf, when he was rewritting in in OCaml: http://roscidus.com/blog/blog/2014/06/06/python-to-ocaml-ret...
17
phkahler 8 hours ago 0 replies      
What widgets are used? (I'm not a web dev and only dabbled in that over 15 years ago) If you use native widgets, then you end up with a cross-platform app framework. If you create your own widgets, then you end up with a cross-platform GUI toolkit and app framework. Which is it? Either way, this seems quite interesting. OTOH, can you do apps this way with anything other than js?
18
1ris 8 hours ago 1 reply      
Is this the first step to switch to servo?
19
shmerl 8 hours ago 1 reply      
Interesting. But on Android Mozilla already deviated form this approach of "webbiness" in favor of using native UI. Same as Sailfish browser does with Qt and Gecko through IPCembedlite.

If not for that, Sailfish browser could reuse the UI.

20
hencq 9 hours ago 1 reply      
Way back when wasn't the original plan for Seamonkey to use Gecko to render the UI as well? Maybe they were too far ahead of their time back then.
21
Edmond 6 hours ago 0 replies      
love the idea, web tools built on web technology.
22
mcao 7 hours ago 0 replies      
The one thing that absolutely annoys me about the current Firefox is the location of the refresh button, which is a tiny icon stuck in the far right of the location bar. Worse of all you can't customize it natively. You have to download extensions to change the UI and still you won't get it to look 100% how you want it. Being able to modify a HTML based UI would be awesome.
6
Wrapping web APIs made easy
29 points by BaiCrazy  58 minutes ago   discuss
7
Magnus Carlsen I am chaotic and lazy (2010)
126 points by radovanb  10 hours ago   34 comments top 8
1
tbrake 4 hours ago 1 reply      
He briefly talks about Kasparov here as they were training together at the time. They later broke up.

I wonder if some of the conflict they felt was was really Magnus's more organic, natural approach clashing with Kasparov's more like systematic and rigid type of training, which he no doubt inherited from Botvinnik. This isn't to say Magnus doesn't work hard at his game but on a chess approach scale of, say, Capablanca to Botvinnik, he would very much tilt towards the former.

edit: Actually he specifically talks about that; somehow I skipped it upon first reading.

2
kizer 5 hours ago 1 reply      
I think there's a great analogy for why he cares so little about his intelligence: do you think Michael Jordan sat around admiring his build, his height, etc? In a competitive environment, it's best to accept that some of your abilities are fixed, so you can focus on training what can be bettered.
3
eddotman 5 hours ago 2 replies      
The brevity and bluntness in his responses is pretty amusing. I'm surprised he so candidly rates his own abilities and the abilities of his peers - I feel like many people would dodge those questions in public interviews.
4
dont_be_mean 5 hours ago 1 reply      
Sometimes you wonder.

I was not particularly interested; I was bad and soon stopped again.

I dont know why I learnt all the countries of the world off by heart, including their capitals and populations. Chess was probably just another pastime.

Im not a disciplined thinker. Organisation is not my thing; I am chaotic and tend to be lazy. My trainer recognised that and as a rule allowed me to practise whatever I felt like at the time.

When I was 13, my parents took me out of school for a year. They travelled around the world with me and my sisters, and on the way they taught us. That was fantastic, much more effective than sitting in school.

5
mhomde 4 hours ago 5 replies      
I think there might be a correlation between being lazy and being creative. I've seen this pattern in many other greats (and in my not so great self :)

Aaron Sorkin talks about procrastinating a lot between writing sessions. Lots of painters and artists procrastinate as well.

I think its a matter of digestion, your mind is focused on a "task" and churning in the background, but you don't actively work on it except when you "feel like it". I saw some scientific article to this affect that the downtime is actually very valuable for the brain to form creative thought.

6
jonny_eh 5 hours ago 1 reply      
I'd love to see Carlsen play a modern game like Hearthstone or Magic. Even just to hear his thoughts on them.
7
sayemm 5 hours ago 1 reply      
Interesting how he says that extremely high IQ may even be a disadvantage:

And thats precisely what would be terrible. Of course it is important for a chess player to be able to concentrate well, but being too intelligent can also be a burden. It can get in your way. I am convinced that the reason the Englishman John Nunn never became world champion is that he is too clever for that. At the age of 15, Nunn started studying mathematics in Oxford; he was the youngest student in the last 500 years, and at 23 he did a PhD in algebraic topology. He has so incredibly much in his head. Simply too much. His enormous powers of understanding and his constant thirst for knowledge distracted him from chess... Right. I am a totally normal guy. My father is considerably more intelligent than I am.

And that his big shtick is his focus, intuition, and domain expertise - not his IQ:

No. In terms of our playing skills we are not that far apart. There are many things I am better at than he is. And vice versa. Kasparov can calculate more alternatives, whereas my intuition is better. I immediately know how to rate a situation and what plan is necessary. I am clearly superior to him in that respect.

8
lukeholder 3 hours ago 0 replies      
My favourite quote: "I listen to music on the Internet, but don't download any songs. It's all totally legal. Many people may find that boring, but I think it is important."
8
Category Theory by Tom LaGatta
47 points by vinchuco  5 hours ago   2 comments top 2
1
josv 1 hour ago 0 replies      
This is so good. I'm not sure how idiosyncratic my reaction is, but this kind of fluid, beer in one hand (and "go look up the axioms on Wikipedia") mode of presentation is such an amazing way to introduce and motivate a topic. Just loved this.
2
Hario 2 hours ago 0 replies      
I know Tom, and I can personally attest to his smarts, passion, and personal warmth. Really excited and proud to see this on the frontpage.
9
Rosetta finds comet's water vapour to be significantly different from Earth's
162 points by twowo  9 hours ago   37 comments top 6
1
tokenadult 8 hours ago 1 reply      
The two comments submitted previously as I type this explain why the article title on HN is as it is. I was puzzled by what the title MEANT, so I looked into the article, and what the article says farther down is "Previous measurements of the deuterium/hydrogen (D/H) ratio in other comets have shown a wide range of values. Of the 11 comets for which measurements have been made, it is only the Jupiter-family Comet 103P/Hartley 2 that was found to match the composition of Earths water, in observations made by ESAs Herschel mission in 2011." When I first read the headline, I wasn't sure if the claim was that the water vapor had mixtures of other chemical molecules in it, or what.

Okay, a different isotope ratio in water from a comet as contrasted with water generally found on earth would indeed be a clue to how water might have traveled from one orbiting body to another early in the development of the solar system. This kind of isotype checking (for isotopes of other elements) is one of the things done to confirm that rocks found on earth are presumptively from other parts of the solar system.

2
acqq 6 hours ago 0 replies      
It doesn't appear to be a result for which the landing was necessary? Also it seems it was something already achieved before ("measuring D/H"). The text mentions that the D/H ratio was already measured in 11 comets, I believe by just flying through their "tail" (1)?

1) http://adsabs.harvard.edu/full/1987A%26A...187..435E

How was it done this time?

3
AYBABTME 4 hours ago 2 replies      
They mention that water already on earth would have boiled off in its youth. I don't understand this, wouldn't the vapor stay around the planet, then return to liquid when the planet cooled?

Or else, what would have attracted the water away from the earth's atmosphere? And if so, why wouldn't it have attracted away the azote and oxygen and all of the atmosphere at the same time? Or was there not yet an atmosphere?

4
grecy 7 hours ago 2 replies      
I didn't realize the standing theory is that Earth's water was delivered by lots of meteor strikes.

Has there ever been an estimate made of the number of strikes needed to deliver the volume of water currently on Earth? I have to think it's enormous.

5
smeyer 9 hours ago 1 reply      
(For future readers, see dang's comment on how he changed the title from "Rosetta first results: ocean water not from comets")[1])

I don't know why you changed the title to something claiming a stronger result than their title: "ROSETTA FUELS DEBATE ON ORIGIN OF EARTHS OCEANS". I'm not a planetary guy, but I think while the evidence is moving that way, the claim in your title is definitely premature.

Here's a distillation of their results (from their post) that seems far more in line with their claims than your title:

>Our finding also rules out the idea that Jupiter-family comets contain solely Earth ocean-like water, and adds weight to models that place more emphasis on asteroids as the main delivery mechanism for Earths oceans.

[1] https://news.ycombinator.com/item?id=8731304

6
dang 9 hours ago 2 replies      
The submitted title was "Rosetta first results: ocean water not from comets". Since the article's own title is not very informative, we changed it to the first sentence, which is. We also changed the url from http://blogs.esa.int/rosetta/2014/12/10/rosetta-fuels-debate..., which points to this.
10
Scientific Peer Review Is Broken Fighting to Fix It with Anonymity
125 points by gkuan  9 hours ago   58 comments top 23
1
dougmccune 7 hours ago 3 replies      
The actual complaint filed shows pretty nasty behavior. Here's the full complaint: https://retractionwatch.files.wordpress.com/2014/10/filed-co...

Someone very clearly was trying to get Dr. Sarkar fired. The PubPeer comments are only one small aspect. But the person then also (I assume anonymously) emailed his new employer to allege fraud, and then went so far as to print out these allegations and stamped them with official looking nonsense about being from the an NIH investigation (which didn't exist), and distributed them throughout Sarkar's department mail boxes.

The complaint makes a decent case for why they think pretty much all the negativity directed at this researcher is likely from one angry person. They obviously can't prove that multiple anonymous comments are from the same person, but reading the content it certainly seems likely. What is assumed to be the same person then took things way past the line of what most of us would consider ethical.

There may very well be a place for anonymous calling out of potential research misconduct. But making fraud allegations anonymously online, then printing out those comments, trying to fake them to look like an official government inquiry, and physically delivering them to the researcher's boss at his place of work isn't the way to do that. This case might be one of those cases where the anonymity should be protected at all costs out of principle, but it's a really shitty case to wave your "we're the good guys" flag for.

2
lisper 8 hours ago 4 replies      
Anonymity can't fix scientific peer review, it can only replace type I errors with type II errors. Instead of suppressing criticism that ought not to be suppressed, anonymity can (and often does) fail to suppress criticism that really ought to be suppressed because it is in fact false and defamatory. And indeed, what this article is really about is a lawsuit that alleges that this kind of error has in fact taken place.

Ironically, the very title of this article is a model of non-scientific thinking that ought to be subject to criticism, but attempts to inoculate itself against criticism by asserting that the subjects of the piece are protagonists "fighting to fix" a broken system. They're not. They're fighting to replace one broken system with a different broken system.

There is no question that scientific peer review is broken and needs to be fixed. But anonymity is not the answer. And holding anonymity up as something that should itself be beyond criticism is certainly not the answer.

3
tokenadult 8 hours ago 0 replies      
The submission here is an interesting article by the founders of PubPeer, which has already been in the news quite a bit for finding examples of shoddy science papers that have had to be withdrawn by journal editors. I learned about PubPeer on the group blog Retraction Watch (RT), and I just bopped over to Retraction Watch after reading the article kindly submitted here. RT reports in detail on the defamation suit against PubPeer that is mentioned in the parent article of this thread.[1] I hope the PubPeer experiment can continue and thrive and promote better scientific research practices.

Some of the other comments here suggest that anonymity of reviewers is dangerous in itself. That's why some researchers promote an open review process. Jelte Wicherts and his co-authors put a set of general suggestions for more open data in science research in an article in Frontiers of Computational Neuroscience (an open-access journal).[2]

"With the emergence of online publishing, opportunities to maximize transparency of scientific research have grown considerably. However, these possibilities are still only marginally used. We argue for the implementation of (1) peer-reviewed peer review, (2) transparent editorial hierarchies, and (3) online data publication. First, peer-reviewed peer review entails a community-wide review system in which reviews are published online and rated by peers. This ensures accountability of reviewers, thereby increasing academic quality of reviews. Second, reviewers who write many highly regarded reviews may move to higher editorial positions. Third, online publication of data ensures the possibility of independent verification of inferential claims in published papers. This counters statistical errors and overly positive reporting of statistical results. We illustrate the benefits of these strategies by discussing an example in which the classical publication system has gone awry, namely controversial IQ research. We argue that this case would have likely been avoided using more transparent publication practices. We argue that the proposed system leads to better reviews, meritocratic editorial hierarchies, and a higher degree of replicability of statistical analyses."

Wicherts has published another article, "Publish (Your Data) or (Let the Data) Perish! Why Not Publish Your Data Too?"[3] on how important it is to make data available to other researchers. Wicherts does a lot of research on this issue to try to reduce the number of dubious publications in his main discipline, the psychology of human intelligence. When I see a new publication of primary research in that discipline, I don't take it seriously at all as a description of the facts of the world until I have read that independent researchers have examined the first author's data and found that they check out. Often the data are unavailable, or were misanalyzed in the first place.

[1] http://retractionwatch.com/2014/12/10/pubpeer-files-motion-d...

[2] Jelte M. Wicherts, Rogier A. Kievit, Marjan Bakker and Denny Borsboom. Letting the daylight in: reviewing the reviewers and other ways to maximize transparency in science. Front. Comput. Neurosci., 03 April 2012 doi: 10.3389/fncom.2012.00020

http://www.frontiersin.org/Computational_Neuroscience/10.338...

[3] Wicherts, J.M. & Bakker, M. (2012). Publish (your data) or (let the data) perish! Why not publish your data too? Intelligence,40, 73-76.

http://wicherts.socsci.uva.nl/Wichertsbakker2012.pdf

4
Fede_V 8 hours ago 3 replies      
I think the increase in fraudulent papers getting published is a symptom, not the disease. The disease is that there are too many scientists doing research for a given research budget.

Due to the ferocious competition for grant money, people are either sloppy and cut corners, or do whatever they need to do and outright cheat to publish in top tier journals.

As a society, we need to make a decision about how much we want to fund scientific research - then, once we've made that decision, insure that we put in place a sustainable system in place - we cannot put in a put of money to finance 100 grants, but then build a pipeline that funnels an ever increasing amount of people into a pool that remains constant.

5
zmanian 7 hours ago 1 reply      
It seems plausible to imagine a system where PeerPub could retain their "published author to register

This model relies on t a blind cryptographic signature.(https://en.wikipedia.org/wiki/Blind_signature)

PeerPub generates a public/private key pair.

1. Alice wishes to register on PeerPub. She generates a Nonce N.

2. She blinds the nonce with a factor B.

3. She submits the blinded nonce and her identity to PeerPub. PeerPub checks her credentials and executes a blind signature of the number and returns it to Alice.

4.Alice now separately registers an account perhaps using a privacy protecting system like Tor. She uses the original N and the unblinded Signature.

5. PeerPub verifies N + signature and registers the account. PeerPub will have no way of linking N to the original credentials. PeerPub can record N and make sure it can only be used once to register an account.

6
jedbrown 7 hours ago 1 reply      
I take the opposite approach: I sign my reviews and have ever since I was a grad student. It compels me to do a better job reviewing and often leads to further discussion and sometimes collaboration with the authors. Some people don't like what I have to say, but by and large, they respect it. For many topics that I review for, the authors will have a pretty good idea that I wrote the review unless I am intentionally vague. I would rather write clearly and directly and stand behind it. It is a professional risk that I don't think anyone should be compelled to take, but I think signing reviews is generally good for science.
7
esbio 5 hours ago 1 reply      
As a person who did research until a few years ago, I must say that the problem is the exact opposite. When you send an article to a journal, the paper gets reviewed by a number of peers, which send their comments back to the Editor on the appropriateness of the claimed work.

The problem with this mechanism is that reviewers have no liability, because their comment is anonymous to the author and won't be available to the readers, as it won't be published as part of the article. The result is that reviewers are not made accountable now or in the future for inaccuracies in their review, blatant attacks, or tactical requests for additional irrelevant investigation just out of spite or to stall you so that they can scoop your paper.

Occasionally, the Editor can step in and disregard a particularly obnoxious reviewer, but it depends on the editor, the journal, and the political/scientific strength of the reviewer.

8
kazinator 7 hours ago 2 replies      
> A prominent cancer scientist, unhappy with the attention his research papers have received on PubPeer, is suing some of our anonymous commenters for defamation

On the other hand, should anonymous commenters have the balance of power: in other words, say whatever they want with impunity, even if it actually is defamatory?

(Not saying that is the case in this situation, but in general).

The problem is that defamation is a legal concept, which can only be tried legally. So for instance, whereas a site can rigorously enforce rules which say that all comments are directed at the research material, and not at persons, and have a factual basis in that material, those measures cannot take away the right of someone, who feels they have been defamed, to take the matter to court (where they will almost certainly lose, which is neither here nor there).

You can't just create a site and declare it above the law, so to speak.

The only way to protect the identities of the anonymous is for the site to take responsibility for the statements it publishes: to assert that the anonymous statements are subject to rigorous standards of review, and when published, they in fact reflect the views of PubPeer, and PubPeer alone, and not of any anonymous persons (who do not publish any statements, but act only as sources of information).

Then if someone feels they have been the target of defamation, the defendant shall be PubPeer.

9
ignostic 4 hours ago 0 replies      
What we need here is a more nuanced approach. Anonymity can solve some problems in research, but it will make other worse.

When people are anonymous, they ARE more likely to be truthful in their criticism. They have less incentive to hold back, and it's just human nature to tone down critical feedback when you're critiquing the work of someone with is either influential or an acquaintance. No one likes to make enemies.

On the other hand, anonymity can pretty clearly bring out some of the worst in us. Some people feel little obligation to be fair or honest when their reputation isn't on the line, and so you see people trying to knock down rivals, people they don't like, or random strangers just for the "thrill of the troll."

Imagine if every time you applied for a job your potential employer had access to anonymous feedback on your past work. Some of it might be fair and honest (whether positive or negative), but some of it might be lies from an anonymous coworker with a grudge. Maybe someone is trying to take you down a rung because you got the promotion over them. You could be penalized for any petty reason, and it would stick with you.

Anonymous feedback communicated publicly is much the same. It holds the reviewer and the object of review on unequal footing. Anonymous feedback would be great for an author or even an editor, but it's just not fair to allow the pettiest of people to attack the works of others while wearing a mask. I'd like a system that helps researchers and others invested in the work to solicit anonymous feedback to make the work better. Public-facing commentary, on the other hand, should be tied to an identity.

10
ejz 8 hours ago 1 reply      
One of the main issues is that this is just a really tedious process. No one wants to go through algebra; that's why it's siphoned off to grad students. Do you really think a tenured professor is going to spend time checking the grunt work on /someone else's/ paper when they won't do it for their own? I'm intrigued by the possibility of using natural language processing and logical system tools like Wolfram Alpha. Wolfram recently posted on his blog about building machines that could store data about complex mathematical objects, and already you can build machines that confirm first order logical statements.

Farming out low-level tasks to automated systems would be interesting. Imagine if the format of papers changed entirely, ie, you had to submit your proofs in certain formats, or at least certain parts in specific ways. I'm sure that many professors would be elated to see the number of papers they have to review go down drastically; although, I'm sure many will be disappointed to get a return letter that says, "I'm sorry, but the low level flaws were so serious that they were automatically rejected and are not fit for review."

11
ylem 7 hours ago 0 replies      
This is an interesting problem. On the one hand, there are merits to a referee being anonymous to the submitter. Part of this may be to avoid reprisal for younger referees, but even for more established referees, you may be freer to comment if you are anonymous to the author. BUT, you are not completely anonymous. Hopefully (though there have been some recent scandals related to this) an editor of the journal knows your work and has chosen to use you as a referee based on that. This can help to keep down some of the noise that another poster (lisper) mentioned. Also, the fact that the editor knows who you are may provide some constraint on how you may phrase a review as compared to if you were completely anonymous.

I don't think this is something one should issue a lawsuit over--but I also don't think that their proposal of completely anonymous review is at all useful.

12
atsaloli 1 hour ago 0 replies      
Dr. Mark Burgess just blogged "Why I stopped caring about peer review, and learned to love the work". http://markburgess.org/blog_peer.html
13
kgarten 3 hours ago 0 replies      
I don't get their stance ... peer-review is already "anonymous". There are also a lot of issues with truly anonymous user forums (see 2chan and 4chan). For me it always seems as if the social aspects of communication disappear when one is truly anonymous, e.g. hate speech. The problem is not lack of anonymity but lack of incentives for reviews (I don't get anything from doing a thorough review of a paper and often it's hard to impossible to judge the contribution without dataset and code). It seems peerpub and similar systems will attract people who have the incentive to attack specific authors (as it happened in this case).
14
LiweiZ 2 hours ago 0 replies      
I don't think push some form of crowd intelligence would change much. I guess the best thing we could do is to create some conditions to let new nodes emerge and complete. I also believe better funding distribution mechanism/system will help a lot.
15
pizza_boy 6 hours ago 0 replies      
With Publons.com (https://publons.com) we have different philosophy: the more transparency we can bring to the review process, the better. At the same time we recognise that both blind and double-blind peer review play an important role in generating quality research.

Our approach is to focus on turning review of all kinds (including both pre- and post-publication) into a measurable research output -- something you can add to your resume. We support both anonymous and signed review with the idea that it will lead to greater transparency in the long run and also motivate reviewers to contribute more.

We have a significant number of both types of review now and are starting to look ways to measure if there are significant differences between blind and open review.

16
pc2g4d 4 hours ago 0 replies      
Much of the discussion here seems to have become "anonymity vs. non-anonymity" --- i.e. either anonymity is good for research or it's bad for research. Why not just accept that in some venues there will be anonymity, and in some not, and let them each develop according to the merits of the respective approaches? No need to have all research anonymous or all research clearly attributed to a public identity.
17
weissadam 5 hours ago 0 replies      
My understanding is that most peer review systems in place at various journals and funding agencies today are already anonymous (except for when people are identified by their well known viewpoints.) If you ask me, the real problem can't be solved with communications technology, the real problem must be solved at the source: The funding agencies need to take the importance of reproduction of results seriously and require their grantees to do a certain amount of rote reproduction work in order to qualify for grants for novel research. Will it slow the pace of things down? Certainly. Will it increase the quality of the science? Certainly.
18
Fuzzwah 8 hours ago 0 replies      
The obvious problem here is with fields that are small and highly specialized, such that anyone knowledgeable enough to comment on a topic are known to be from a small circle of scientists.
19
HandleTheJandal 8 hours ago 0 replies      
Thank you for helping to promote reproducibility of published results by supporting anonymous peer review! Science Exchange (YC S11) is also making great progress in the facilitation of scientific reproducibility. We just completed independently validating select results from 50 cancer biology papers. https://news.ycombinator.com/item?id=8731274
20
Animats 7 hours ago 0 replies      
Another anonymous review system. What could possibly go wrong?

If there's anything we've learned by now about "crowdsourced" review systems, it's that, without an elaborate way to evaluate reviewers, they fail. Badly. Facebook "likes", Google "+1", and Yelp reviews are heavily spammed. This just does not work.

21
nemoniac 4 hours ago 0 replies      
What the scientific peer review needs is not anonymity, but accountability. Authors put their reputation on the line. Let reviewers put their reputation on the line too. Good reviews and good reviewers need to be appreciated, not anonymized.

Let valued scientific reviewers gain reputation in a similar way to how contributors to HN, SE and other sites do.

There could even be a viable business model in this. Publishers, be creative!

22
colechristensen 2 hours ago 0 replies      
Let's be sure to point out some terrible journalism, reading the following makes me completely disinterested in finishing the article and have considerably less respect for Wired.

> Have you ever questioned the claims that scientists make? For example, last years discovery of the so-called God particle,

Using the nickname for the Higgs in the context of questioning science claim is nothing but bait for the foolish and misinformed, and in this way anyone scientifically literate should seriously question any claims or opinions in this publication.

23
vacri 8 hours ago 1 reply      
Fortunately, the First Amendment is on our side. It protects the right to anonymous speech.

I don't understand how "The government cannot outlaw your speech" means "Your anonymity is protected by law from private parties". How does that interpretation come about?

11
Ancient doodle hints that Homo erectus was smarter than we thought
24 points by mbrubeck  4 hours ago   1 comment top
12
What if journalists had story writing tools as powerful as those used by coders?
255 points by danso  14 hours ago   124 comments top 28
1
jawns 13 hours ago 7 replies      
When I was working as a web editor at a metro daily paper a few years ago, I proposed something similar: an XML-like syntax that would allow for metadata to be included in drafts of news stories, some (but not all) of which could be made use of in online versions of the story (such as a link to a map when you're referencing a location).

A lot of wire copy already includes metadata, but it's generally just in a header that accompanies the story.

What I was envisioning was something more like what is being proposed for the semantic web:

<name id="1394">John Smith</name> was elected president of the <organization id="2315">New Castle County Council</organization> on <date value="2014-12-10">Wednesday</date> at the <place lat="39.685881" long="-75.613047">county headquarters</place>.<source id="23" name="Mila Jones" title="New Castle County public relations officer"></source>

I also wanted to use the metadata to help copy editors trim wire stories:

<priority value="1">This amounts to de facto resegregation. <priority="4">(And we all know how we segregation worked out the first time.)</priority> If the school district still values integrated schools, it must act swiftly to correct this effect.</priority>

It turns out, though, even when you create a UI that lets reporters and editors easily plug in this metadata without having to understand XML, they are not apt to fill it in, because they are just so overworked as it is.

Plus, in order for this to work on a larger scale, you'd have to get an incredible amount of buy-in. You'd have to get reporters and editors to agree that it's worth their time. You'd have to build software to support it. You'd have to get all of the different media companies out there to agree on standards.

It's just ... not what the media industry is (or should be) focused on right now. They've got bigger things to think about, like how to find a viable business model.

2
Alex3917 13 hours ago 6 replies      
I've been messing around with something similar, specifically though focused on tracking strengths and weaknesses in US infrastructure:

http://www.alexkrupp.com/Citevault.html

It's actually a lot easier than you'd think, because thanks to Zipf's law pretty much every article in these evergreen topic areas is using the same set of 1,000 or so facts. And these facts mostly come from the same set of government or NGE reports, which are updated at most once a year, and often only once every ten years.

The cool thing is that you can then use a javascript snippet to track which facts are being used in which documents, automatically mark facts as outdated when they change, etc.

3
pavlov 11 hours ago 2 replies      
It's true that coders have lots of great tools for working with textual representations of programs... But IMHO programmers' tools are stuck in a certain kind of local maximum due to the difficulty of moving beyond text. We've done all we can to make textual programs easier to manipulate, but there are fundamental difficulties that can't be solved this way.

I'm personally interested in this question: What if coders had design tools as powerful as those used by architects and construction engineers?

I wrote a blog post about this recently: http://blog.neonto.com/?p=44

4
tezza 10 hours ago 3 replies      
Most journalists are barely informed cut'n'paste merchants.

Journalists value the sensational over the factual, and work hard (true) to tight deadlines.

So they already do not care about the tech features promised, namely indicating:

* there is not enough evidence to make a given point

* a certain person or company has not been investigated thoroughly enough

* a certain point is not relevant

[edit: data points]

Cut'n'paste obituary from Wikipedia: http://www.theregister.co.uk/2007/10/03/wikipedia_obituary_c...

"Hack": A self referential term journalists use for each other in the UK http://en.wikipedia.org/wiki/Hack_writer

5
declan 10 hours ago 0 replies      
I worked full-time as a journalist for a bunch of different news organizations before leaving to found the forthcoming http://recent.io/.

My suspicion is that neither I nor other journalists I know would use this newsclip.se tool. In a single newsroom I've seen people using Word, TextEdit, Google Docs, Notepad, BBEdit, Gmail, phone-based email clients, phone dictation, and even emacs (me, a few times) to write news articles. While the CMS is newsroom-wide, the writing and editing processes tend to be very personal. Journalists also tend to be individualistic and dislike being forced to use a standardized system without clear benefits.

Where something like newsclip.se might be beneficial would be as a kind of preprocessor/lint for the CMS. It could do a lot of what's being described in the linked article but without replacing the entire journalistic stack.

6
Hansi 13 hours ago 1 reply      
When I wrote my disertation I used Scrivener: http://www.literatureandlatte.com/scrivener.php which is a wonderful tool for writing. An expanded open source application similar to that with a good plugin system might be very useful.
7
cryoshon 13 hours ago 2 replies      
Making a tool like this has occurred to me, but my idea had a slightly different focus.

Rather than attempt to link evidence to statements, which, if we're being honest, doesn't really bring us closer to the truth since many "sources" are merely other people's words, it was much simpler: identify weasel words, euphemisms, or use of the passive voice. Between these three features, I think that most factual writing could be improved a colossal amount.

I would certainly appreciate an easier way to keep track of claims, citations, evidence, and interplay between a story's moving parts, though. I think that the article mentions a few tools which work toward that goal admirably. Right now, to make a really bulletproof piece, you need to be extremely scrupulous with self-identifying your claims and then providing written explication of evidence or hyperlinked evidence.

Additionally, it'd be really useful to have a tool which kept track of sentence structure and also allowed you to track logical rhetoric by keeping track of "If this, then that" style statements.

This is probably asking too much, though. A final hurdle is that journalists and writers tend to be old-school when it comes to technology, so it might be a hard sell to the older segment of that market.

8
LukaAl 11 hours ago 6 replies      
What do you mean for "writing tools as powerful as those used by coders"? In my experience good coders doesn't need very powerful tool. Super-complex IDE are usually a disturbance to good coders rather then an help and even the uber-geeks that use Emacs don't really use in their jobs all the macros.

A lot of good software engineer I know prefer lightweight editor like the good old Vim or SublimeText over IDE like Visual Studio, Xcode or Eclipse. That's because when you write complex code your brain is slower than your finger and the syntax and structure of the language you are using is something under your skin, you don't need to think at it.

So, no, great coder doesn't use powerful editors, if they can chose they use very simple one with the minimum level of features they need to help them think faster (syntax highlighting, autocompletion, indention) and stop. All other features of complex IDE are a disturbance rather than an help.

I don't know what a good journalist need but my guess is that they have the same problem, their brain is slower than their finger and the grammar of the language they are using should be well known. I don't know which tool they use, but in my opinion a simple text editor with a minimal spellchecking system (but not correction) well integrated with the publishing process is more than enough.

From an Innovation point of view I'm usually suspect when someone claim to have an innovative tool for an established industry. If the industry is established the actors have already optimized their tools and if there could be an improvement is very small. What usually happens is that something changes in the industry and make the actual tool obsolete (e.g: a new process or a new technology became available). But if you want to create something really innovative, you have to find this change. Simply bringing something existing in a field in a different field rarely works. And if it works is because you have a really good understanding of both fields.

9
dredmorbius 6 hours ago 0 replies      
There's a standard called hNews, created by Jonathan Malek (Associated Press), Stuart Myles (Associated Press), Martin Moore (Media Standards Trust), and Todd B. Martin (Associated Press), which addresses this:

hNews is a microformat for news content. hNews extends hAtom, introducing a number of fields that more completely describe a journalistic work. hNews also introduces another data format, rel-principles, a format that describes the journalistic principles upheld by the journalist or news organization that has published the news item. hNews will be one of several open standards.

http://microformats.org/wiki/hnews

I learned of it (yesterday) re-reading Readability's Article Publishing Guidelines:

https://readability.com/developers/guidelines

hNews is an XHTML microformat -- the tags are entity names and classes added to standard HTML entities.

Key among them are: hnews, hentry, source-org, dateline, geo, item-license, principles.

Other microformats, including hCard, can be used to identify people, companies, and organizations, similar to vCard properties. The elements will be familiar to those who've worked with address, Active Directory, or LDAP data: fn, n, nickname, org, email, tel, adr, and more.

http://microformats.org/wiki/hcard

An IDE that tied into these (or, possibly, other standards) could be useful.

You'd likely need some sort of natural-language processing in the copyediting or publishing process to apply this uniformly. Field reporters may work on a wide range of equipment and software (including pen-and-paper or simply voiced-in reports). And expecting reporters to incorporate tags into their copy is likely a stretch.

10
aethertap 13 hours ago 1 reply      
This is a great idea. So much so that I have "Why didn't I think of that?" syndrome.

Here are a couple of suggestions that I could use if you're looking for feature requests. Most of these things exist in one place or another, but having them integrated into a one-tool workflow would be awesome.

1. Some kind of crowdsourced reputation system for sources (i.e. medical journal sites have high reputation, naturalnews.com has low)

2. Auto cross-referencing between articles based on content.

3. TODO list management

4. License-aware relevant image suggester (please!!!) This alone would be a killer feature for me. Pick out topic words and search selected image sites, then give me thumbnails to choose from.

11
Fede_V 12 hours ago 2 replies      
My distinct impression is that the limiting factor in quality journalism is gum-shoe reporting, not incredibly powerful CMS.

Buzzfeed has the most advanced CMS, but their reporting pales with respect to the NYT. When they do bother to do proper reporting (McKay Coppins, etc) they get excellent stories - the rest of the time they use filler, because it's cheap to produce.

12
phkahler 13 hours ago 8 replies      
I've always wondered why writers don't use things like revision control and decent diff tools. I'm not sure the existing tools are well suited to them (yet).
13
morgante 12 hours ago 1 reply      
This is a very cool prototype, and I especially like the idea of calling it a "journalism IDE" (as opposed to a CMS).

At work (http://cafe.com), I've actually been working on something similar with our CMS (Monsoon). We're trying to use technology to make telling cohesive online narratives a lot easier.

Interestingly, one of the biggest hurdles so far has been in decomposing stories. Media traditionally treats each story as a big blob of text (in most cases, HTML), but we're trying to change that so that each story is actually just an arrangement of smaller tidbits (we call them droplets). Switching to that model helps us to encode a lot more semantic information, and also to reflow stories effectively for context.

We're not yet to the point where we integrate/suggest droplets from other stories automatically, but that's definitely the goal. Maybe we could integrate something like Newsclip.se to encourage that.

(PS. If you want to help us get there, we're hiring: http://cafe.com/careers)

14
pudo 13 hours ago 3 replies      
Hi all, post author here! We're working on the code base and would love everybody's input here: https://github.com/pudo/tmi
15
taylorbuley 6 hours ago 0 replies      
The developer in me likes this very much. I mean, Taxonomies! Unfortunately many non-journalists tend to romanticize and over-complicate the craft. Journalists care mostly about the story telling, and it's not clear to me how this translates into a better story. I believe this to be a (beautiful and well-intentioned) example of overengineering a domain model that, at it's most basic, involves just titles, deks and content such as text, images and video. To me it does not clearly push journalism toward a more profitable and sustainable future and thus, is a distraction from more challenging problems at hand.
16
couchand 12 hours ago 0 replies      
This somewhat plays into Rebecca Parsons' talk [0] at hack.summit() last week. She emphasized that we've done a great job building tools for other technical folks, but we've really dropped the ball on supporting non-technical fields. Her perspective was specifically on the potential of DSLs, but the call to developers to be more relevant is likewise compelling.

[0]: https://www.youtube.com/watch?v=3KO2YmU5RBY

17
spypsy 12 hours ago 1 reply      
It's funny, I was thinking about that while writing my dissertation.

After a year of coding on Visual Studio, during writing I would say something like "As mentioned in previous chapters, the decision was based on..." and then just click on "previous chapters" and press F12, expecting to be taken to the original reference.

Unfortunately I found out Word does not offer such functionality just yet

18
Spooky23 9 hours ago 0 replies      
We had some Github religious adherents who made a lot of noise about this. The net in our case ended up being "You can eliminate complex stuff like Word and just use markdown & Github!" (I almost spit out my coffee laughing)

I don't think that an IDE model makes sense because writing in human language is more complex and nuanced than programming languages, which are more limited in scope. At the end of the day, IDEs are shims to match human desires to the language the machine expects.

19
rtpg 2 hours ago 0 replies      
reading this I feel like it would totally be possible to use CRM software like Salesforce to manage investigating a story. Leads and the like, it's almost like the workflow is the same.
20
jseliger 10 hours ago 0 replies      
Great article, and great idea.

I'm not a journalist but I do a lot of writing, and I actually use a program called Devonthink Pro in the manner described here by Steven Berlin Johnson: http://www.stevenberlinjohnson.com/2005/02/devonthink_cont.h... . It's surprisingly close to the tools Lindenberg is describing.

21
rrggrr 7 hours ago 0 replies      
http://www.literatureandlatte.com/scrivener.php

Scrivener comes closer than anything else I've seen to meeting the OP's goal.

22
mvc 11 hours ago 0 replies      
Interesting fact. The Django web framework was initially created by the dev team at the "Lawrence Journal", a newspaper in Kansas.
23
davidgerard 7 hours ago 1 reply      
This is vaguely reminiscent of some of the professional tools for scriptwriters and novelists, which do in fact start to resemble IDEs: you can shuffle around characters, outlines, plot fragments ...
24
heyts 11 hours ago 0 replies      
This article, incidentally by one of the creators of the Django Framework, still seems relevant, even 8 years later:

http://www.holovaty.com/writing/fundamental-change/

25
vonnik 11 hours ago 0 replies      
Newsclip.se is a great idea.

As a former journalist, I think that http://hypothes.is will become another very important tools for reporters... It's an annotation layer for the Web.

26
iondream 12 hours ago 0 replies      
I writing enough like coding to make this worthwhile?
27
jtth 9 hours ago 0 replies      
It's called Tinderbox and it's awesome.
28
dash2 11 hours ago 1 reply      
Hmm...

* Your weekend newspaper would come out on Wednesday.

* There would be new editions many times daily... not with new stories, just corrections of the first edition, which was blatantly inaccurate and partly written in Moldovan.

* Every day, the newspaper would be in a different format which didn't fit the newspaper rack you just bought.

* Every week, the newspaper would get bigger, but contain no more content (just a new font). You would regularly be forced to buy a new newspaper rack.

* Also, once a week the paper boy would break into your house and steal your old papers. He would offer to sell them back for you in the new format, for a higher price.

* Also, the newspaper rack sellers would not let you store newspapers of which they disapproved.

* Rather than telling you about the world, the paper would track your behaviour and tell the world about you.

* Once a week, the front page would be 404 NEWS NOT FOUND.

* Reporters would be paid high six-figure salaries, but would be unable to relate or talk to anyone but other reporters.

* Many journalists would consider themselves brilliant, world-changing geniuses, with plans not just to report on government, but to replace it.

* At the same time, they would have secret deals with those governments to report people who read "subversive" news.

* et cetera...

13
Databases at 14.4Mhz
217 points by chton  14 hours ago   58 comments top 11
1
ChuckMcM 9 hours ago 1 reply      
This is Foundation DB's announcement they are doing full ACID databases with a 14.4M writes per second capability. That is insanely fast in the data base world. Running in AWS with 32 c3.8xlarge configured machines. So basically NSA level data base capability for $150/hr. But perhaps more interesting is that those same machines on the open market are about $225,000. That's two rack, one switch and atransaction rate that lets you watch every purchase made at every Walmart store in the US, in real time. That is assuming the stats are correct[1], and it wouldn't even be sweating (14M customers a day vs 14M transactions per second). Insanely fast.

I wish I was an investor in them.

[1] http://www.statisticbrain.com/wal-mart-company-statistics/

2
hendzen 9 hours ago 4 replies      
This is very impressive, however...

See this tweet by @aphyr: https://twitter.com/aphyr/status/542755074380791809

(All credit for the idea in this comment is due to @aphyr)

Basically because the transactions modified keys selected from a uniform distribution, the probability of contention was extremely low. AKA this workload is basically a data-parallel problem, somewhat lessening the impressiveness of the high throughput. Would be interesting to see it with a Zipfian distribution (or even better, a Biebermark [0])

[0] - http://smalldatum.blogspot.co.il/2014/04/biebermarks.html

3
jrallison 10 hours ago 1 reply      
We've been using FoundationDB in production for about 10 months now. It's really been a game changer for us.

We continue to use it for more and more data access patterns which require strong consistency guarantees.

We currently store ~2 terabytes of data in a 12 node FDB cluster. It's rock solid and comes out of the box with great tooling.

Excited about this release! My only regret is I didn't find it sooner :)

4
bsaul 9 hours ago 1 reply      
Just watched the linked presentation about "flow" here : https://foundationdb.com/videos/testing-distributed-systems-...

Is it really the first Distributed DB project to have built a simulator ?

Because frankly, if that's the case, it seems revolutionary to me. Intuitively, it seems like bringing the same kind of quality improvement as unit testing did to regular software development.

PS : i should add that this talk is one of the best i've seen this year. The guy is extremely smart, passionate, and clear. (i just loved the The Hurst exponent part).

5
felixgallo 10 hours ago 1 reply      
This looks very interesting and congratulations to the FoundationDB crew on some pretty amazing performance numbers.

One of the links leads to an interesting C++ actor preprocessor called 'Flow'. In that table, it lists the performance result of sending a message around a ring for a certain number of processes and a certain number of messages, in which Flow appears to be fastest with 0.075 sec in the case of N=1000 and M=1000, compared with, e.g. erlang @ 1.09 seconds.

My curiosity was piqued, so I threw together a quick microbenchmark in erlang. On a moderately loaded 2013 macbook air (2-core i7) and erlang 17.1, with 1000 iterations of M=1000 and N=1000, it averaged 34 microseconds per run, which compares pretty favorably with Flow's claimed 75000 microseconds. The Flow paper appears to maybe be from 2010, so it would be interesting to know how it's doing in 2014.

6
w8rbt 6 hours ago 0 replies      
I thought it was DB connections over radio waves just above 20 meters. Also, it's MHz, not Mhz.
7
shortstuffsushi 9 hours ago 1 reply      
As someone who has no idea about the cost of high-scale computing like this, is $150/hr reasonable? It seems like an amount that's hard to sustain to me, but I have no idea if that's a steady, all the time rate, or a burst rate, or what. Or if it's a set up you'd actually ever even need -- seems like from the examples they mention (like the Tweets), they're above the need by a fair amount. Anyone else in this sort of situation care to chip in on that?
8
dchichkov 9 hours ago 2 replies      
I'm only familiar with other key-value storage engines, not FoundationDB, but it seems like the goals are: "distributed key-value database, read latencies below 500 microseconds, ACID, scalability".

I remember evaluating a few low latency key-value storage solutions, and one of these was Stanford's RAMCloud, which is supposed to give 4-5 microseconds reads, 15 microseconds writes, scale up to 10,000 boxes and provide data durability. https://ramcloud.atlassian.net/wiki/display/RAM/RAMCloud Seems like, that would be "Databases at 2000Mhz".

I've actually studied the code that was handling the network and it had been written pretty nicely, and as far as I know, it should work both over 10Gbe and Infiniband with similar latencies. And I'm not at all surprised, they could get pretty clean looking 4-5us latency distribution, with the code like that.

How does it compare with FoundationDB? Is it completely different technology?

9
mariusz79 10 hours ago 0 replies      
MHz not Mhz.
10
imanaccount247 8 hours ago 1 reply      
Why the deliberately misleading comparisons? If you are doing something genuinely impressive, then you should be able to be honest about it and have it still seem impressive. One tweet is not one write. Comparing tweets per second to writes per second is complete nonsense. How many writes a tweet causes depends on how many followers the person who is tweeting has. The 100 writes per second nonsense is even worse. Do you just think nobody is old enough to have used a database 15 years ago? 10,000 writes per second was no big deal on of the shelf hardware of the day, nevermind on an actual server.
11
lttlrck 11 hours ago 2 replies      
"Or, as I like to say, 14.4Mhz."

Sorry, I don't like that at all.

14
A City That Is Moving 9 Kilometers Down the Road
97 points by Thevet  9 hours ago   42 comments top 11
1
michaelleland 7 hours ago 2 replies      
This is an interesting article to me, and fairly near to my heart--I'm the founder of a small "remote-enabled" startup based in Washington State that moved to Kiruna for a year. I'm 6 months into the trip now, typing this within site of the mine in question.

The interesting part of this article to me was the part that discussed the anthropologist's findings, towards the end of the article. It confirms what I've seen anecdotally as well--the men are mostly ambivalent, some slightly optimistic, and the women are slightly anxious. It's a big project, but the timeframe of multiple years greatly reduces the stress on people. The businesses are changing plans slowly to accommodate, as are government offices and government-owned housing authorities, but there is no rush.

I've already seen a bit of the effect on my emotions--the first time I traveled here, I got on and off at the old train station, which is now on its way to demolition. Physical places that hold memories are for me a sort of safety, and seeing them go is slightly painful. It would be hard, I imagine, to watch my parent's home be destroyed.

Overall, the move makes me confident of mankind's ability to cope with climate change. Kiruna is only a small town, and it's moving only a short ways away, but it is a massive undertaking that requires a lot of money and (perhaps more difficultly) a lot of cooperation. If Kiruna can do it, larger towns can too, at least when faced with an obvious existential threat.

2
mrb 6 hours ago 2 replies      
The title is incorrect. It is only the downtown area that will be relocated, not the whole city. And it will be moved 2.5 km, not 9 km. The downtown area is currently geographically located on the west edge of the city, and will move to the east edge of the city, see the map in this article: http://ecosistemaurbano.org/english/ecosistema-urbano-prequa... I have no idea where the "9 km" claim comes from. It is not quoted in the article. This is possibly a mix-up with an unrelated city mentioned in the article (Newtok in Alaska) which is considering relocating "9 miles" away.

Edit: @troymc: the BBC article is likely incorrect. Notice it quotes no sources. Maybe it assumed the whole city will move. Or maybe it takes into account population growth from people moving from other cities to Kiruna, because of the mine's expansion and increased economic/job opportunities. Either way, it is pretty obvious from all the relocation maps you find online that only the western edge of the city will move. Even the rendering at the original article http://nextcity.org/images/made/2033_plan_1200_862.png shows that only the western edge will disappear (compare it with a Google Maps satellite view).

3
jnbiche 8 hours ago 1 reply      
Kiruna is in an unfortunate predicament, but if I had a company willing to relocate my home, family, and community 10 miles down the road in order to save the only job I would ever have in the region (and a well-paying one, at that), I think I'd be pretty happy about it.

And this is far from the only city to have ever been moved (as the article noted). A much more tragic story is that of those towns and villages in Palestine, where communities of over 700,000 people were uprooted and families forced to relocate wholesale to refugee settlements when Israel was declared in 1948. These settlements in Gaza and West Bank that have retained the names of the original communities whence their inhabitants came several generations before.

4
jahnu 6 hours ago 1 reply      
"Meanwhile, the imminent move offers people little incentive to refurbish their properties and as an inevitable result, the city has become rundown."

https://www.google.com/maps/place/Kiruna,+Sweden/@67.8553152...

Looking around the place, while it's not exactly Paris it does seem the author has a different definition of "rundown" than most people.

5
SwellJoe 6 hours ago 1 reply      
Ah, yes, "plan B". That was a great episode of the Simpsons'. https://en.wikipedia.org/wiki/Trash_of_the_Titans

On a more serious note, it's kinda scary to what lengths people will go to continue doing what they've always done, even if it destroys their very homes. I'm not saying they're wrong to want to keep their jobs, but, humans operate on such a huge scale and without a lot of foresight, and it's scary to think of how that will play out with 9 billion humans today and even more tomorrow.

6
ChuckMcM 7 hours ago 2 replies      
Interesting. I expect if I were moving a city I would first build a layer for transport and utilities, and then build the city on top of that. In the past we've built cites on the ground and then dug sewers and subways etc underneath. But if you built first a layer that was going to be the "underneath" of the city, and then built the city on top (including parks and lawns and what not, you could really do a good job of making sure the connections were solid.
7
phkahler 2 hours ago 0 replies      
I like the idea of high density right to the edge of town. You get all the walkability but also green (or white that far north) space right nearby. I think the ideal might have some characteristics of fractals to allow all the dense areas to border on the empty areas.
8
grecy 7 hours ago 2 replies      
> Located well within the Arctic Circle, the bleak, almost unpopulated region around Kiruna

That sentence makes me think of some of the s__t-hole mining cities and towns in and around the Arctic around here in Alaska / Yukon and NWT.

The photos in the article show that by North American standards, it's an extremely beautiful city.

9
Animats 7 hours ago 2 replies      
In 1900, Galveston, TX, was moved about 10 feet vertically. After a major hurricane, a big seawall was built. Then the buildings behind it were jacked up and the ground filled in,. It worked quite well; when the next hurricane hit in 1910, the damage was far less.
10
thrownaway2424 7 hours ago 2 replies      
I'm surprised that mining iron in such conditions can be profitable. Isn't it incredibly abundant (~5% of the earth's crust)?
11
beat 6 hours ago 0 replies      
Towns move. Shawneetown IL was moved a couple of miles away from the Ohio River, after the flood of 1937 wrecked the original location.
15
When your IP traffic in AWS disappears into a black hole
146 points by schimmy_changa  10 hours ago   48 comments top 13
1
jcollins 7 hours ago 0 replies      
We saw this earlier this year after upgrading to a new Linux kernel.

The solution for us was to set this in sysctl.conf:

net.ipv4.neigh.default.gc_thresh1=0

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1331150...https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1331150...

2
danesparza 10 hours ago 7 replies      
Am I weird because I actually muttered 'ARP caching issue' halfway through your article? :-)

Love the technical write-up -- thanks!

3
ChuckMcM 10 hours ago 1 reply      
Interesting trace down to stale ARP entries. It gets worse when the switches are running mac address filtering and they get out of date. We had that issue with some Blade G8052 top of rack switches with their upstream 10G ports. They sometimes "forget" which upstream port has the MAC address that they are switching too, and those packets just spew out messily into the data center leaving a mess. The "fix" it to force the switch to ping up through a specific upstream port periodically to the center switch's management IP address. Sigh.
4
spectre256 9 hours ago 1 reply      
This reminds me of a time at a previous company years ago, where we experienced an issue that felt similar, although the root cause was quite different.

Basically, we had multiple teams all launching/terminating web servers. Unfortunately, they were all in the same EC2 deployment, and more often than not our load balancers from one team would send traffic to the web servers of another team. Furthermore, our setups were similar enough that this would sometimes cause bad results for users. We fixed it by making sure that our web servers on every team spoke on different ports. Not elegant, but effective (until two teams accidentally picked the same ports).

These days we have good enough infrastructure tools that this problem should never happen. But in 2009, at a company that was overwhelmed with growth, those sort of things happen.

5
schimmy_changa 10 hours ago 0 replies      
I think the biggest thing I was surprised by with this investigation was the lack of documentation about data-layer tools. At one point I was looking through the source of the 'ip' command to try to find out exactly which conditions caused a 'STALE' entry in the ARP table...
6
falcolas 9 hours ago 0 replies      
Try an arping from the new workers on first startup? Ran into this quite a bit when using VIPs for DB failover, and an arping fixed the caching issue in most cases.
7
girvo 5 hours ago 0 replies      
We had a fascinating bug on EC2 -- we could connect to the instance, but no network traffic made it out. It wasn't security group problems, it was literally a really weird bug in EC2's network that we somehow triggered, the engineer over at Amazon that looked at it was really excited when he came across our case as it was so weird, heh. They fixed it, I can't remember exactly what was done on their end, but it was one of the weirder problems I've attempted to debug. Nothing I tried worked!
8
maerF0x0 10 hours ago 4 replies      
I wonder if this is a problem for any cloud provider, I also wonder if ipv6 could help mitigate? Then the IP collusions would be rarer.
9
zenocon 5 hours ago 1 reply      
I just experienced this early this week. Very frustrating. I also posted to AWS forums and got zero assistance; am currently not paying for AWS support plan. This article came at an opportune moment -- it makes sense and removes the shroud of mystery around why it "works sometimes" which leaves me with an uneasy feeling for a production setup.
10
wahnfrieden 9 hours ago 1 reply      
FYI, Clever: I click "Engineering Blog" at the top, and all links to blog posts on that page 404.
11
perlgeek 8 hours ago 1 reply      
Wouldn't it be a better solution to not reuse IP addresses quickly? If I understood it correctly, they are in a private network anyway, so they could afford it.
12
kiyoto 6 hours ago 1 reply      
Looking at the port number, it looks like Clever is a MongoDB user =)
13
j-kidd 6 hours ago 1 reply      
You shall read this first: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1331150

There was a rather significant change to the kernel ARP caching behavior introduced in early 2013: https://github.com/torvalds/linux/commit/2724680bceee94eac39...

It seems to work fine everywhere, except in EC2 VPC, where the arp cache can sometime becomes stale. We too reported this issue to AWS support, but have no idea if they are doing anything about it.

The workaround is to apply a sysctl change to revert to the old behavior prior to the commit. Or to use a subnet larger than /24 to reduce the chance of getting the same IP.

16
Elm 0.14 Simpler Core, Better Tools
204 points by teamonkey  13 hours ago   57 comments top 14
1
dcre 12 hours ago 0 replies      
If you're interested in Elm but don't really know what it's about, I highly recommend Evan's Strange Loop talk, "Controlling Time and Space: understanding the many formulations of FRP."

It will help to already have a bit of background in functional programming.

https://www.youtube.com/watch?v=Agu6jipKfYw

2
pflanze 8 hours ago 2 replies      
The Elm website is blank without JavaScript enabled. Here's a script to extract the Markdown in the given page and optionally format it without having to enable JavaScript in the browser:

https://github.com/pflanze/de-elm

(Edit: here's a copy of the content of the linked page:https://github.com/pflanze/de-elm/blob/master/examples/annou... .Tell me if you don't like this and I'll remove it again.)

3
rnhmjoj 10 hours ago 3 replies      
What is the point of changes like [] to List, mod to %, . to << and >>?Is it just aesthetics?

When I looked at Elm it was really similar to Haskell. After I learned a bit of Elm I dove into Haskell. Now that I'm coming back to Elm I'm having problems because I confuse this different syntaxes many times.

I hoped they could maintain the language more similar to it to make the switch between them easier.

4
couchand 9 hours ago 1 reply      
The benefits are actually much deeper though. Now that we know exactly how the API has changed, it is possible to automatically enforce strict versioning rules. If there are breaking changes, the new release must be a major version bump.

This is very interesting to me. It demands consideration: do we need a person to manually bump the version anymore at all? If you can statically verify that a change meets the criteria, couldn't the package publishing process autoincrement the appropriate version segment and be done with it?

5
StevePerkins 12 hours ago 1 reply      
Have there been any performance improvements under Firefox? Almost every Elm demo that I've seen so far has been basically unusable under that browser. I've been told that the issue is some difference between Chrome and FF with canvas drawing... but regardless of where the fault lies, Elm is pretty much a Chrome-only technology until that gets worked out.
6
bjz_ 12 hours ago 3 replies      
It's nice to see that they have replaced Either with Result[0]. I wonder if Rust's Result type[1] was the inspiration? (not sure if there is prior art)

[0]: http://elm-lang.org/blog/announce/0.14.elm#making-error-hand...

[1]: http://static.rust-lang.org/doc/master/std/result/index.html

7
AlexanderDhoore 10 hours ago 1 reply      
Elm really is fascinating... Am I the only one who thinks FRP is just a reincarnation of dataflow programming?

I'm learning Verilog in my spare time, because I want to implement a dataflow/FRP language in hardware. (People tried to build massively parallel hardware in the past to run dataflow languages... I'm pretty sure it's a good idea.) Should be interesting!

8
davexunit 12 hours ago 0 replies      
Awesome. I like the name change from 'lift' to 'map'. Also, the public API diffs that the packaging tool provides looks very neat. Congrats to the Elm team!
9
lsjroberts 11 hours ago 0 replies      
I used Elm for my Ludum Dare entry. As a first attempt using a functional language I definitely did many things wrong, but I found it very easy to iterate. Great language and I'm going to look into going further with it for game development.
10
shard 5 hours ago 0 replies      
For a second I had thought that someone had released an update for the Elm mail client. Color me disappointed (although I am more of a Pine guy myself, up to ~2006 at least).
11
thewarrior 12 hours ago 3 replies      
Is anyone here using Elm in production ?
12
tree_of_item 12 hours ago 3 replies      
Eh, that's a tagged union, but I think "union type" usually refers to untagged unions like you see in more dynamic languages such as TypeScript and Facebook's Flow. Tagged unions are typically called sum types. That is, searching for "union type" is gonna give you results about untagged unions or C style unions. I'm not sure this new terminology helps.
13
jmgrosen 12 hours ago 0 replies      
That's pretty ridiculous. Literally the day after I start working on my Elm project in a couple months...

Anyway, the changes here do look pretty nice. Looking at the docs, it seems that they've added more examples, which is always appreciated! I still don't like the impurity that comes with the Signal.Channel, though... not that I have a better idea.

14
lucian1900 12 hours ago 2 replies      
Very cool.

Interestingly enough, the dice is loaded. If you click long enough, you'll get an upside down V shape.

17
Writing a Phase-locked Loop in Straight C
30 points by blueintegral  6 hours ago   1 comment top
1
phunge 7 minutes ago 0 replies      
PLLs are fascinating. Software PLLs are less fun than messing with the hardware, in my experience!

I started working on a software PLL recently -- not analog like this one, digital. Specifically, I was trying to work out the math on a fully event driven type-II PLL -- i.e. instead of updating at a fixed sampling rate, it'd wake only when an edge happened on the input or VCO. I was starting to get curious whether there's a solution for that somewhere on the web.

This is a good intro to (hardware) PLLs in general: https://www.youtube.com/watch?v=0jzLDe950AY followed by http://www.ece.tamu.edu/~spalermo/ecen620.html

18
A Twenty-First-Century Shipwreck
6 points by Vigier  4 hours ago   discuss
19
High Temperature Superconductivity Record Smashed by Sulphur Hydride
86 points by jonbaer  13 hours ago   18 comments top 4
1
throwaway_yy2Di 8 hours ago 3 replies      
Tangentially, the metallic hydrogen mentioned is one of the craziest chemical substances to possibly-exist [0].

Besides likely being a room-temperature superconductor (at ridiculous pressures, like 500 GPa), it's postulated to be metastable -- like diamond, you could create it at pressure, and it might stay a solid metal at STP conditions. It's postulated to be made of atomic hydrogen -- lone H atoms, without the molecular bonds of H_2. The recombination energy H + H -> H_2 suggests [1] it's the most energy-dense chemical fuel that exists, with 20 times the specific energy of {H2 + O2}. It could allow [1] rocket engines with I_sp of 1,700 seconds -- four times higher than LH2/LOX. It's thought to be the main phase of hydrogen inside the planet Jupiter [2] and responsible for its dynamo [3] (but as an ordinary conductor, not a superconductor). It's also speculatively a structural material, one that's less dense than water [4].

It might have been created in a lab, in 2011 [4], but it's not clear.

[0] https://en.wikipedia.org/wiki/Metallic_hydrogen

[1] http://www.nasa.gov/pdf/637123main_Silvera_Presentation.pdf

[2] https://en.wikipedia.org/wiki/Jupiter#Internal_structure

[3] https://en.wikipedia.org/wiki/Magnetosphere_of_Jupiter

[4] http://www.nature.com/news/metallic-hydrogen-hard-pressed-1....

2
jlarocco 47 minutes ago 0 replies      
Completely off topic, but I really hate articles where all I initially see is a headline and a large, unrelated image.

I hope the trend dies quickly...

3
perlgeek 8 hours ago 2 replies      
When I studied physics five years ago, my impression was that nobody really understood high-temperature superconductivity.

There were some mathematical models, and some professors who claimed to understand it, but nobody was able to give a coherent explanation to the (mostly nearly finished) students, much less predict which materials would exhibit hight-temp superconductivity.

Does anybody know if that changed significantly? The article reads as though the measurements were inspired by the theory, which is always a good sign.

4
ableal 8 hours ago 1 reply      
"[...] have measured sulphur hydride superconducting at a temperature of 190 Kelvin (-83 degrees Celsius). There is a caveat, of course. The material has to be squeezed at pressures greater than 150 gigapascalsthats about half the pressure at the centre of the Earth."

And 1.5 million times the atmospheric pressure at sea level ...

20
Seizing Control of Yahoo Mail Cross-Origin Again
36 points by anglebracket  7 hours ago   1 comment top
1
_asummers 2 hours ago 0 replies      
I'm willing to give Yahoo! the benefit of the doubt that they're working on that general class of issues, but it needs to be a priority to fix the underlying cause of this exploit. That it took 8 months for this guys report to get taken seriously is silly.
21
Travis CI: From Open to Minimum Vacation Policy
607 points by michaelochurch  15 hours ago   305 comments top 45
1
Someone1234 14 hours ago 15 replies      
I've said it before and I'll say it again, unlimited days off (or "open") and zero days off are identical.

If you have unlimited vacation people are inclined to take less and people respect the vacation you take less ("as you can always take more!").

Here's a study[0] (PDF) called "Overwhelmed America: Why Dont We Use Our Earned Leave?" It is biased (travel association) but interesting nonetheless.

According to this study[1] you need at least ten consecutive days of leave to "de-stress" from work. Short vacations aren't as effective as long ones. In "europe" a two week vacation (10 work days) is common/standard. As opposed to American's "long weekends."

A lot of these "unlimited" places have a "as long as your work gets done" policy, meaning you can take tons of short days off or afternoons off, but almost no extended holidays (e.g. travel abroad, out of state, etc).

[0] http://traveleffect.com/sites/traveleffect.com/files/Overwhe...

2
sportanova 14 hours ago 7 replies      
Of course the other reason for 'unlimited vacation' is so when an employee leaves, the company doesn't have to pay them for unused vacation days. Huge red flag when a company has 'unlimited vacation'
3
jcadam 14 hours ago 1 reply      
I like this idea of minimum vacation. Most companies I've worked in use vacation days as some sort of reward for sticking with the company for X number of years (i.e., "When you hit your 5 year anniversary, you get an extra 1.5 days of paid vacation. Won't that be nice!"). And when I say paid vacation, I really mean PTO with no separate 'sick leave' days. Which means people show up to work sick.

Funny enough, the company I work for was recently bought, and in the interests of making things 'uniform', we were forced to adopt the purchasing company's vacation policy, which meant an across the board cut in everyone's PTO hours (to include folks that negotiated additional vacation days when they were hired -- those agreements were declared null and void). Apparently the execs thought the hit to morale/retention was worth the 'cost savings'.

On the other hand, I once interviewed at a company with an 'open' vacation policy and was immediately suspicious of it. My first thought was "I'm guessing people don't take much vacation?" That didn't go over so well with the interviewer.

It's nice to see an employer that treats vacation as a means to increase employee productivity and retention (burnout prevention, etc.), rather than as a pure cost to be minimized as much as possible.

4
cellover 13 hours ago 0 replies      
I really like the tone of this post. It makes me think that this person is trustworthy and I'd be glad to work for him.

I wish my company had such an open communication!

5
danielweber 13 hours ago 1 reply      
> A company has to learn how to function when people are on vacation and unavailable, however important their role is.

Yes, this is essential for having a viable company. If the company grinds to a halt without person X, the solution is not to get rid of person X's vacation.

Also, in finance, required vacations are often a compliance aid. If someone is running a scam, it tends to need near-daily tending, and it will explode when they are out for two weeks (unless they find a co-conspirator).

6
KennyCason 8 hours ago 0 replies      
I see a lot of people saying that they would "never work for a company with unlimited vacation." I also understand their reasoning and have seen similar examples.

At DataRank we balance this with a small amendment. It's unlimited vacation but you MUST take a minimum of two weeks off. We are also very flexible about remote work. If you feel the itch to travel to Japan for a 2 week vacation + 2 weeks working remote, go for it. If you just feel burned out and want to watch Netflix at home all day. Go for it, just let someone know.

7
tdumitrescu 12 hours ago 1 reply      
I last moved from a shop with unlimited PTO (where in practice people settled around 4-5 weeks per year) to one with 15 days per year. Quality of life is noticeably worse with the limited PTO, and people tend to hoard their days because they're scarce. Travis's tracked minimum vacation sounds better than either of those policies.
8
superuser2 5 hours ago 1 reply      
Mandatory vacation is not only a good idea, it is a security feature.

When one employee takes over another's "world" within the company, it is very likely that they will notice if the employee on vacation has been embezzling, defrauding customers, etc. In the financial industry, this is not only good practice, it is a regulatory requirement. People who do not want to take vacation are seen by auditors as highly suspicious because that behavior often indicates a desire to cover up fraud.

9
asafira 12 hours ago 1 reply      
As a graduate student, I feel that (at least within my research group) there is this "unlimited time off" policy, but in the end it's not really taken advantage of very much and it certainly doesn't take any stress away from work. Even this holiday season, I'll be taking a week off from work, and I always phrase it as "I promised my mother I'd be home for at least a week".

The minimum vacation policy sounds like an interesting alternative, but I'm not sure it could fit within the academic community. Any professor has the power to instate such a policy, but at any given moment there's an "important" project that rests on one or two people working hard to get results. Often, in academia, you have those people that are precisely the kind that will work hard and get shit done, and honestly I can't imagine there being an understanding of "minimum time off". It's more convenient to the professor and the ambitious PhD student/post-doc to have the "take time off when you want" policy, and let the unwritten rules and undertones dictate when you should take time off (e.g., when you just finish a project).

I know the article wasn't trying to apply things to academia, I just thought I would consider it in that light.

10
ngmaloney 13 hours ago 1 reply      
I jokingly refer to Unlimited Vacation Policy as Guilt Driven Vacation Policy. IMHO, it is a sham so companies don't have to take liability for accrued vacation time on their books. While I commend Travis for coming up with a workable solution wouldn't it be easier to just say: "Everyone has 4 weeks vacation, use them!".
11
padobson 14 hours ago 4 replies      
If I worked somewhere that had an unlimited vacation policy, I think I'd probably stop working on Fridays all together. An unlimited vacation policy could be an open invitation for a 4-day work-week (or 3 day or 2 day...)
12
foolinaround 14 hours ago 1 reply      
What I found to be even more relaxing is if all the employees of a company are forced to take the same time off ( 2 days to a week) and other flexible time off where the employee can choose his own time.

Then, there are no email or updates to come back to...

13
acd 13 hours ago 0 replies      
You can also view it likes this.

In the end of your life when looking back at it. You can think of asking two questions of yourself.

Did you wish you worked more hard days for your company? orDid you wish you spent more time with family and gaining new experiences?

Obviously you will need to find a work / life balance but if you only work/play hard you might regret it later and you cannot get lost time back.

14
qeorge 13 hours ago 1 reply      
Another upside is forcing the company to learn to run without its founders or other key employees for a meaningful period of time.

This forces good processes to be put in place, and is an excellent fire drill in general.

15
javery 14 hours ago 7 replies      
We have an unlimited vacation policy at Adzerk and I try to lead by example by taking at least 2-3 weeks off a year (especially in the summer). We also try to encourage everyone to take at least one solid week off a year and it seems to work pretty well.

I think that's the real issue with unlimited policy - if the leaders don't take vacation no one else thinks they can.

16
chton 13 hours ago 3 replies      
"We tried something that didn't fit our culture, and it didn't work."

Taking less vacation time is not necessarily a bad thing. For myself, not having to meet a minimum quota means I enjoy the vacations I do take more, since I take them when I need and want instead of mandatory ones. It means, as long as I'm feeling good, I can keep the momentum up.

The bigger problems is that employees didn't respect others' vacation time. Or their own, for that matter. That is a cultural problem, and a change in vacation policy won't help (much). If you want the people in your company to be happier with their time off, work on respecting it.

  "The guilt of taking time off takes over, and you "just check in" or promise to be available if anything comes up."
That is, to me, a sign of an unhealthy culture. Fix that before trying yet another vacation experiment.

17
butwhy 14 hours ago 10 replies      
Why do companies even try out these weird policies? They've gone from having unlimited days (then admitted it was a mistake for 2 whole years) and are now blindly trying out another unusual strategy. I don't see why they don't just offer a standard solution which seems to work for most companies, instead of trying to be extremely controlling of people.
18
dyadic 7 hours ago 0 replies      
There are a few people here commenting about how their employer's unlimited vacation policy allows them to take two weeks vacation and then work remotely for a week or two extra. That's great, and it's good to see companies allowing it but remote work is still work, and it certainly isn't a vacation.

Many people are working remote by default* and already have to fight certain perceptions that they are just sitting in their pants, watching tv, and eating chips all day, and it certainly doesn't help when others liken working remotely to being a holiday.

It's definitely a good thing that more companies are opening up to remote work and trusting their people to get things done without being watched over, let's not perpetuate this idea of it just being a bunch of slackers that do it.

* including me, obviously, and hence this response.

19
eyeareque 12 hours ago 0 replies      
My job went from 20 days paid off per year where we accrued hours per pay period, to an open vacation policy. The company did this because of the cost of having to keep money saved up to pay out to employees who have saved up their vacation hours.

Like the article describes, most people including myself took off less time. I know for myself I felt guilty and didn't want to be seen as taking advantage of the new open policy.

I'm going to talk to my new job's management and ask them to setup a minimum vacation day policy.

20
dalke 13 hours ago 0 replies      
If anyone wants to read more about unlimited vacation policies, here are some of the previous HN threads on the topic: https://news.ycombinator.com/item?id=5125973 , https://news.ycombinator.com/item?id=4874743 , and https://news.ycombinator.com/item?id=7613526 . The last contains comments about the Jacob Kaplan-Moss essay that was mentioned in this thread's link.
21
my_username_is_ 13 hours ago 0 replies      
I'm glad to see someone trying this out. It's something I've been wondering about [1], and it seems pretty uncommon. I'll be looking forward to seeing how this works out for them, and any other companies that are so inspired to try to improve the status-quo.

Frankly the American culture of working continuously seems like a negative to employee and employer both--and even if it were only a negative to one, it would be something worthy of change.

[1] https://news.ycombinator.com/item?id=8053580

22
sqs 12 hours ago 0 replies      
This is a great post. At Sourcegraph, we had a similar realization, based on our own experiences.

We wrote a blog post about it and it got picked up by the BBC:

https://sourcegraph.com/blog/mandatory-vacationhttp://www.bbc.com/capital/story/20140903-relax-or-else

23
genghisjahn 13 hours ago 0 replies      
The company I work for has open vacation and it works out fine. People take a few days, a week here and there and it winds up being 4-5 weeks a year. This is in the US. I'm sure there's no big payout for unused vacation when people leave, but no big deal. When you're sick, you stay home, when you need time off, you take it. The CTO and upper managers are all coders and have worked hard at maintaining a good work culture.
24
spir 14 hours ago 0 replies      
It's like all these high-powered technology executives never opened an economics textbook.
25
jbb555 14 hours ago 3 replies      
very good.Except "now has a required minimum of 25 (paid) vacation days per year, no matter what country they live in." Well no. If they live in the UK for example they have a mandatory limit of 28 days not 25 because it's the law.
26
legohead 12 hours ago 1 reply      
So here's a slightly different view from someone I know -- they are slotted X number of vacation days, but actually need more.

He is divorced, and his kid is across the US, so he ends up using all his vacation to see his kid. Now he is left with nothing to take a real vacation with his current spouse, and they both really want to.

27
larsberg 14 hours ago 0 replies      
I definitely saw the "being totally unplugged" problem both previously when I worked at Microsoft and now when working at Mozilla. It's arguably even worse at Mozilla with the shift of many people (myself included) to IRCCloud, which is a wonderful service but turns what was supposed to be an asynchronous, loose availability chat system into a realtime, all-hours interruption mechanism.
28
asgard1024 13 hours ago 1 reply      
I would go even further and postulate that lack of rules about who is in power kills societies. Many equate "lack of rules" with "freedom", but it's a problem. People without conscience will freely abuse the lack of rules or start to fight with each other, while the good people will get screwed. Eventually, the good people, who usually just want to do the work, will get unhappy and leave.

This applies not only to vacation, but to governance as well. So called "consensual democracy", where there is no hard and fast rule about who has how much power, has the same kind of problem. The most important thing about democracy is that it is very specific - everybody has the same amount of power. That discourages political fights, because you cannot (without explicitly changing the rule) accrue more power. In my opinion, many institutions (for example Wikipedia) suffer from this problem.

29
turnip1979 14 hours ago 7 replies      
Genuine question: how do companies/teams that run services deal with peak vacation periods? E.g. Christmas holidays. Even if everyone wants to take the last two weeks of the year off, you have to have someone keeping the lights on. Also, what happens if a high priority problem comes in? Are the team leads/managers careful not to schedule work around the holidays?
30
evanpw 11 hours ago 3 replies      
I don't think I see a single negative comment here about long vacations, so allow me to play devil's advocate. (These questions don't necessarily reflect my own opinions.)

1. If this is such a free lunch (employees are both happier and more productive overall), then why isn't lots of vacation offered everywhere (in the US)? Can it really be the case that the vast majority of US companies are too short-sighted to make a policy change that would be in their own best interest?

2. If, alternatively, more vacation is really good for employees, but slightly bad for employers, why doesn't the labor market adjust? In this case, you would expect employees to offer to work for a lower salary in exchange for more vacation time.

31
whoisthemachine 14 hours ago 0 replies      
Interesting take on paid time off that I've never seen before. Will be looking forward to the update on how this works out.
32
thoman23 8 hours ago 0 replies      
The only thing surprising to me about this is to hear that this founder apparently had noble intentions at the start. I just assumed that every company that does this is cynically inducing more (short-term) work out of their employees while clearing some liabilities from the books, all at the cost of long-term success and morale. I would never work for a company with one of these "unlimited" vacation policies.
33
gadders 13 hours ago 1 reply      
As a counterpoint, most investment banks enfore a mandatory 10 working day break where all remote access capabilities are disabled (including blackberries).

This is less out of altruism, and more so that any untoward activities can be uncovered whilst the person is out of the office.

34
mrottenkolber 13 hours ago 0 replies      
Just be a freelancer, every day will be a holiday and christmas is just just another hackaton. ;p
35
mariusc23 13 hours ago 0 replies      
Glad to see a new approach on the open vacation policy. I think this policy would work better if vacation days are only tracked until the minimum is met, otherwise I have a feeling it will be perceived as exactly 25 days, not more.
36
hgh 12 hours ago 0 replies      
".... and you "just check in" or promise to be available if anything comes up. You respond to just one email or just one GitHub issue."

When people need to be "available" while on vacation it usually means that there is way too much dependency on that person or insufficient knowledge-sharing. It shows the glaring holes on what's being hacked together and where systems can strengthen.

37
hero454545 11 hours ago 0 replies      
Nice post... I especially liked the touch of posting a link to their hiring page right after describing their mandatory 25 paid vacation day policy.
38
fishnchips 12 hours ago 0 replies      
In all fairness "open vacation" policy is a yet another iteration on the age-old idea of Panopticon ;)
39
stevenkovar 13 hours ago 0 replies      
I can see the merit for enforcing more vacation; but is more vacation the solution to burnout, or is working less overall?
41
zakvyn 12 hours ago 0 replies      
"Minimum" vacation gives employee the authority to take vacation without feeling guilt. Else some companies use open vacation policy to reduce the number of vacations that employee take.
42
zakvyn 12 hours ago 0 replies      
I like the idea 'minimum' vacation that allow people able to take more than 'minimum' vacation.
43
pierotofy 11 hours ago 0 replies      
Society needs more free time in general. http://t.co/68fyM3950D
44
gutsy 12 hours ago 0 replies      
This is phenomenal.
45
rebootthesystem 11 hours ago 0 replies      
I'd like to see the inclusion of data related to the structure and dynamics of the family unit. In the US the family desintegrates as kids are encouraged to leave home as soon as they finish high scool, say, 18 years of age.

This, in a huge number of cultures and countries around the world is an unthinkable attrocity. It isn't uncommon to have children live with their parents until they are ready to form their own families, say, 25 to 30 years of age. Under the stereotypical US culture this is looked down upon to the point of making fun of those still living with their parents.

Having millions of socially unprepared kids go off to try to make it on their own created, in my opinion, a society filled with various levels of dysfunctionalities. People tend to grow up, to some degree, in a "wild" setting where selfishness can become a necessity. In this context it is easy to see how work time can take the place of the social context lost due to having left the nest.

Having grown up and lived in three cultures it is easy for me to remove myself from my US culture and watch it from afar as a visitor from another planet might. Americans are said to seem socially inept and dry from the perspective of other cultures, and this is true. All you have to do is spend six months in Italy or Argentina to undrerstand this. Americans men develop a weird homophobic form of "macho" that is down-right funny from the perspective of other cultures. Hugging, or worst, kissing, another man is frowned upon. Human contact, in general, is just not considered to be "normal". I have lived in cultures where it was quite normal to greet your kids parents with hugs and kisses on the cheeks. In fact, it would be rude to come to a party or gathering and not go around the room kissing everyone, man, woman, kids. Yes, you kiss your friend's wives. School friends do this in high school when they greet each other. Again, unthinkable in the US. In fact, a teacher can get in trouble with the law for greeting a student with a hug and a kiss. Showing affection is alien. Weird.

In general terms, as much as TV shows and commercial try to stereotype this warm southern cowboy culture at the ranch with grandma this is, for the mist part, not the norm. The US family scatters and the kids are left to navigate a very important phase of their lives on their own.

Sorry to harp on this but i do think this is a very important part of the equation and one that explains so much about US society, their behaviors, beliefs, relationships, work and family life. Once you realize that some of these people become politicians that shape US policies and laws it is easy to see where some of our problems might come from.

There are subtle examples of this. For example, let's say you are working with a US friend on a project in their garage. He will use phrases like "give me my hammer" or "it's in my toolbox". In other cultures this becomes "give me the hammer" and "it's in the toolbox". I am convinced this egocentric view of the world is connected to leaving the nest early.

Another example that is particularly bothersome to me are cases where parents pay their children for things that in what I am going to call more socially adjusted societies is simply unthinkable. One of my friends pays his 18 year old kid to go pick him up at the airport. Another pays his kids to help paint the house. My neighbor across the street pays his kid to mow the lawn. Viewed from far more family-centric cultures these woukd be examples of seriously dysfunctional family units. As a teenager I helped my parents in their business. Money was never a part of it. This is simply how a family behaves in other cultures.

Of course I am painting with a wide brush. There are lots of cases of families that behave very differently from this. And, of course, the US has lots of multicultural families, such as mine. Yet I still think that a huge portion of the attitude towards work has to do with a bunch of single people existing "in the wild" and a setting where work can easily become a substitute for the family unit they effectively lost.

22
Wouldn't it be fun to build your own Google?
96 points by martinkl  14 hours ago   41 comments top 13
1
Smerity 8 hours ago 4 replies      
[lightly modified version of a comment I put on the article as I love HN for discussion!]

Great article -- we're excited there's so much interest in the web as a dataset! I'm part of the team at Common Crawl and thought I'd clarify some points in the article.

The most important is that you can download all the data that Common Crawl provides completely for free, without the need to pay S3 transfer fees or process it only in an EC2 cluster. You don't even need to have an Amazon account! Our crawl archive blog posts give full details for downloading[1]. The main challenge then is storing it, as the full dataset is really quite large, but a number of universities have pulled down a significant portion onto their local clusters.

Also, we're performing the crawl once a month now. The monthly crawl archives are between 35-70 terabytes compressed. As such, we've actually crawled and stored over a quarter petabyte compressed, or 1.3 petabytes uncompressed, so far in 2014. (The archives go back to 2008.)

Comparing directly against the Internet Archive datasets is a bit like comparing apples to oranges. They store images and other types of binary content as well, whilst Common Crawl aims primarily for HTML, which compresses better. Also, the numbers used for Internet Archive were for all of the crawls they've done, and in our case the numbers were for a single month's crawl.

We're excited to see Martin use one of our crawl archives in his work -- seeing these experiments come to life the best part of working at Common Crawl! I can confirm that optimizations will help you lower that EC2 figure. We can process a fairly intensive MR job over a standard crawl archive in afternoon for about $30. Big data on a small budget is a top priority for us!

[1]: http://blog.commoncrawl.org/2014/11/october-2014-crawl-archi...

2
mark_l_watson 11 hours ago 0 replies      
A really nice idea.

I volunteered a bit early this year for Common Crawl (not much, just some Java and Clojure examples for fetching and using the new archive format).

Common Crawl already has many volunteers (and a professional management and technical staff) so it would seem like a good idea to merge some of the author's goals with the existing Common Crawl organization. Perhaps more frequent Common Crawl web fetches and also making the data available on Azure, Google Cloud, etc. would satisfy the author's desire to have more immediacy and have the data available from multiple sources.

3
smoyer 3 hours ago 0 replies      
A couple thoughts:

1) I like the idea of human curation, but in combination with some sort of automated crawler (or other tool) that helps in the browser.

2) Why can't we also distribute the act of crawling, the maintenance of the index and the map-reduce (or other algorithm) that produces the data.

I've been thinking about architectures that would allow (in essence) a P2P search system. Would anyone be interested in talking about architectures to make this work? There are millions of computers on the web at any given time ... if it's built into the browser (or plugs in), you could have human input at the same time.

4
JDDunn9 8 hours ago 0 replies      
I've always wanted to experiment with my own search algorithm. Unfortunately, I think this is still out of the budget of average programmers. Just the hard drives to download 1.3 petabytes would cost six-figures.[1][2]

[1] https://www.backblaze.com/petabytes-on-a-budget-how-to-build...

[2] https://www.backblaze.com/blog/why-now-is-the-time-for-backb...

5
andrewhillman 9 hours ago 3 replies      
Yeah, this sounds all well and good in theory, but after visiting thousands of sites over the years, it might be a better idea to help engineers build a search engine for their own site/data first. I can't recall many websites that have amazing search. It's a problem when I have to use google to find what I want on xyz.com because if I go search for what I am looking for on xyz.com I cannot find it even if I know its on that site.

It would be so nice to go to xyz.com and actually find what I am looking for in under 1 second.

6
discardorama 7 hours ago 2 replies      
Google's power comes not from the crawling, but from the retrieval and ranking. They use many more signals than the hyperlinks and anchor text (which is all you'd have if you crawled yourself). Indexing crawled content would have been OK in the year 2000; but today, the users demand more. Relevance is the top priority, and no one does it better than El Goog.
7
sparkzilla 5 hours ago 1 reply      
The problem with algorithmic/scraper search methods, is that they only work with existing data. For example, most Google searches gives a list of websites on one side, and some data scraped from Wikipedia on the other. There is not much meaning there. That's because Google's algorithm cannot combine the results into something original, because that would require human creativity. As such, I see the rise of different kinds of search based on what humans create, rather than what computers can scrape. I wrote a (longish) blog post on this problem: http://newslines.org/blog/googles-black-hole/
8
ryanthejuggler 13 hours ago 0 replies      
This would be really cool to participate in, especially if it could be packaged la Folding@Home/SETI@Home and widely distributed. I wonder if there's some clever method using crypto that can provably discourage bad actors if the network has certain properties (e.g. Bitcoin is nearly impossible to cheat unless one group owns >50% of the network).
9
dmritard96 7 hours ago 0 replies      
suprised not to see a mention of a bloom filter in url dedupe. Another tough problem now is the portion of the web in walled gardens or that is expensive to crawl (needs a js context).
10
mjklin 9 hours ago 2 replies      
I thought Wikimedia tried this once. Big announcement, then nothing. Is that code still available?
11
angersock 8 hours ago 1 reply      
For anyone interested, there's a hilariously bitter and practical paper on the trials and tribulations of building a search engine:

http://queue.acm.org/detail.cfm?id=988407

EDIT:

Article is clearly from an earlier era, but it's really cool to see how far we've come and how much more computing power we have available now. There are entire categories of problems that simply don't exist anymore.

12
imranq 8 hours ago 0 replies      
what about Algolia? HN uses it
13
pjbrunet 39 minutes ago 0 replies      
At first Google was a search algorithm, but at some point they decided to have humans review and rank the important queries. Important as in query volume.

Why use humans? People can decide if your navigation is intuitive. They can decide if your page looks like crap. If 230,000 people are searching for "coconut oil" per month (actual numbers) then it's worth having an intern spend 15 minutes to make sure page 1 of "coconut oil" looks right.

Google can afford that. They need a human to decide if the "user experience" is actually good vs. disallowing the back button and forcing the browser to crash, which is how I suppose you could fake a "time on site" metric if this was just an algorithmic problem.

Google is now more like playing Zork. You type "Go North" like 10 million other people before you typed "Go North" and Google has already crafted that experience you'll find in next room. (Which makes me wonder, do they score how boring you are based on predictability?) This is becoming more and more obvious over time as a search for "calculator" shows you an actual calculator that a human at Google created. That's not an algorithmic response.

Similarly, I see that human touch coming more into play with voice recognition, Google Glass, Siri, etc. Call that "AI" or whatever. You ask Google a question and Google has already sculpted a slick answer based on tons of testing. That's how I see Google as a search engine now. Part of the crawling is interesting (recognizing objects in photos?) but I think human reviews of all the important websites and SERPs, that's harder for a competitor to reproduce.

23
Differences Among IEEE 754 Implementations
9 points by xvirk  4 hours ago   discuss
24
Techstars to open up shop in Detroit
137 points by nwest  11 hours ago   88 comments top 14
1
valarauca1 11 hours ago 2 replies      
As a Detroiter hopefully this will give me a reason to start going downtown.

The current problem with the Detroit Metro area is when ever outside money comes in. Most the time the offices open in the northern/western suburbs.

I say this because most high paying jobs are in the suburbs. Because that's where the businesses are. The suburbs are doing well. At first I was very excited and hoped they'd open relatively close, but all things consider that wouldn't help the city as much.

2
ryanmarsh 9 hours ago 8 replies      
As a Houstonion this gives me pause. Houston is a massive economic center by nearly every measure. It is also home to the most diverse county and university in the US. There are large successful technology companies here. Yet, Detroit and Dallas (arch enemy of Houston) are getting TechStars before us.

Part of the reason we don't attract accelerators like TechStars is the lack of a healthy startup ecosystem. I am NOT saying that the well intentioned community organizers aren't working hard. It's just that most of the VC here goes to Oil & Gas projects. If your tech isn't energy related it won't hit the radar. The technology startup successes I've seen in my 30 or so years here were built inspite of a (nonexistent) local startup ecosystem. Lastly, 9 out of 10 great people I've worked with here have left to go places with more tech opportunity. The talent is voting with its feet. Frankly, I'm done too. I'm on the first airplane out of here once I get my feet under me. I'm sad that it has to be this way. Maybe if I make it rich I'll come back and invest.

edit: spelling

3
phkahler 11 hours ago 1 reply      
While this is great news, I think YC-Detroit would have been even better ;-)

There is a lot of talent here. As evidence, here's the First Robotics Competition team map:https://www.google.com/maps/d/viewer?mid=zIJNq656OeBY.ke6fB8...

Our teams also have a long history of winning ;-)

4
alexalex 5 hours ago 1 reply      
If true and successful, this would be huge for Detroit.

I once looked into the suitability of Detroit for a start-up. It seemed that professional services were generally not as interested in working with young companies as in the bay area. The availability of deferred legal, accounting, or other services was non-existent. Being in the bay area, my company has benefited greatly from having deferred legal work. I also worried about finding good advisers.

With Techstars there, it would definitely solve these problems for their awardees.

While there aren't a wealth of affluent, urban customers in the city, there would be great opportunities for consumer or educational startups that address problems for the base of the pyramid. Whole Foods opened a store in Detroit, and talked more about it in their last quarterly conference call than any other topic. Ostensibly, figuring out how to be successful in markets like Detroit is a huge opportunity for growth for them. Finally, the city of Detroit has neighborhoods that are young and affluent. Inventory is very limited, and rents in the highest profile areas are as high as most big cities -- think Lake Merritt in Oakland.

I really hope TechStars is successful there. Detroit is a place dear in my heart and not as scary as many think. I lived in SF's SOMA 10-ish years ago, and can say that I had way crazier stories and more dangerous moments than my mom, who has been working in Detroit Public Schools since her retirement.

5
ThomPete 10 hours ago 1 reply      
I have said it before and I will say it again.

Detroit is the perfect place to build the robotics industry and they should call it Uncanny Vally.

6
drawkbox 2 hours ago 0 replies      
Happy for Detroit, I think there is a responsibility of tech leaders to get this throughout our nation not concentrate it, no single point of failure, plus more input from different areas means better products + systems which are more robust.

However, when is someone going to setup in Phoenix (Chandler, Scottsdale, Tempe)? Huge market, short flight to CA, always overlooked. Gangplank (web/app/product), Game Co-lab (games) co-working locations + ASU are doing a good job for startups but it would be awesome for more investment here, fairly untapped and a blue ocean in the desert where we have to stay indoors working most of the time anyways.

7
bruceb 7 hours ago 0 replies      
There is already at least one accelerator in Detroit.http://bizdom.com/ It is partially funded by the Dan Gilbert, founder of Quicken Loans. He also owns the Cleveland Cavilers and partial funds same thing in Cleveland.

Detroit is cheap compared to SF but actually downtown is filling up, rents are not as cheap as you might think.

8
youssifa 5 hours ago 0 replies      
If you're a startup founder accepted into Detroit Tech Stars (or even a YC) who lives in a foreign country, there should be a policy in place to give you an accelerated visa to work / live in the US provided you base your startup out of Detroit. Could be a good policy experiment on urban renewal.
9
marksc 10 hours ago 1 reply      
This is great news!

Though, I'm surprised they aren't coming to Ann Arbor- the real startup hub of Michigan.

10
borat4prez 11 hours ago 1 reply      
How do I apply my Detroit area startup? The application on the website is only for Austin.
11
azilnik 9 hours ago 0 replies      
Does anyone else feel like Detroit is going to be an attractive place for young tech folks to start their businesses in the next 10 years?

Are there any Detroit-centric REITs out there?

12
iwantagrinder 9 hours ago 0 replies      
Gilbert's plan is working.
13
monsterix 11 hours ago 4 replies      
I wish there was something like this here in DC. Very little startup scene. All I get to see are those sweettalk recruitment shops and those three lettered agencies that I couldn't imagine working for.

Good news for you Detroit!

14
johnvschmitt 10 hours ago 1 reply      
Great news & I have more hope for the community.

However, many people outside of the startup scene have misconceptions, as they see survivor bias/success bias in the reporting. One item in the report feeds this misconception: "The average company got $1.9 million in funding." The median raised would be far more revealing of the actual experience that a startup faces than the average.

25
Mysterious 2008 Turkey Pipeline Blast Opened New Cyberwar Era
49 points by happyscrappy  10 hours ago   14 comments top 6
1
rdtsc 3 hours ago 1 reply      
Well in that case the New Cyberwar Era started in 1982 not 2008:

https://en.wikipedia.org/wiki/Siberian_pipeline_sabotage

2
usbreply 2 hours ago 0 replies      
How convenient that this article comes out just when people are discussing the US/British Regin malware.

https://firstlook.org/theintercept/2014/11/24/secret-regin-m...

3
nosuchthing 6 hours ago 2 replies      
It feels like there's been a weird lukewarm warm war going on, mostly via economic malice.
4
louwrentius 5 hours ago 0 replies      
As investigators followed the trail of the failed alarm system, they found the hackers point of entry was an unexpected one: the surveillance cameras themselves.

That's not unexpected for anyone even vaguely familiar with security.

This is not about there cannot be security with physical access. This is about not understanding the possible risk scenario's and mitigating them.

Please note that the US once fed malicious software to USSR spies and this caused a Russian pipeline to blowup. http://www.zdnet.com/article/us-software-blew-up-russian-gas...

5
revelation 6 hours ago 0 replies      
Well, instead of their laptops the two people seen near the pipeline segment could have just brought explosives. That would probably have been just as effective. I guess they just didn't want to bother digging it up.

The notion of any security goes out of the window pretty fast once theres physical access. That goes doubly when you realize that the same Windows 98 running your ATM is controlling pipelines and other safety relevant systems and some middle manager decided to replace local personnel with a serial-port-over-TCP-IP directly connected to the logic controller in charge.

6
sp332 6 hours ago 1 reply      
The only sources for this information are four unnamed people?
26
A DIY Astro Tracker in Two Nights
42 points by skazka16  9 hours ago   6 comments top 4
1
theophrastus 4 minutes ago 0 replies      
Astro-know-nothing question: why wouldn't it be more directly 'tracking' to forget the constant clock drive (which we know will always be off by some amount) and set up a suitably long sight tube on some bright star with a photo sensor at the bottom of it. When the star's light moves out of the tube, a fine-stepped stepper tracks west until it re-appears ..?
2
jhallenworld 2 hours ago 0 replies      
The overly high tech methods used in the project amuse me: design it with a CAD system, use a laser cutter to cut the wood, design your own gears, write C program for microcontroller to driver stepper motor- (advantage: easily adjust for astrophotography from other planets). It's too much meta-work.

So here is my new challenge: make it using only parts and tools available from Home Depot.

(even so, I have to get one of these laser cutters..)

3
Florin_Andrei 3 hours ago 1 reply      
There are many projects like this, but a lot of them are done wrong: they use a straight main bolt and they don't compensate for the error in the driver motor.

This one is done right - the main bolt is curved, so no error (or very little error) is introduced while tracking. Good job.

4
codezero 4 hours ago 0 replies      
Can't wait to see the results of this, good job!
27
Prime Gap Grows After Decades-Long Lull
62 points by digital55  9 hours ago   23 comments top 3
1
typomatic 8 hours ago 3 replies      
I laughed out loud when I saw that formula--y = loglogloglog x is a nonsense function. For those of you who haven't thought about logs for a while, that function is increasing (bigger x give bigger y), but it grows so slowly that y won't be larger than 1 until x is larger than 2.33 x 10^1656520.
2
lkbm 9 hours ago 3 replies      
> This is the first Erds prize problem Tao has been able to solve, he said. So thats kind of cool.

There are only fifteen, right? Possibly even fewer open ones. It'd be pretty impressive if he/they manage to solve multiple Erds prize problems.

3
darkstar999 5 hours ago 3 replies      
What is the significance of studying this problem? I mean, say they prove the twin primes conjecture. What does that mean? Do we benefit like we would if the travelling salesman problem was solved?
28
Postmates, Powering On-Demand Logistics
32 points by prostoalex  8 hours ago   4 comments top 3
1
walterbell 4 hours ago 0 replies      
Good ad copy.

Needs a large button at bottom of post which says "Learn about API" or similar, rather than a small hyperlink on "here" in the middle of the post.

2
clay_to_n 2 hours ago 0 replies      
Really cool, surprised there isn't more discussion about this. Postmates seems like a great service to exist as an API - I predict many hackathon projects using this in the future.
3
bastian 3 hours ago 1 reply      
Good idea. We will fix asap.
30
Speeding Up PostgreSQL with Partial Indexes
85 points by drob  11 hours ago   25 comments top 4
1
chdir 58 minutes ago 0 replies      
In case of Django, the RunSQL command [1] in migrations makes it very easy to use partial indexes. There are a couple of examples on Stackoverflow. Definitely a low hanging fruit and worth those 10-20 minutes to setup.

[1] https://docs.djangoproject.com/en/dev/ref/migration-operatio...

2
infogulch 10 hours ago 2 replies      
What about one multiple-column index? Using three separate indexes on three columns is almost never ideal so I don't know why it was considered in the first place (unless to make a contrived "normal" to compare partial indexes against).

My guess would be: nearly the same read performance as the partial index, a bit less than half the storage of three separate indexes. The write performance isn't as "free" as the partial index, but it's a third of three separate indexes and still allows for many more types of queries than the partial index.

3
losvedir 7 hours ago 4 replies      
This is a cool feature.

Unfortunately, when I looked into this for our codebase, I found that it doesn't have full support in Rails 3. You can make the index, via a migration, but rails won't include it properly in your `schema.rb` file. So restoring via schema (as opposed to running through all your migrations from scratch) or just trying to understand your database by looking at schema.rb won't give you a full picture.

However, looking at `add_index` in rails 4, it seems to support it!

edit: one thing I did try out was you can change your schema file from `schema.rb` to `schema.sql`, which supposedly fixes it, but I had some issues with that which I don't remember at the moment.

4
dsugarman 9 hours ago 4 replies      
this is a really cool feature I did not know about. just curious, why do you use such crazy json in postgres? why not store the data with relational database standards?
       cached 11 December 2014 05:02:03 GMT