hacker news with inline top comments    .. more ..    18 Jan 2014 Best
home   ask   best   5 years ago   
1
OpenBSD will shut down if we do not have the funding to keep the lights on marc.info
762 points by openbsddesktop  1 day ago   372 comments top 45
1
jxf 1 day ago 10 replies      
Just to make the call to action a little more direct, the donation link is here:

http://www.openbsdfoundation.org/donations.html

2
jdludlow 1 day ago 10 replies      

  > The OpenBSD project uses a lot of electricity for running the  > development and build machines.  A number of logistical reasons  > prevents us from moving the machines to another location which might  > offer space/power for free, so let's not allow the conversation to go  > that way.
I don't understand this comment. If the choice came down to moving versus shutting down entirely, why is moving an unacceptable answer?

3
4ad 1 day ago 4 replies      
4
PaulRobinson 1 day ago 11 replies      
So let's deal with the elephant in the room: the OpenBSD project is run by complete and utter jerks. Not just Theo, but he has set the bar quite low when it comes to friendliness and tolerance of questions from younger/less experienced contributors. Linus' rants on the Linux kernel lists are almost cookie cutter copies of Theo's.

There is "opinionated software" and then there is Theo being an intolerable, obnoxious, ego-maniac.

As such many people are going to see this and laugh and think "good riddance", and will be happy to see OpenBSD disappear.

That will only be enhanced by the fact the books are closed, the shortfall on the electric bill is inexplicably $20k, and nobody is prepared to explain the detail.

In essence rudeness + shady accounting practice != open source community that should feel a sense of entitlement from non-core users

It's a shame because the code (especially the crypto code) is really good. Seriously, go read it: I used to love reading the OpenBSD source, but I never contributed anything because Theo was such an absolute jerk.

I hope the guys who work on the crypto stuff at least either keep doing so elsewhere (Free- or Net-), or a new project without the need for $20k in electricity bills spins up to keep going.

5
orbitur 1 day ago 2 replies      
I feel like it's useful to point that OpenBSD won't say exactly why the bill is so high, and apparently they don't have open accounting. That bothers me a bit.

Link to relevant reddit comment thread: http://www.reddit.com/r/linux/comments/1vakm9/openbsd_develo...

6
justin66 1 day ago 1 reply      
I get that a lot of people don't care. I even get that some people have an unrealistic expectation that they should know how every dime of their donation is going to be accounted for, an expectation that must either not extend to their other charitable donations or serve as an indicator that they don't give a lot of money to charities at all.

What I don't get is where people - presumably people involved in information technology - conclude that $20k/year is a lot of money to spend on electricity. What utopia of free electricity for all are these people living in? Can I host a few servers there?

7
blhack 1 day ago 1 reply      
They accept bitcoins, here is their donation page, and they're the people that maintain openSSH, the software that I'd bet ALL of us use every day:

http://www.openbsdfoundation.org/donations.html

8
simias 1 day ago 6 replies      
As a long time FreeBSD user I wish the BSDs would find a way to "unite" in a way, try to put redundant infrastructures in common.

Right now I feel like linux is slowly eating all the market share, if it continues that way the BSDs will regress back to the lines of Hurd and Plan9.

Competition is always a good thing, even in the OSS world.

9
pyvpx 1 day ago 0 replies      
all I want to say on this (to me, tiresome) topic is: if you or your company (especially if that is one and the same!!!!1) have ever used OpenSSH you should at the ABSOLUTE MINIMUM buy a damn CD. you really ought to donate a solid hundred in your local currency but if you can't, skip a couple pints this month and buy a CD.

if you use CARP or OpenBGPd and have never bought a CD, you need to buy two ;p

10
xradionut 1 day ago 5 replies      
They could probably kill support and power for some of the "dead silicon" platforms they support. If the CPU hasn't been manufactured in the last decade or two, why support it?
11
fidotron 1 day ago 1 reply      
As I commented when this appeared here the other day as: https://news.ycombinator.com/item?id=7059581

The fact this was submitted here and disappeared is kind of indicative of their problems.

12
ajkjk 1 day ago 1 reply      
Can someone explain why I would want to support OpenBSD?

I get, from the comments, that: openSSH is great (sure, I can believe it) and Theo is an asshole but you should compromise and give them money anyway.

I don't get why a parallel operating system ought to continue existing, especially one that comes from the same legacy codebase as several others. I also don't get why, if openSSH is so good, the goal isn't to write a new SSH or fund openSSH separately and drop the rest that can't be supported. Why does the world needs OpenBSD instead of having the talent focused on making a smaller number (..Linux) better.

Certainly the answer might be 'because that's what the talent wants to work on' but that's not a good reason to fund it.

I have also gotten the impression, from limited observation, that there are real problems in the open source community with too many people doing the same thing and not working together, and with people being jerks (or just generally anti-social or at least not-highly-personable) and making it hard to get involved or care about otherwise important projects. I don't think supporting Theo, from what I've read here, helps that trend.

I know very little. Can someone fill me in?

13
SwellJoe 1 day ago 1 reply      
If the price of OpenSSH is keeping all of OpenBSD running, I'll send OpenBSD some money.

I've never used OpenBSD except to tinker with it, and likely never will. But, I use OpenSSH hundreds of times a day (both in automated jobs and in the terminal). It is utterly necessary to me; likely necessary for all of us, really. So, while a company would have shed the extra weight and focused on its core product that users love years ago, OpenBSD keeps OpenSSH as a side effect. I guess I'm OK with that.

Maybe it's even necessary for OpenSSH developers to understand systems level programming at a level only OS developers can grasp, and maybe OpenSSH wouldn't be the absolute beast it is today without that kind of influence and widespread knowledge found in the OpenBSD team. I dunno. Whatever it is, I support the people and the things they do for me in OpenSSH even if I don't care much about OpenBSD.

14
cpprototypes 1 day ago 7 replies      
A little bit off topic, but this reminds me again of how much the web needs an easy payment system (as easy as in-app purchases in mobile). EFF, wikipedia, I often see notices or news of things they're doing and think to myself that I should donate. But I'm usually in the middle of something and stopping everything to take out my wallet, get the CC, fill out a form, etc. is just too disruptive. So I try to remind myself to do it later, but then forget.
15
jnazario 1 day ago 0 replies      
i used to have commit but was kicked out (along with a bunch of other people over the years). i even wrote a book on openbsd. what i see from afar is that obsd alienated a lot of people with theo's behavior, theo's gone and surrounded himself with sycophants, and the rest of the world caught up to them (largely) in features. as a result the userbase has shrunk - why put up with tirades if you have a secure platform elsewhere - and the features have remained stale.

theo was a visionary, and together with some other really brilliant folks (not counting myself as one of them) accomplished what people said no one could. since then he's been fine tuning that vision but has more or less kept it at around 2003. the world has changed, theo - and as a result the project - hasn't. so, the world caught up and users moved on.

16
soapdog 1 day ago 0 replies      
Just sent 50 USD their way. Its not much but it is as far as I can spend. I recommend everyone helping. OpenBSD is one of those projects that benefit everyone not only those using it.
17
mrbill 5 hours ago 0 replies      
I can see their point about "We need to keep this VAX around because building on actual hardware is different from building on emulation", however I'm sure there's bits of infrastructure, CRTs hooked to KVMs, etc, that could be replaced with newer and more efficient gear that can help with the power bill. You don't have to run the ENTIRE place on cast-off donations and stuff out of a dumpster.
18
brasetvik 1 day ago 2 replies      
You may not be using OpenBSD, but the same organization is behind OpenSSH.

Imagine being without ssh, then go donate. :)

19
midas007 23 hours ago 0 replies      
OpenBSD overall is interesting. The installer alone signals how simple and elegant is the rest of the OS (take a look at the source [0]). A dozen basic questions give or take, and one can have a fully-functional box.

Revenue-wise, the best move would be for a shop like iXsystems, Pair or ByteMark to step up to cover costs. And, any shop that uses OpenSSH on a large scale should be able to pony up some cash to keep Open{SSH,BSD,CVS,{NTP,BGP,OSPF,SMTP,IKE}D} alive. For example, it would be nice to see OpenBSD on Amazon, and AWS might even be willing to fund kernel changes and more to accomplish that.

Finally: check out this handy script which makes it OpenBSD a whole lot easier to get started and complete common tasks. [1]

References:

[0] http://www.openbsd.org/cgi-bin/cvsweb/src/

[1] https://gist.github.com/steakknife/6120072/raw/shave

20
kscottz 1 day ago 1 reply      
I would love to see more FOSS projects on GitTip (https://www.gittip.com/). It would seem to me that regular small donations that can be budgeted would be more helpful than just scratching where and when it itches. Giving $1 a week versus $50 at once is so much more convenient. We as the FOSS community need to own up that writing and hosting software isn't free, and most of us as highly paid engineers are in a position to be charitable and help out. My resolution for this year is to give away 1% of my income to the FOSS community and related charities (EFF, Wikipedia, Ada Initiative, PSF, etc). I challenge everyone on HackerNews to do the same. Stop bitching and put your money where your mouth and let's go help make a better world.
21
jlgaddis 1 day ago 1 reply      
Although the priority at this point is certainly paying the electric bill, you might also consider taking a look at the "Hardware Wanted" page [0] and seeing if you have anything laying around that one of the developers can use. It's typically not brand new top-of-the-line gear they're looking for so you might be surprised.

[0]: http://openbsd.org/want.html

22
kriro 1 day ago 0 replies      
Seems insane that they can't get enough funding given the current security climate. Think of some of the eccentrics whatever you want but they have always been fighting the good fight as far as I'm concerned.

No BLOB is a very sane idea for example. I think we'd be further along if that was enforced stricter by other projects.

23
plainOldText 1 day ago 0 replies      
I'm not an OpenBSD user, but I like their focus on security. And this could be a positive influence on other platforms. Donated.
24
jlgaddis 1 day ago 1 reply      
I read through some of this thread last night and was about to throw them a few bucks. I don't really consider myself an OpenBSD user although I do have a router here that runs it for pf. Other than that, I haven't touched it in probably a decade.

I'm glad I read through the whole thread because by the time I got to the end of it I had changed my mind. In one e-mail to the list, Theo basically said (in effect) that a donation of $20 wasn't even worth it. (Granted, $20 isn't much in the grand scheme of things but I feel that it reasonable covers my use of OpenBSD.)

There are many people who think that Theo is the worst thing for the Project (because he's such an asshole). OTOH, however, there are many who think that the Project wouldn't exist if it weren't for Theo.

26
openbsddesktop 1 day ago 1 reply      
VIDEOS

Exploit Mitigation Techniques: an Update After 10 Years (by Theo)http://tech.yandex.com/events/ruBSD/2013/talks/103/

An OpenBSD talk by Michael Lucashttps://www.youtube.com/watch?v=BXPV3vJF99k

27
akulbe 1 day ago 0 replies      
If you use OpenSSH... you benefit from OpenBSD's work, even if you're not an OpenBSD user.

SSH alone, and all the utilities that use it... have made my work/life SOOOOOO much easier. That may sound silly, but when you don't have to search for some 3rd-party utility because what you need is built-in... it makes life easier.

I'm donating.

28
Aqueous 13 hours ago 0 replies      
Perhaps it would be best to move all non-specialized builds into Amazon or Rackspace or some automated CI service, and to move the specialized builds to a differnet, less expensive place, or keep them where they are. At the very least your electricity costs will go down.
29
jlgaddis 1 day ago 1 reply      
I'm excited about buying a cupcake from one of the OpenBSD developers at the bake sale.
30
ryen 19 hours ago 0 replies      
I think OpenBSD should reach out directly to manufacturers of the many hardware platforms they support (HP, SPARC (now Oracle), etc) and ask for donations. Of course, only the companies that are still in business.
31
ThinkBeat 13 hours ago 0 replies      
A lot of the discussion so far revolves around whether or not Theo can be abrasive. Who cares?

I would rather have a non compromising, highly secure operating system available, with all the source code available for me to see (none of that binary blob business)than a mediocre operating system that was somewhat securefrom a guy who was really chill.

Funding the OpenBSD project is not a decision about the personality of Theo, its a decision on the usefulness and and the quality of work that the OpenBSD community creates.

When I am setting up and configuring and relying on a server with OpenBSD, I could care less if Theo is eccentricor not. I am in awe of the technical brilliance of OpenBSD.

And yes I have donated and do donate to the project, and I encourage clients I have that are running on OpenBSD to do the same.

32
blahbl4hblahtoo 14 hours ago 0 replies      
I donated 50 bucks. These guys have been awesome for a long time. Sorry it took me this long to donate.

EDIT: Here's the thing. I don't directly use OpenBSD, but they have influenced more than just UNIX for a long time.

33
jms703 1 day ago 1 reply      
What will happen to OpenSSH if OpenBSD can't keep the lights on?
34
jpessa 1 day ago 0 replies      
what i find interesting (read: suspicious) is how they have seemingly prioritized and budgeted other things ahead of paying for electricity.

when i make budgeting decisions (whether personal or in business), i start with the needs before going to the "nice to haves". for openbsd, i can't help but assume powering their various servers/systems is kiiind of a priority...

so what i want to know is:- the over all budget $ amount for 2014- what was the cost of power in 2013 * how did you get to $20k for 2014?- which priorities are worth funding over power

my suspicion is that there's plenty of room for give and take here.

35
bhaile 1 day ago 0 replies      
Thanks for linking it here and would be better if it was linked to an article providing additional details other than electricity costs. Other users have posted the relevant links.

On another note, the readability of the font was a turn off for me. Fortunately, there is an option to view it in plain text. adding &q=raw at the end of the URL.http://marc.info/?l=openbsd-misc&m=138972987203440&w=2&q=raw

36
annnnd 16 hours ago 1 reply      
I don't see the point - they should ask for subscriptions, not for donations. What will they do next month / year, ask again? And again?
37
diestl 1 day ago 3 replies      
I wonder if this is a case of OS Darwinism. The fact that it seems to be hanging on by a thread shows most companies are using and contributing to Linux. I have never used OpenBSD so not sure what it overs over Linux as a Unix implementation?
38
siculars 1 day ago 0 replies      
just donated 0.10 btc. really easy with bitpay integration. would rather not give paypal or my cc the fees.
39
bolle 1 day ago 1 reply      
22 seems an appropiate amount. Or whatever your portnumber is for your SSH-server.
40
mariuolo 1 day ago 0 replies      
What about emulators?
41
elwesties 1 day ago 2 replies      
This may be a silly question but could they virtualise the process on EC2?
42
SilverSurfer972 16 hours ago 0 replies      
I would love to donate some Litecoin
43
jijji 1 day ago 2 replies      
The bigger question is who pays $2000/month for electricity for a server?
44
Senkwich 1 day ago 0 replies      
Donated. Hope it helps.
45
ffrryuu 1 day ago 0 replies      
The one with the gold makes all the rules.
2
Show HN: Kimono Never write a web scraper again kimonolabs.com
669 points by pranade  2 days ago   228 comments top 85
1
randomdrake 2 days ago 5 replies      
The presentation is beautiful and the website is great, but the tech broke so I have no idea how or if this even works. This is a wonderful concept and one I've talked about doing with others. I was really excited to try this. I watched the demo video and it seemed straightforward.

I went to try and use it on the demo page it provides, going through and adding things, but when I went to save it, I just received an error that something went wrong. Well, crap. That was a waste of time. Oh well, maybe it's just me.

Alright, I'll give it another shot using the website they used in the demo. Opened up a Hacker News discussion page and started to give it a try. Immediately it was far less intelligent than the demo. Clicking on a title proceeded to select basically every link on the page. Somehow I clicked on some empty spots as well. Nothing was being intelligently selected like it was in the demo. Fine, that wasn't working tremendously well, but I wanted to at least see the final result.

Same thing: just got an error that something went wrong and it couldn't save my work.

Disappointing. I still might try it again when it works 'cause it's a great idea if they really pulled it off. So far: doesn't seem to be the case.

2
DanBlake 2 days ago 2 replies      
Show me it working with authentication and you will have a customer. Scraping is always something you need to write because the shit you want to get is only shown when you are logged in.
3
dunham 2 days ago 3 replies      
The Simile group at MIT did something similar back around 2006. Automatic identification of collections in web pages (repeated structures), detection of fields by doing tree comparisons between the repeated structures, and fetching of subsequent pages.

The software is abandoned, but their algorithms are described in a paper:

    http://people.csail.mit.edu/dfhuynh/research/papers/uist2006-augmenting-web-sites.pdf

4
georgemcbay 2 days ago 1 reply      
I've written more web scraping code than I care to admit. A lot of the apps that ran on chumby devices used scraping to get their data (usually(!) with the consent of the website being scraped) since the device wasn't capable of rendering html (it eventually did get a port of Qt/WebKit, but that was right before it died and it wasn't well integrated with the rest of the chumby app ecosystem).

This service looks great, good work! But since you seem to host the APIs created how do you plan to get around the centralized access issues? Like on the chumby we had to do a lot of web scraping on the device itself (even though doing string processing operations needed for scraping required a lot of hoop jumping optimization to run well in ActionScript 2 on a slow ARMv5 chip with 64mb total RAM) to avoid all the requests coming from the same set of chumby-server IP addresses, because companies tend to notice lots of requests coming from the same server block really quick and will often rate limit the hell out of you, which could result in a situation where one heavy-usage scraper destroys access for every other client trying to scrape from that same source.

5
GigabyteCoin 1 day ago 1 reply      
I'm curious how you plan to avoid/circumvent the inevitable hard IP ban that the largest (and most sought after targets) will place on you and your services once you begin to take off?

I could have really used a service like this just yesterday actually, I ended up fiddling around with iMacros and got about 80% of what I was trying to achieve.

6
hcarvalhoalves 2 days ago 3 replies      
This is excellent. Even it if doesn't work for scraping all sites, it simplifies the average use case so much that it's not even funny.

Feature proposal: deal with pagination.

7
sync 2 days ago 0 replies      
Undo button is awesome.

More web apps need an undo button.

8
thinkzig 1 day ago 1 reply      
Great work so far. The tool was very intuitive and easy to use.

My suggestion: once I've defined an API, let me apply it to multiple targets that I supply to you programatically.

The use case driving my suggestion: I'm an affiliate for a given eCommerce site. As an affiliate, I get a data feed of items available for sale on the site, but the feed only contains a limited amount of information. I'd like to make the data on my affiliate page richer with extra data that I scrape from a given product page that I get from the feed.

In this case, the page layout for all the various products for sale is exactly the same, but there are thousands of products.

So I'd like to be able to define my Kimono API once - lets call it CompanyX.com Product Page API - then use the feed from my affiliate partner to generate a list of target URLs that I feed to Kimono.

Bonus points: the list of products changes all the time. New products are added, some go away, etc. I'd need to be able to add/remove target URLs from my Kimono API individually as well as adding them in bulk.

Thanks for listening. Great work, again. I can't wait to see where you go with this.

Cheers!

9
fsckin 2 days ago 1 reply      
Constructive Tone: I figured that it might be nifty to scrape cedar pollen count information from a calendar and then shoot myself an email when it was higher than 100 gr/m3.

This would be a pretty difficult thing to grab when scraping normally, but the app errors before loading the content:

https://www.keepandshare.com/calendar/show_month.php?i=19409...

JS error: An error occurred while accessing the server, please try againError Reference: 6864046a

10
rlpb 2 days ago 1 reply      
Are you familiar with ScraperWiki? I'm wondering how your work fits in with it.

Edit: looks like they've moved away from that space, but have an old version available at: https://classic.scraperwiki.com/

11
aqme28 2 days ago 2 replies      
I would seriously consider rethinking that Favicon.
12
bambax 1 day ago 2 replies      
> Web scraping. It's something we all love to hate. You wish the data you needed to power your app, model or visualization was available via API. But, most of the time it's not. So, you decide to build a web scraper. You write a ton of code, employ a laundry list of libraries and techniques, all for something that's by definition unstable, has to be hosted somewhere, and needs to be maintained over time.

I disagree. Web scraping is mostly fun. You don't need "a ton of code" and "a laundry list of libraries", just something like Beautiful Soup and maybe XSLT.

The end of the statement is truer: it's not really a problem that your web scraper will have to be hosted somewhere, since the thing you're using it for also has to be hosted somewhere, but yes, it needs to be maintained and it will break if the source changes.

But I don't see how this solution could ever be able to automatically evolve with the source, without the original developer doing anything?

13
tectonic 2 days ago 1 reply      
14
jval 1 day ago 0 replies      
Great job guys.

One problem I've had though is that I think you guys are hosted on AWS - a lot of websites block incoming connections from AWS.

Are there plans to add an option in future to route through clean IPs? Premium or default, this would be cool and make it a lot more useful.

15
trey_swann 2 days ago 0 replies      
This is a great tool! In a past life we needed a web scraper to pull single game ticket prices from NBA, MLB, and NHL team pages (e.g. http://www.nba.com/warriors/tickets/single). We needed the data. But, when you factor in dynamic pricing and frequent page changes you are left with a real headache. I wish Kimono was around when we were working on that project.

I love how you can actually use their "web scraper for anyone" on the blog post. Very cool!

16
guptaneil 2 days ago 1 reply      
Nice work, this is much better than I expected! Does it require Chrome? It doesn't seem to work in Safari for me. Also, does Kimono work for scraping multiple pages or anything that requires authentication?
17
jjcm 2 days ago 1 reply      
Very cool, and I like that the link is your announcement page running inside of the demo. Really drives home the idea.

That said, it looks like it can't do media right now. I would love it if it could at least give me a url for images/other media.

18
fnordfnordfnord 2 days ago 1 reply      
>Sorry, can't kimonify

>According that web site's data protection policy, we were unable to kimonify that particular page.

Sigh... Oh well... Back to scraping.

19
ForHackernews 2 days ago 1 reply      
This looks really slick. What happens if a website you're scraping changes its design? Do you respect robots.txt?
20
IbJacked 1 day ago 1 reply      
Wow, this is looking good, I wish I had it available to me 6 months ago! Nice job :D

I don't know if it's just me or not, but it's not working for me in Firefox (OSX Mavericks 10.9.1 and Firefox v26). The X's and checkmarks aren't showing up next to the highlighted selections. Works fine in Safari.

21
pknight 2 days ago 1 reply      
That UI made me go wow, this could be an awesome tool. Idea that pops into my mind is being able to grab data from those basic local sites run by councils, local news papers etc and putting it into a useful app.

How dedicated are you guys to making this work because I'd imagine there are quite a few technical hurdles in keeping a service like this working long term while not getting blocked by various sites?

22
alternize 2 days ago 1 reply      
looks promising!

to be fully usable for me, there are some features missing:

- it lacks manual editing/correcting possibilities: i've tried to create an api for http://akas.imdb.com/calendar/?region=us with "date", "movie", "year". unfortunately, it failed to group the date (title) with the movies (list entries) but rather created two separate, unrelated collections (one for the dates, one for the movies).

- it lacks the ability to edit an api, the recommended way is to delete and recreate.

small bugreport: there was a problem saving the api, or at least i was told saving failed - it nevertheless seems to be stored stored in my account

23
dmunoz 2 days ago 1 reply      
I'm normally a bit worried when a thread quickly fills up with praise, but this looks very nice.

It's something I have thought about, as I'm sure many people who have done any amount of scraping have, but never went forward and tried to implement. The landing page with video up top and in-line demo is a pretty slick presentation of the solution you came up with. Good job.

24
tlrobinson 2 days ago 1 reply      
I built something very similar last year, but sadly never got around to polishing and launching it: http://exfiltrate.org/

(There's a prototype of an API generator hidden in a menu somewhere but it's nowhere near production ready)

25
ThomPete 2 days ago 1 reply      
Thank you for building a tool I been wanting so I don't have to!

Can't wait to play around with this tonight.

Suggestion. Allow one to select images.

26
blazespin 2 days ago 0 replies      
There's a huge business here if you keep at it. I'll throw money at the screen if you can make this work.
27
jlees 2 days ago 1 reply      
I like how you've thought through the end to end use case: not just generating an API, but actually making it usable. I've done my fair share of web scraping and it's not an easy task to make accessible and reliable -- good luck!

It makes me wonder if there isn't a whole "API to web/mobile app with custom metadata" product in there somewhere. I can imagine a lot of folks starting to get into data analysis and pipelines having an easier time of it if they could just create a visual frontend in a few clicks.

28
rpedela 1 day ago 1 reply      
Definitely awesome presentation and product.

The example doesn't seem to work right on Firefox. On Chrome, if I click "Character" in the table then it highlights the whole column and asks if I want to add the data in the column. On Firefox, clicking "Character" just highlights "Chatacter" and that is it.

Ubuntu 12.04

Firefox 25.0.1

29
phillmv 2 days ago 2 replies      
The UX is great and a journalists everywhere will thank you.

But outside of government websites I don't see how a lot of this is even legal, per se?

30
thatthatis 1 day ago 1 reply      
This is my third time trying to get an answer to this question: does your crawler automatically respect robots.txt?
31
ameister14 2 days ago 0 replies      
I really like how you guided me in to demoing. Nice job.
32
jmcgough 2 days ago 1 reply      
Really sleek interface, and looks like it could be extremely useful (I just spent a few hours cranking out Nokogiri this morning).

Oh, typo: "Notice that toolbar at the toop of the screen?"

33
lips 1 day ago 0 replies      
I'm experiencing login errors (PEBKAC caveat: password manager, 2x checked, reset), but the support confirmation page is a nice surprise.

http://i.imgur.com/w01CoUy.jpg

34
eth 1 day ago 0 replies      
Great tool!

I'm coming at things from a non-coder perspective and found it easy to use, and easy to export the data I collected into a usable format.

For my own enjoyment, I like to track and analyze Kickstarter project statistics. Options up until now have been either labor intensive (manually entering data into spreadsheets) or tech heavy (JSON queries, KickScraper, etc. pull too much data and my lack of coding expertise prevents me from paring it down/making it useful quickly and automagically) as Kickstarter lacks a public API. Sure, it is possible to access their internal API or I could use KickScraper, but did I mention the thing about how I dont, as many of you say, "code"?

What I do understand is auto-updating.CSV files, and that's what I can get from Kimono. Looking forward to continued testing/messing about with Kimono!

35
jfoster 2 days ago 1 reply      
Cool concept. One concern I'd have about this type of tool is that when it encounters something it can't handle, I'm stuck. Writing your own scraper means that you can modify it when you need to. I think the ultimate solution would be something like Kimono with the ability to write snippets of custom javascript to pull out anything that it can't handle by default.
36
rafeed 2 days ago 0 replies      
This is awesome. Really nice implementation and so useful for many different applications. Just signed up and looking forward to trying this out.
37
chevreuil 1 day ago 0 replies      
We all know there are a lot of existing tools that does the same things. But I've not met one with such a polished UX. Kudos to the Kimono team, I'll definitly recommend your product.
38
rmason 2 days ago 0 replies      
I thought to myself oh boy yet another web scraper as a service but got surprised. I haven't been this impressed with a product video since Dropbox.
39
garyjob 1 day ago 0 replies      
I found the one click action for selecting an entire column of values as well as the UI/UX on the top column of the page to be very impressive. We were thinking of a nice clean way to represent that particular UI/UX flow in this browser extension we built as well. Will incorporate that in our next release.

https://chrome.google.com/webstore/detail/krakeio/ofncgcgajh...

Would love to meetup and exchange some ideas if you are based in Bay area.

40
critium 1 day ago 1 reply      
Please get this off the ground. I would also possibly suggest a separate business, website regression testing.

Selenium is WAAAY to painful.

41
kenrikm 2 days ago 1 reply      
Looks awesome, however I keep getting errors and 404s. Could this be an issue on my end (seems to be working for others) or just HN making the servers beg for mercy?
42
cbaleanu 2 days ago 1 reply      
Does it do logging in to websites then fetching?Do you plan to add scripting to it?
43
mhluongo 1 day ago 1 reply      
Any chance you guys plan to add link hrefs to CSVs? I'd love to use this now, but I need the href for backlinks and future inference.
44
twog 2 days ago 0 replies      
Well done on the product & solving a clear need! This is extremely useful for hackathons/prototyping. I also loved the live demo in the blog post and you did a wonderful job with the design/layout/colorscheme of the site.
45
pranade 2 days ago 0 replies      
Thanks guys, glad you like it. Welcome any feedback so we can make it better!
46
shekyboy 2 days ago 1 reply      
Like the parameter passthrough feature. Take a look at places where the parameters are part of the URL structure. For example a Target product pagehttp://www.target.com/p/men-s-c9-by-champion-impact-athletic...

In order to get data for a different product, I will have to modify the URL itself. I think same holds true for blog posts.

47
ph4 2 days ago 1 reply      
Very nice job. What about scraping data from password-protected pages?
48
catshirt 2 days ago 1 reply      
really excited to see this. i've had the idea (and nearly this execution) in mind for years but no use or ambition to get it done.

given the pricing though i'm almost motivated to make my own. as a hosted service the fees make sense with the offerings. but not only would i rather host my own- it would be cheaper all around. would you consider adding a free or cheap self hosted option?

aside, i think there is a mislabel on the pricing page. i'm guessing the free plan should not have 3 times the "apis" than the lite plan.

49
xux 2 days ago 0 replies      
Wow looks amazing. I tried doing some queries on public directories, and it even supports parameter passing. Will be using this for some side projects.
50
PhilipA 1 day ago 0 replies      
It looks cool, but very expansive compared to Visual Web Ripper, which you pay way less for (but has to host yourself).
51
lucasnemeth 1 day ago 0 replies      
Nice job! I really liked, it's a fantastic idea! And your UX is great! Just one thing I've found when testing: I've had some problems with non-ascii characters, when I was visiting brazilian websites, such as this : www.folha.com.br.
52
shamsulbuddy 1 day ago 0 replies      
Does such Webscraping is allowed legally. Since it is not done directly from our servers and if any legal action will be taken by the scraped website , will it be on kimonolabs..or the user..
53
bigd 2 days ago 1 reply      
Seems it can't see the stuff inside angular views.. well at least mines..

But for the rest, awesome product. Thanks.

54
paul1664 1 day ago 0 replies      
Reminds me of Dapper

http://open.dapper.net/

This allowed you to do similar, before being consumed by Yahoo. Might be worth a look.

55
BinaryBird 2 days ago 0 replies      
Nice tool, slick UI. It worked for some pages and not for others. Currently I'm using Feedity: http://feedity.com for all business-centric data extraction and it has been working great (although not as flexible as kimono).
56
joshmlewis 1 day ago 1 reply      
Is there an ability to scrape more than one page of data?
57
cycnusx 1 day ago 0 replies      
58
kyriakos 1 day ago 1 reply      
It appears that it doesn't work with websites containing international characters.
59
dikei 2 days ago 1 reply      
I don't think this can beat the speed of a hand-tune crawler. When I write crawlers, I skip rendering page and javascript execution if it isn't needed, which massively speed up the crawling process.
60
dmritard96 2 days ago 0 replies      
as someone building a home grown proprietary scraping engine. Consider alternative locations of elements. Most sites are using templating engines so its fairly reliable to find things in the same place, but more often than you might expect, things move a round ever so slightly. Navigation is a fun one also. ;)
61
ewebbuddy 2 days ago 1 reply      
Really cool idea and tool. Still need to test this out properly. Is it possible to scrape note just one page but a stack of them? For example - a product catalog of 1000 SKUs extending upto 50pages.
62
keyurfaldu 1 day ago 0 replies      
Awesome! Hats off.. How about extracting hashtag/GID of any record if applicable, which are typically not rendered on page, but hidden under the hood.
63
wprl 2 days ago 0 replies      
It's easy not to write web scrapers even without this tool ;)
64
tchadwick 2 days ago 1 reply      
This looks really useful, and I'm trying to figure out if I could use it on a project I'm working on, but hitting an issue. I sent a support message. Nice job!
65
diegolo 1 day ago 1 reply      
It would be nice to have a view also on the raw html code, e.g., to create a field containing the url of an image in the page.
66
cullenmacdonald 2 days ago 1 reply      
the reason i ever have to write a scraper is because of pagination. while this looks awesome, i'll have to stick to scraping until that is solved. :(
67
bluejellybean 2 days ago 1 reply      
How (if at all) does this run on javascript heavy sites?
68
yummybear 1 day ago 1 reply      
Looks very nice. There seems to be an issue with international characters though (//).
69
iurisilvio 2 days ago 1 reply      
What about some navigation tools there?

Looks pretty good, but it does not really replace my scrappers. Maybe some of them...

70
narzero 1 day ago 0 replies      
I like the concept. Would love to see page authentication
71
NicoJuicy 2 days ago 1 reply      
This is really slick! Btw. Who made your intro video?
72
dome82 2 days ago 1 reply      
I like the concept and it looks similar at Import.io
73
aaronsnoswell 2 days ago 1 reply      
Man that demo is impressive!
74
mswen 1 day ago 0 replies      
How does this compare with Mozenda?
75
szidev 2 days ago 0 replies      
great idea. i'll have to keep this in mind for future projects.
76
timov 1 day ago 0 replies      
You can use the utility without registration or login by blocking the login prompt with, for example, AdBlock.
77
taternuts 2 days ago 0 replies      
That looks quite swift
78
abvdasker 2 days ago 0 replies      
I kind-of enjoy writing web scrapers.
79
byteface 1 day ago 0 replies      
use any chrome xpath plugin and give that to YQL
80
harryovers 2 days ago 0 replies      
so what do you do that import.io doesn't?
81
rismay 1 day ago 0 replies      
OMFG.
82
iamkoby 2 days ago 0 replies      
i love this! and amazing video!
83
pyed 2 days ago 0 replies      
actually I love scraping :(
84
tonystark 2 days ago 0 replies      
neat.
85
nnnn 2 days ago 0 replies      
"Never write a web scraper again"... yea right.. sick and tired of such gimmicks and self promotion on the net today.
3
Introducing our smart contact lens project (for diabetics) googleblog.blogspot.com
568 points by dboyd  1 day ago   168 comments top 38
1
morganherlocker 1 day ago 10 replies      
Type I diabetic here. Assuming current tech stays where it is (not saying it will), this could easily tack 10 years on to my lifespan. For many who watch their diabetes less closely (something I cannot fault anyone for), this could add 20-30 years.

For anyone who does not know, type I diabetes is not something you can just follow a doctor's direction on and be ok. Even if you follow your doctor perfectly, there can still be serious complications, and type I diabetics with the best control are actually more likely to die from severe low blood sugars.

The reason for this is that the optimum blood glucose level is around 100. <70 and you start to be severely mentally impaired, making it difficult at times to seek treatment (finding and eating sugar, in a nutshell). On the flip side, if you are lax on insulin, your blood sugar might hover around 250 for months, and you will feel close to normal. Having a blood glucose this high on a long term basis will have long term effects that are what kill most diabetics in the long run. A low blood sugar, however, can be fatal within minutes to hours.

Either way, a continuous feedback mechanism would help tight control diabetics, and diabetics who do the minimum. Tight controllers could get faster feedback about when they are going into the serious danger zone without having to initiate any action (checking blood sugar), and lax diabetics would get a constant reminder of how they are letting there life slip away (which they normally would rarely see, since they hardly ever check their blood sugar anyway).

I have to say though, I am still a bit skeptical for a few reasons:

- One, I have been told about this sort of miracle technology ever since I was diagnosed 15 years ago.

- Two, the medical complex locks down their tech and extracts the maximum value out. There is not a single glucose device on the market that lets you extract the data out of your glucose monitor and crunch the data how you want. I have worked on hacking these devices to extract data and the legal verbiage around these activities has strongly discouraged me from releasing anything. Previous continuous glucose monitoring systems. These companies would prefer you rot in the dark, than to lose one bit of profit.

- Three, if one of these devices is not 100% perfect, it gets shot down and banned from the market. This is probably a combination of profit-motivated industry and caution-motivated government. A great example of this is a continuous glucose monitoring, non-invasive watch that came out ~ a decade ago. It was on the market for several years, before being banned. I, like just about every person in the thread I linked, would pay $10k+ for one of these, despite the reduced accuracy over traditional devices. Entrepreneurs in the health industry take note.

[1] [http://www.diabetesdaily.com/forum/testing-blood-sugar/61908...]

2
awolf 1 day ago 3 replies      
As a type 1 diabetic I can say this would be a huge improvement over current continuous glucose monitoring systems. No expensive disposable parts. No needing to inject a new sensor each week. No strange, uncomfortable, and (often) painful bulge stuck to your abdomen.

>Were in discussions with the FDA, but theres still a lot more work to do to turn this technology into a system that people can use

I'm chomping at the bit. Anyone familiar with process know how soon this could possibly be available?

3
dshankar 1 day ago 2 replies      
Microsoft was working on similar technology 3 years ago [1]

It looks like Google poached MSFT's engineers to work on this -- Babak Parviz was working on this at Microsoft 3 years ago, and is now cofounder of the smart contact lens team at Google.

[1]: Functional Contact Lens Monitors Blood Sugar Without Needles (http://research.microsoft.com/apps/video/dl.aspx?id=150832)

4
psbp 1 day ago 0 replies      
Solve for X talk about this technology: http://www.youtube.com/watch?v=d6g581tJ7bM

This particular implementation: http://www.youtube.com/watch?v=d6g581tJ7bM#t=10m15s

5
fesja 1 day ago 3 replies      
All my praise to these Google engineers and scientists. Another completely crazy idea that will really help millions of people every day. Thanks Google!

We have discovered a lot of Google X technologies in the last months. It seems that Google X is really working. We may have to stop having fun of Google+.

6
cargo8 1 day ago 0 replies      
Google never ceases to amaze me for taking the initiative to really pursue and commercialize these things.

If people are interested, here is a relevant research paper in IEEE about the massive potential contact lenses have to mate technology and bioinformatics:http://spectrum.ieee.org/biomedical/bionics/augmented-realit...

7
kamens 1 day ago 1 reply      
Would immediately and significantly improve my life.

Crossing my fingers very hard. Want this to be reality.

8
mortov 15 hours ago 1 reply      
I'm amazed at how many people on HN are diabetic !

Could be an interesting study on cause/effect - is all this sitting at screens contributing to an epidemic or are there other factors ? Perhaps respondents are just self-selecting because of the subject matter ?

It's also interesting how anyone who needs to track their blood sugars likes the idea of an easier and more convenient method - typical finger prick readings up to 4 times a day can leave your fingers in a real mess and pretty painful so even sticking something to your eyeballs sounds attractive !

9
TeMPOraL 15 hours ago 0 replies      
This is why I still love Google. Between this, self-driving cars and other world-changing projects, I say take my data if you need it. You're one of the few companies in this world that seems to bring a big, direct net benefit to humanity.
10
oh_sigh 21 hours ago 1 reply      
Where's all the FUD about collecting user data and doing evil things with it that we've been seeing in the Google+Nest stories lately?

"But what about when google sells your data to insurance companies, who then penalize diabetics for not maintaining specific glucose levels?'

"Do you really want google to know every single thing you put into your body?"

"Can we trust google to not put advertisements in the contact lens, making you watch a 15 second commercial before being able to read your gluose levels?"

11
lazerwalker 10 hours ago 1 reply      
I don't at all want to downplay the honest importance of this project this is the true sort of real "tangibly improving people's lives" technology that not enough people are aspiring to these days but the timing of this announcement is very strategic on Google's part. Days after people get VERY upset at Google for buying Nest, and moaning about Google's evil surveillance state, Google turns around and announces something that's a legitimate force for good in the world. Very clever.
12
inetsee 1 day ago 0 replies      
I am encouraged by the fact that Google is working in this area. If any company can overcome the obstacles to this technology becoming available soon and at a reasonable price, it would probably be Google.

I am discouraged by the fact that the underlying technology (measuring glucose from tears) was first reported more than two years ago. There is another (sort of) non-invasive glucose measuring technology that involves injecting a biofluorescent dye under the skin, then using a device that measures the fluorescence that varies with the blood glucoe levels. This technology was also first reported years ago, and is also apparently nowhere near being available.

13
caseydurfee 1 day ago 2 replies      
Continuous glucose monitors have been around for a while. I'm not sure who this would help.

The fundamental problem is that glucose levels in non-blood fluids do not exactly match blood glucose. The current monitor solutions use interstitial fluid in the skin. They still require the user to test themselves several times a day and recalibrate the monitor based on blood glucose, and they can't alert the user if their blood sugar is low until it's already a serious situation. They also frequently give false positives.

This is a new (but very clever!) way to do something that has been around for a while, not a revolution, unless tears track blood glucose much closer than interstitial fluid does. Simply based on first principles, that seems unlikely.

And there are basic hygiene problems wearing contacts while you're asleep, which is when monitoring would be most useful. If google has the technology to make contact lenses that you can wear 24*7 without getting ulcerative keratitis, that's more revolutionary than another way to monitor blood sugar.

14
jerryhuang100 1 day ago 3 replies      
One major concern I would have is that, in diabetic patients their eyes experience more dryness than non-diabetic patients. This might lead to more scratches on the cornea and prone to further infections and ulcers. As diabetics care 101, diabetics patients have mucher high risk of systematic infections. And this is all way before any diabetic retinopathy develops in those patients. So why Google[x] thinks it's a good idea to have diabetics patients wear contact lens?
15
cpeterso 22 hours ago 0 replies      
A few years ago, there was research into nano ink tattoos that can continuously monitor glucose, but I haven't heard any recent news.

http://web.mit.edu/newsoffice/2010/glucose-tattoo-0528.html

16
sytelus 17 hours ago 1 reply      
As always, my question for all miniature cool looking devices is just this: How do you power this thing?
17
prawn 20 hours ago 1 reply      
Anyone else have a problem where if they read an interesting story like this about potential future technology, they are virtually incapacitated and unproductive for the rest of the day thinking about the potential?

Obviously, a Google Glass or Oculus Rift equivalent but with contact lenses has to be one thing everyone's long thought of. But what about activating a contextual display by closing one eye briefly (a map or information about a person you're meeting), or seeing a definition of a word spoken in conversation by closing the other eye, or watching a movie with both eyes closed, reading a book the same way, getting song recognition data at any point, etc.

Could we see high-res displays worked into lenses so that they worked, were eventually cheap enough and able to operate in a "pass through" mode so they didn't other interrupt regular vision?

Are any companies working on it? Is it possible? What would be the key challenges?

18
j_s 13 hours ago 0 replies      
Diabetics will find Scott Hanselman's posts and the comments there useful. Three or four of them are linked from here:

http://www.hanselman.com/blog/HackingDiabetes.aspx

19
notatoad 1 day ago 4 replies      
How is this powered? are they generating electricity from the body somehow?
20
_paranoia 1 day ago 0 replies      
This seems like a prototype of what Google Glass will evolve into. The medical device will offer a first generation of solutions to several major problems for augmented reality contact lenses: fitting "an antenna for wireless data communications, a chip to process data, and tiny battery onto a tiny, thin, curved surface..."[1]

Then, a later version will need to solve the problem of projecting crisp images from the contact lens onto the user's retina. Google's experience with Glass seems like it could inform that effort. Perhaps we'll see this product on the market with significant usage within 10 years.

Augmented reality contact lenses have other implications. For example: what does it mean for privacy and advertising to not being able to shut one's eyes?

1. http://www.technologyreview.com/view/517476/google-glass-tod...

21
f-debong 16 hours ago 1 reply      
I think this is one of many such implementations which we will see in the next few months. Not long ago, the FDA posted their guidelines on mobile health, which will most likely be part of the system, finally establishing a hint of what they will require to give an approval to a mobile system in the medical field. I have some experience in this, and can tell you it is very exciting. No blatant ad here, sorry!

Since the risks for ventures in this field have dropped significantly, devices such as this lens now have a much higher probability to actually see the light of day and not just be hidden in the archives, on thrown away napkins and spreadsheets.

Yes, Microsoft worked on it a few years back yet seem to have dropped the ball or shifted their focus, I have also heard of such a project at Sanofi and research institutes around the world yet a google X project may potentially be what this concept needs to make progress and actually have an impact. My sincere gratefulness to you guys at X for going at it!

Forgot to mention, like many others in this thread I am a type 1 diabetic since 30 years, so my gratitude goes a tad further than only thinking it's cool.

22
azernik 23 hours ago 0 replies      
To take this to a purely business/tech place - this is an interesting market for prototyping contact-lense electronics; maybe in 10 or 20 years the new Glass will just be a contact, but for now this looks like one of the few applications where a sensor and two LEDs (high and low) can provide a lot of user value.
23
fjcaetano 22 hours ago 1 reply      
At first, I thought this could be quite intrusive. I mean, you're putting something in your eye! It's the same discussion as the biometry authentication systems (retina scanning, saliva, etc).

But then I realized: the alternative is to put a piece of hardware under your skin! It will be embeded in you "forever" and can only be removed via surgical procedures...

These contact lens, if ever available, will in fact revolutionize the diabetes scene and may open precedent to new "wearable" technologies targeting health.

24
ginzaerin 1 day ago 0 replies      
Type 1 also, and I fully admit that part of what prevents me from using pump therapy is the CGM technology and how invasive and frustrating it is. The lens project gives me hope, but as others in the thread have mentioned - I've gotten my hopes up on several occasions about potential technologies and agree that it seems like a lot of it comes down to profit. Full disclosure, I'm the COO of a tech company and I love data - so a constant frustration is lack of consistent data that can be manipulated in usable ways. (Also mentioned by others in the thread.)
25
ctrl 1 day ago 0 replies      
Type I also, Looks like its time to get over that huge fear of contact lenses.

I dont care if i have to clockwork orange my eyelids, this sounds awesome

26
luuio 19 hours ago 0 replies      
Nothing new under the Sun: January 5, 2012 - http://www.gizmag.com/microsoft-electronic-diabetic-contact-...
27
jisaacks 23 hours ago 0 replies      
> integrating tiny LED lights that could light up

So wait a minute, your eyes will start flashing when your glucose levels are spiking?

28
blueskin_ 16 hours ago 1 reply      
Coming Soon: Targeted advertising based on blood sugar levels.

With a google lens, they could even project adverts directly onto people's eyes.

29
lowglow 1 day ago 4 replies      
Am I the only one that thinks Google[x] is just the arm of google that looks for far future intellectual property to patent and never really produces usable tech?

Please correct me if I'm wrong.

30
nfoz 1 day ago 0 replies      
This is a legitimately fantastic project. Can't wait for some details about how it works.
31
RA_Fisher 15 hours ago 0 replies      
For those that are interested, I'm a data scientist and I spent a bit of time visualizing my Type 1 partner's data: http://statwonk.github.io/blog/2014/01/05/visualizing-diabet...
32
BrainInAJar 21 hours ago 1 reply      
Jeez, Google really has no limits on how much data about you they want
33
sarojt 21 hours ago 0 replies      
All diabetics really would appreciate this innovation - my grandmother was delighted to hear it.
34
efremjw 14 hours ago 0 replies      
ohhhh, because it's just so comfortable to have contacts in the first place. what's wrong with embedding somewhere else?
35
ericthegoodking 19 hours ago 0 replies      
Wonderful news! I hope this thing works!
36
kimonos 1 day ago 0 replies      
Wow! This is great news for my father!
37
dia473 20 hours ago 0 replies      
We have developed a non-invasive system for measuring blood glucose (patent protected, many publications in peer reviewed magazines, working prototype).

If someone has contacts in the VC scene or is working in the VC scene and is interested please let me know.

(throw away account)

38
guidefreitas 1 day ago 1 reply      
Great. Now your are going to sign up with Google+ to blink.
4
Requirements for DRM in HTML are confidential w3.org
567 points by duncan_bayne  4 days ago   400 comments top 29
1
simonsarris 3 days ago 23 replies      
I suppose that the title assertion is to be expected. DRM only works if you don't know how it works.

~~~

I'm not sure I see anything wrong with DRM per se (this could be my fever talking), there are probably good uses I'm too dim to think about, but I do think it's unnecessary as part of the HTML specification.

There's no industry or company that has switched to DRM-free content, that I know of, that has failed or suffered because of it:

* Music is largely available DRM-free now, thanks to Amazon's MP3 store (at the least, I'm sure there are others)

* For games, Steam makes it easy to avoid SecuROM Hell

* Despite DRM, all of Netflix's original series House of Cards was available on The Pirate Bay within hours of release. This doesn't seem to hurt Netflix's wish to create more content, or police it more heavy-handedly. (Maybe they would if they could)

For that matter, I think in the modern case every single time a business went DRM free it turned out OK. Isn't that right? In all modern cases, maybe after 2006-ish, DRM-free businesses were accompanied with an easy way to get the content online, and sales did not seem to suffer because at the end of the day piracy can appear (or be) shady and people (rightfully) don't trust shady websites, even The Pirate Bay with all of its popups.

I wish we had better numbers. I would like to see a real analysis on all the reasons people don't pirate and instead buy on Steam. I wish there was a good way to convince media businesses at large.

But I guess this is all water under the bridge, and I'm preaching to the choir.

2
Nursie 3 days ago 5 replies      
Great. DRM. The best example of shooting yourself in the foot ever.

Give customers encrypted content and the keys, try to prevent them from freely using the two together, undermine copyright fair use and first sale doctrines as you go along.

Intended effect - No Piracy

Actual effect - Paying customers get crippled products, pirates carry on regardless

It's crazy. And the more they try to lock it down the worse their products become and the better piracy looks in comparison. Pirates don't only beat the legit industry on price, they beat them on quality and availability. How can the industry allow this to stand? Let alone continue down the same path with their fingers in their ears shouting LALALALALALA I CAN'T HEAR YOU!?!

3
andybak 3 days ago 2 replies      
A key paragraph:

link: http://lists.w3.org/Archives/Public/public-restrictedmedia/2...

Well, as I say, the actual requirements that lead to the proposal of EMEwould be a start. This is how it looks to those who don't agree thatEME is a good fit with the Open Web:

- 'big content' has certain requirements relating to preventing users copying data streams

- they won't make those requirements public (as you've said, the agreements are confidential)

- their licensees propose a technical solution that is unacceptable to many others because it necessitates the use of non-user-modifiable client components

- all proposed alternatives (e.g. FOSS DRM, server-side watermarking, client-side watermarking, no DRM at all) are shot down as being either too expensive or inadequate to the (secret) requirements

In a normal software project, I'd take an apparently insoluble conflict(the requirement for non-user-modifiable client components) to mean thatwe have done a poor job of determining requirements.

Hence my request for either a real user to talk to (e.g. an MPAA rep) orthe actual requirements docs, which you've told me are confidential.

And that sets off my spidey-senses ... something is not quite righthere.

4
Daiz 3 days ago 2 replies      
This should really be at the top of every HTML DRM discussion:

HTML DRM will not give you plugin-free or standardized playback. It will simply replace Flash/Silverlight with multiple custom and proprietary DRM black boxes that will likely have even worse cross-platform compatibility than the existing solutions. In other words, giving in to HTML DRM will only make the situation worse.

Some vendors will keep pushing for it, but at the very least we should not officially sanction what they are doing.

5
rlx0x 3 days ago 4 replies      
This is all so ridiculous, rtmp for instance is as secure a DRM as its ever gonna get and that never stopped me from downloading a stream. Even things like HDMI/HDCP is broken beyond repair. And all of this should justify damaging the w3c reputation forever, what are they thinking?!

This whole concept of DRM is just idiotic, its enough if one guy breaks the DRM and releases it. Why should I even bother booting a propertary OS (windows) and buying a stream everytime I want to watch something if I can just download a release and watch it, and its not like they can do anything against that either.

Why should I bother and buy HDCP capable new hardware, bother with proprietary NSA-compliant US software I much rather buy the DVD, trash it and just download it in a open and free format (I don't even bother with ripping (and breaking CSS) anymore).

6
josteink 3 days ago 2 replies      
Email the W3C. Tell them what you think of this bullshit (in reasonably polite manners).

I've done it. I've gotten a non-canned response.

But clearly they need more people at the gates bitching. This needs to be stopped.

7
belluchan 3 days ago 5 replies      
Can't we just fork the w3? Start using Firefox and forget about these people. Oh I'm sorry your browser is a little slower, but at least it's not Google made.
8
ronaldx 3 days ago 0 replies      
Why is W3C involved in this?

Not only does this create a lack of openness and transparency in the core of the web, but "big content" creators get to pass on the costs of DRM that nobody else benefits from, including to people who are not consuming their content.

Meanwhile, browser vendors will become uncompetitive - since nobody else can compete against a closed standard - and they become even more motivated to work against openness to maintain their existing oligarchy.

Could not be worse for the web.

9
duncan_bayne 4 days ago 1 reply      
It's worth mentioning that the CEO of the W3C, Jeff Jaffe, is trying to rectify that:

http://lists.w3.org/Archives/Public/public-restrictedmedia/2...

10
girvo 4 days ago 11 replies      
Sigh. Look, I'm okay with DRM, as long as it works on all my devices. EME won't, under linux, I guarantee the DRM Vendors won't bother releasing Linux binaries. That annoys me.
11
alexnking 3 days ago 1 reply      
Maybe instead of getting everyone to adopt Silverlight, we could just make the web more like Silverlight. Like more closed and stuff, because movies!
12
shmerl 3 days ago 1 reply      
> So, the DRM vendors have solved the problem of creating solutions that meetstudio requirements and what we are trying to do with EME is provide aclean API to integrate these solutions with the HTML Media Element.

Which reads as: studios have nonsensical requirements, which are implemented and soon broken. And "we" (i.e. W3C) need to oblige this insanity for the sake of <...>.

Put your own reason, but I bet it won't be good.

13
dschleef 3 days ago 0 replies      
Compliance rules for Microsoft Playready: http://www.microsoft.com/playready/licensing/compliance/

The encryption part of DRM systems is effectively the same as client-side SSL certificates with a secret SSL certificate. How well it's kept secret is defined in the compliance documents. This secret, plus a secure decoding and output path, are the engineering core of DRM systems.

Studios require "industry standard DRM" for movies and TV shows, with lesser requirements for SD. This effectively means "DRM backed by some entity with lots of money that we can sue if things go wrong". Studios approve each individual device that you serve to, usually with compliance targets at some particular future date for various existing loopholes.

Flash (Adobe Access) is somewhat different, and has an obfuscated method for generating the equivalent of a client cert, thus on laptops it's only rated for SD by most (all?) studios. Apparently studios don't care too much about people copying SD content.

Studios would theoretically approve watermarking DRM systems, but there are two major barriers: having a large (ahem, suable) company offering it, and some way to serve individualized media through a CDN. Neither seem likely. So nobody loses too much sleep about whether studios would actually approve watermarking.

14
hbbio 3 days ago 0 replies      
Looks like the W3C may have been the inspiration for Games of Thrones...

Seriously, if there are men and women of honor in this organization, they should stand up against any form of standardization for DRM. DRM can be a proprietary extension for the people who want it.

15
Zigurd 3 days ago 3 replies      
Why should DRM be part of a standard? Aren't plug-ins sufficient?
16
alkonaut 3 days ago 2 replies      
The only benefit I can see from standardizing something is that browser makers who want to claim to be compliant actually have to support it, so you won't end up in the flash/silverlight situation where some platforms don't support it.

But if a plugin framework is standardized, why settle for only DRM? Why not fix the whole crapfest that is plugin applications entirely? A standardized interface to a fast sandboxed virtual machine with good hardware support would be excellent. Currently there is javascript, ActiveX, flash, java applets, Silverlight, NaCl, WebGL and a number of others, each having their own benefits and drawbacks.

If I want to write a web based multi-threadced GPU accelerated webcam-using application that works on any compliant browser on any platform, what do I do? Isn't that what the next kind of web standards should be addressing?

17
duncan_bayne 3 days ago 0 replies      
From the mailing list: "[with EME] ... the publisher will have the possibility of deciding which platforms may access their content."

That was from one of the proponents of EME, touting this as a good thing. The response from another list regular was excellent:

"In non-web-terms this is the publishers deciding on what brands of TV you're allowed to play their content."

That's where EME will take the Open Web. We need to oppose it, strongly, urgently.

18
drivingmenuts 3 days ago 0 replies      
Since many are using Steam as an example of DRM - the important difference is that Steam is a free product, but is not open-source (though it can be used to distribute open-source). It is produced by a company as a means of distributing their products.

It is not even a valid comparison to the blinkard pig ignorance of the secret DRM requirements in HTML, which is an open standard.

I'd just like to know what dipshit at the W3 signed off on this.

19
kevin_bauer 3 days ago 0 replies      
I guess, the "another backdoor" proposal will go very well in Europe, where most citizens are just static about americas view on privacy and respect for constitutional rights. Way to go, maybe the W3C will finally get Europe and the rest of the "free" world to create their own web!
20
pjakma 2 days ago 0 replies      
I download movies and TV shows using Bittorrent and index sites like TBP because of DRM. Often these DRM systems are not available for Linux, or if they are, they require installing some big blob of binary code. It is easier and more secure for me to use bittorrent.

I would happily use the legal services, if not for this DRM. Those services sometimes are even free (e.g. BBC iPlayer). I would happily pay for a subscription service (I pay subscriptions to a number of different of online sites, mostly journalism or data-organistion - I've no problem with that).

The industry standardising proprietary DRM in W3 will just ensure that I continue to support the distributed, end-user provided services which are DRM-free.

21
pyalot2 3 days ago 0 replies      
HTML-DRM, proudly building "solutions" to problems nobody has, by following requirements nobody knows about, to create a landscape of content nobody can play.

Way to go W3C, keep up the "good" work.

22
mcot2 3 days ago 2 replies      
If our end result is to see Netflix using HTML5 video on Desktop browsers, how do we get there from a technology and business point of view? Keep in mind that Netflix has content created and owned by the major studios. If any form of DRM is not the way, than what? How do we get to this end goal? Do we make streams 'free' to copy and rely more on the legal system for protection? We are all keen to slam DRM, but what is a viable alternative?
23
xyjztr 3 days ago 2 replies      
Hey Guys, can somebody create a simple guide, FAQ or something similar for non-tech people to understand what is going on with HTML and DRM? It will help to spread the word.
24
PavlovsCat 3 days ago 0 replies      
Here are some thoughts by Cory Doctorow on web DRM. Spoiler: he's not a fan.

http://mostlysignssomeportents.tumblr.com/post/72759474218/w...

25
aquanext 3 days ago 0 replies      
Can't we just boycott this entirely?
26
jlebrech 3 days ago 1 reply      
why can't they just build it in NaCl and leave the open standard alone.
27
dreamdu5t 3 days ago 2 replies      
What's the problem? Don't support companies that distribute any DRM content. Standardizing DRM and propogating DRM aren't the same thing.
28
Fasebook 2 days ago 0 replies      
The internet was nice while it lasted.
29
silveira 3 days ago 0 replies      
HTML6 = HTML5 - DRM
5
Today I Briefed Congress on the NSA schneier.com
506 points by edwintorok  1 day ago   138 comments top 16
1
bargl 1 day ago 7 replies      
On his blog Schneier comments that this meeting was kept small on purpose. I wanted to see who was present at this meeting so I looked up some of their information. Not hard to find, but here you go. I plan on emailing each of them and thanking them for consulting someone the tech industry considers a Security SME.

Rep. Logfren (Democratic)http://lofgren.house.gov/biography/ Wikipedia - http://en.wikipedia.org/wiki/Zoe_Lofgren

Rep. Sensenbrenner (Republican)http://sensenbrenner.house.gov/biography/ Wikipedia - http://en.wikipedia.org/wiki/Jim_Sensenbrenner

Rep. Scott (Democratic)http://www.bobbyscott.house.gov/biography/ Wikipedia - http://en.wikipedia.org/wiki/Bobby_Scott_%28U.S._politician%...

Rep. Goodlate (Republican)http://goodlatte.house.gov/pages/about-bob Wikipedia - http://en.wikipedia.org/wiki/Bob_Goodlatte

Rep Thompson (Democratic)http://mikethompson.house.gov/biography/ Wikipedia - http://en.wikipedia.org/wiki/Mike_Thompson_(California_polit...

Rep. Amash (Republican)http://amash.house.gov/about-me/full-biography Wikipedia - http://en.wikipedia.org/wiki/Justin_Amash

Edit: Cleaned up one of the links and got rid of a repetitive sentence.

2
tokenadult 1 day ago 4 replies      
Schneier links to a Wikipedia article about Sensitive Compartmented Information Facilities (SCIFs) and explains that he wanted to speak to the members of Congress in such a facility, but could not do so because he is denied access to such facilities as someone without the appropriate security clearances. And that puzzled me, and prompted me to read the Wikipedia article after reading Schneier's blog post submitted here, because I had been told years ago, when I definitely did not have the appropriate level of clearance for a SCIF (as I have never had and probably never will have) that I did have access to a special hearing room for one of the congressional intelligence oversight committees. I was inside the room at the time, with foreign visitors. (I was there as the interpreter for the foreign visitors.) I saw the hearing room back in the 1980s, as I recall, as part of a people-to-people program funded by the United States government that brought over people from other countries to look at how society works in the United States. The look inside the secure room was part of learning about how intelligence agencies in the United States are overseen by Congress. It was represented to us that when the room in the Capitol was closed up, it was impervious to any then-known form of surveillance. So now what Schneier says has me puzzled about whether or not there is any level of secure room between what he calls a "regular room" and a room that a United States citizen can only enter with a very high level of security clearance. It seems to me that there ought to be some kind of in-between room like that, precisely for meetings like the one Schneier just had. That would be better for effective congressional oversight, I think.
3
Zelphyr 1 day ago 3 replies      
"...Congress has such a difficult time getting information out of the NSA that they have to ask me."

THIS PEOPLE! THIS is why the Snowden revelations are such a big deal! We have a rogue branch government with arguably little to no oversight!

4
rdl 1 day ago 3 replies      
It makes sense that you need TS/SSBI and the correct SAP read-in to be inside a given SCIF belonging to a SAP; otherwise one might leave a recording device or otherwise damage the integrity of the SCIF. Plus, the SCIF would need to be cleared of all sensitive materials before you entered.

Clearing a SCIF, letting someone inside, then recertifying it would probably be the correct choice; It could be done, of course, but it's not cheap. (You can also potentially keep the person under observation the whole time, but given the purported Chuck Norris powers of Bruce Schneier, that seems insufficient.)

5
shmerl 1 day ago 0 replies      
This could be almost hilarious if it wasn't so sad. The Congress has to be briefed by someone with access to leaked documents to get an idea what a government agency is doing, because the later "is not forthcoming".....
6
AnimalMuppet 1 day ago 5 replies      
Imagine that the NSA was only doing legitimate, useful, necessary, highly secret things (humor me here).

Can they brief Congress on what they're doing? All 535 publicity-seeking chatterboxes? No way. That's the same as issuing a press release.

So not telling Congress everything is (in principle) necessary. But who are these representatives Schneier briefed? Are they on the Intelligence Committee? If so, and they still can't get straight answers out of the NSA, that's a big deal. (And maybe the rest of Congress is saying that they don't think that the Intelligence Committee has done an adequate job of oversight.)

7
vaadu 1 day ago 1 reply      
There is no oversight except on paper.

You can't have oversight unless you have expertise in the area you oversee.

You can't have oversight unless the overseers can impose immediate consequences upon the overseen.

Without these you are nothing but a spectator.

8
ChrisAntaki 1 day ago 2 replies      
Currently, the Intelligence Committees have special access to top secret information. Every member of Congress should have this.
9
chippy 1 day ago 1 reply      
What's a "SCIF"?

Edits: my guess: (Secure Communications Internal Facility)

10
peterkelly 1 day ago 1 reply      
"Chief, shouldn't we use the Cone of Silence?"
11
blahbl4hblahtoo 14 hours ago 0 replies      
I was just listening to Slate's political gabfest podcast...they talked about the NSA. It occurred to me that the media is still only talking about phone call metadata while the rest of us have been in the weeds of the rest of their activities, which are far more intrusive than metadata...

How do you get that message to people?

13
higherpurpose 1 day ago 1 reply      
The Intelligence committees and the FISA Court need to be overhauled, to start with. Clearly there's not enough/proper oversight of the agency.

I don't know how they do it now, but the Intelligence committees should also brief the rest of Congress at least twice a year, and I think they should be allowed by explicit laws to declassify anything they want in that briefing. No approval from the White House or anyone else needed. They are, after all, the ones that are supposedly in charge with oversight of the intelligence community.

So next time someone like Ron Wyden knows the agency is lying to the public, he should be able to tell the rest of the Congress in the briefing, all by himself (he shouldn't need approval from the rest of the committee), and it should be defined in laws that he's allowed to do that, just so there's no confusion, and no fear of repercussion.

14
Helianthus 1 day ago 3 replies      
It's good to know that we do have some legislators on (essentially) our side, even if their hands are, presently, tied.
15
aantix 1 day ago 1 reply      
>Of course I'm not going to give details on the meeting, except to say that it was candid and interesting.

Why wouldn't he?

16
clubhi 1 day ago 0 replies      
The obvious answer to me seems to have multiple disjoint intelligence committees.
6
Building an open source Nest spark.io
482 points by simonbarker87  13 hours ago   216 comments top 41
1
imroot 7 hours ago 2 replies      
I've just built something very similar to this last weekend -- For around $43/sensor (Raspberry Pi Model B, DigiSpark, and 1-Wire Temperature Sensor) I made 20 of these for my home, farm, and hackerspace for temperature logging. I did this because we're getting another 'polar vortex' next week and the cows don't like it if it's colder than 20 degrees out.

This allows me to measure the temperature inside, outside, and get the relative humidity (not nearly as accurate as the $20 honeywell sensor that they're using, but, it's close enough for my needs). I then built a simple website using mrtg (for temperature trending) and a ruby script that checks the temperatures versus what the set points are and mounted the raspberry pi's in various locations around my places.

My "Controller" nodes are a beagleboard with a 4 or 8 channel relay board attached that allow me to turn on or off the individual controls on the furnace. It works well with my two stage heat pump and fan at my home, but, I need some work to get it 100% at the hackerspace and at the farm.

I mainly did this because I needed something that allowed me to cover more rooms than the Nest (and I'm adding duct dampers and fans to my heating system, so I can selectively heat and cool more rooms to better temperatures).

2
noonespecial 18 minutes ago 0 replies      
One thing I found about hardware is that the prototype is only 10% of the effort. Sourcing components for mass production, government regulatory hurdles, and then that damn enclosure are 90% when everything goes right.

I can build all kinds of things with my arduino and all of those awesome little one-off function boards you can snag on ebay from china theses days. I can't build 10000 of any of them.

3
parfe 11 hours ago 4 replies      
This makes me happy. I have a house with electric heat and eight thermostats pushing Nest costs into unreasonable territory. I'd love to be able to remotely set all my thermostats to 55 degrees or get certain zones to react based on events fired from my phone, (e.g. coming, leaving, charging with screen off aka sleeping, pending alarm)

Unfortunately, with my electric heat the thermostats sit inline with the heater's power source so I need devices that can safely handle 120v.

4
grinich 12 hours ago 2 replies      
Funny-- the first Nest thermostat prototypes were also built with acrylic and wood. (I used to work there.)

I always hoped they would switch back to wood, but it's incredibly hard to do right in mass manufacturing.

5
mrfusion 13 hours ago 13 replies      
I like the use of short 2-5 second videos instead of pictures. They did it tastefully and made it useful.

I never thought I'd see a good use case for auto playing videos.(It kind of reminds me of Harry Potter too)

6
pessimizer 8 hours ago 4 replies      
>we built our own approximation of the Nest Learning Thermostat in one day and weve open sourced everything. In this process, weve come to respect the incredible technical challenges that Nest has solved while also coming to understand how much the game has changed since they first started.

I missed the technical challenges - this seems trivial, and exactly how easy that I would imagine it to be. The only challenge that I see is figuring that people would want a thermostat controlled by a phone app.

Since that's been figured out, I'm going to be very surprised if within 2 years 10 vendors don't have $50 versions sold at Wal-Mart, and there aren't 2-3 different open source software stacks competing to support a few of them.

7
zedpm 10 hours ago 2 replies      
I'm trying to understand if you can self-host the server-side piece of this. I've wanted to have a networked thermostat for a while, but all the ones I found connect to the vendor's server, which is silly. I'd like to be able to point the device at my own server so I have full control.

EDIT: Yep, the Common Questions section of their website[1] says that they'll be releasing an open source version of their Cloud. Awesome.

[1] https://www.spark.io/

8
mwsherman 10 hours ago 1 reply      
I think this is great but there are lessons here from desktop Linux, Facebook clones, etc, which is that retail is hard.

In order to ship a widely used operating system, you need a support infrastructure, consumer research, drivers for lots of hardware, warranties, marketing, payroll, operations, accountants, regulatory compliance. The product is almost the easiest part.

I imagine that Nest understands all this. Putting a piece of hardware in someones house one thats connected to a furnace or which claims to protect against fire means a lot of liabilities, broadly defined.

Id love to see an open source version get to that level of maturity and support. It does happen but it takes a lot of people.

(Tangent, but when I started at Stack, a lot of people said they could (and did) build a clone in a weekend. Sure, as an approximation of the technical product. But that aint the retail product, which is actually comprised of community, goodwill, SEO, quality control, and a lot of other things.)

9
zellyn 8 hours ago 1 reply      
I was under the impression that one of the major challenges faced by Nest was running off only the power available by safely drawing from the existing wires.

Without that constraint, it's a much easier problem.

10
davexunit 12 hours ago 1 reply      
There is a project called GNU remotecontrol that I just discovered that could be used for this purpose. It's important that you can be in control of your thermostat data instead of handing it over to Google/Nest/some other malicious vendor.
11
malandrew 6 hours ago 1 reply      

   "At Spark, were making it easier to bring connected    devices to market with the Spark Core, our Wi-Fi    development kit, and the Spark Cloud, our cloud service for    connected devices."
I found SparkCore on github[0] and the C++ communication lib for Core to communicate with SparkCloud [1], but I did not find SparkCloud itself on Github. Is that component going to open-source as well?

It would be nice if you had the option to host your own cloud service. You could protect your business model at least partially by using an open source license that requires people to change the name if they decide to fork it and productize it, such as the Artistic License v2.

[0] https://github.com/spark/core

[1] https://github.com/spark/core-communication-lib

12
skue 5 hours ago 2 replies      
Are there any specifics on how the underlying Spark platform handles security? On their product page it says that Spark Cloud "creates a secure environment without forcing your web browser and the Core to speak the same language, which would be taxing on a low-power, low-cost microcontroller." Which isn't reassuring.

They don't provide any specifics in the docs either, only this:

"Security is hard. Its especially hard on an embedded system, because encryption is resource intensive. But its also important, because you dont want anyone turning on and off your lights, or worse, locking and unlocking your front doors.We hand-picked a set of rock-solid security protocols that are secure and efficient, so they work great on an embedded system. Theyre baked into the Spark Protocol, which is open source and ready to be extended to other products."

I get that encryption may be difficult on embedded systems, but I would also argue that if a small embedded system can't handle strong encryption then it's not ready to connect devices to the web. I can't find any links to source code - anyone know what sort of encryption they use?

13
excellence24 1 hour ago 0 replies      
this is great but with these devices and 'the internet of things', the most important part is not the devices but the data. And with this, instead of Google getting all the data, Spark is getting the data, and the data is where the money is. And once they get more data and learn to use it, they will become a more valuable company and eventually might get acquired.

I think for something to truly be open-source and beneficial for everyone, everything about it must be open, including the data. The data from all the connected devices globally could be stored on an open database that anyone can access and use. Its one thing to 'learn' with the limited data that one device might generate, but for a machine to 'learn to learn' it should be able to study ALL the data that might be useful.

This kind of organization could lead to a type of opensource corporation where anyone can be an 'employee'. Employment and compensation could be based off a public list of contributions to the project. To each according to his contribution.

This idea could be applied to anything that's used in public and generates data. Autonomous cars, home automation, drones, (NSA data, slightly more complicated but still could be open sourced). But as long as we're tricking ourselves into thinking we need 'money' to survive, the organization or company with the most of it wins.

14
g8oz 10 hours ago 1 reply      
Nice, but I'd rather not be tied to a 3rd party service like Spark Cloud. Indeed thats my problem with Nest. It would be great if it would just connect to my private VPS or something.
15
batoure 10 hours ago 2 replies      
I think that this is a really cool project. But I think that the problem here is still fundamentally the same as the one faced by the nest.

The thesis of spark.io is "you can trust us with your data" not you have control of your data.

The spark is built on a cloud connected platform. even if you can see and control outputs from your board you still exist as part of their ecosystem. Which is basically the functional equivalent of using the dropbox api to build something instead of google drive.

I won't be excited about home automation until someone goes the way of an open protocol for these types of devices that doesn't require a centralized pass through.

Because if history has been any kind of teacher, it shows us that spark.io will probably get sucked up by google or somebody in the near future.

16
emmelaich 3 hours ago 0 replies      
Also have a look at ninjablocks.com; their hardware is open source.

The software is partly open sourced.

http://ninjablocks.com/pages/open-source

17
dzhiurgis 3 hours ago 0 replies      
God damn it. I've just received STM32F4 Discovery board yesterday, tried to run Espruino on it to no avail. And now this thing popped up!

Is it possible to run your software on other dev boards?

Does it have enough processing power for HTTPS POST? I see someone complain here: https://community.sparkdevices.com/t/how-to-send-http-post-r...

18
nilkn 10 hours ago 3 replies      
Any ideas for stuff to build with this besides a thermostat? I'm talking about for a fun side project to learn the ropes, not necessarily the next $3.2B IoT company.
19
strick 13 hours ago 2 replies      
If your next iteration includes a physical switch to put the fan in 'always on' mode, it will already be superior to the Nest.
20
jaredcwhite 9 hours ago 1 reply      
I gotta say -- the use of video in this blog post is outstanding. Best use case of HTML5 Video I've yet come across, frankly. Sorry, I'm supposed to comment on the actual comment...haha. Just saying I love the format. :)
21
coreymgilmore 10 hours ago 0 replies      
I have built a similar system, and expanding it to more devices (think: devices other than thermostats). However, I use my own custom messaging/web server for communicating with the device from anywhere in the world. Think controlling your (ex: toaster) in NYC from LA without configuring any networks, vpn, ports,...aka Nest-like. Combined with some machine learning and machine "thinking", its pretty powerful.

The Spark Thermostat is great minus the fact that you need their web api for communicating with it. But for a 1-day build, how can anyone disappointed! Great job Spark team.

In regards to my own devices, I am definitely going to have to take a look at Spark now. Cool hardware.

22
spyder 8 hours ago 0 replies      
Cannot view the page in Firefox because it's freezing the browser and the memory usage jumps from 155 MB to 880 MB (even with clean profile without add-ons) :(

Edit: It's doing the same in Chrome too

23
boise 7 hours ago 0 replies      
you can buy the components for ~$36: http://octopart.com/bom-lookup/x7lnOrCu

(not including the Spark Core at $39)

24
Aardwolf 9 hours ago 0 replies      
Make it run on mains or USB power, with the battery only being used if the power is disrupted.

After all, one wouldn't want to have to charge their smoke detector every day like a smartphone, right? (Not once a month either).

25
josephpmay 6 hours ago 0 replies      
The site is broken on mobile (Safari). An autoplay video pops up and keeps reopening when closed.
26
aabalkan 5 hours ago 0 replies      
Wow those HTML5 videos totally caused my browser to freeze on a very good hardware.
27
blcArmadillo 9 hours ago 0 replies      
This is pretty cool. For some time I've been interested in building a Nest clone. I like the concept of Nest however it doesn't work for me because my wife's work schedule can't be predicted with machine learning and therefore I think Nest would actually end up being less efficient for us. She keeps her work schedule in a calendar though so my plan was to have the thermostat use her work calendar to optimize our heating/cooling plan. This project looks like it could be a good starting point.
28
analog31 2 hours ago 0 replies      
What are the failure modes?
29
sixothree 8 hours ago 0 replies      
I don't see any information about how they interfaced with the hvac system. Did they use a relay board or optoisolators?
30
lowglow 11 hours ago 0 replies      
If you're interested in building IoT, wearables, and externals, I'm getting an expo + hackathon together called Hackendo (http://hackendo.techendo.co) for April. I would really love the community's support in helping make this awesome, so anyone with experience in this area or feedback on how I should run the event, please reach out.

Also if you're in the bay area, you should check out this meetup group run by my friend Nick Pinkston: http://www.meetup.com/HardwareStartupSF/

31
potench 11 hours ago 1 reply      
How does the thermostat control temperature? Am I crazy, I feel like I'm missing a section on how this device connects to the central air, Ac, heater, fan or something that can affect temperature. Under hardware: "relays to control the furnace and the fan." But I don't see details on the relay.
32
Eduardo3rd 12 hours ago 0 replies      
Documenting a company hardware hackathon this way is super interesting. I think we'll have to give something like this a shot next time we do some rapid prototyping over here. Way to go Spark!
33
650REDHAIR 11 hours ago 0 replies      
This is beautiful. Love your 'Nest'!

Looking back to 5-10 years ago you would have had a really hard time building this in a week let alone 1 day.

34
auvi 9 hours ago 0 replies      
Who knows Google will also acquire Spark some day. They have bought a bunch of robotics companies, Nest and so on. A cloud connected controller, why not?
35
codex 6 hours ago 3 replies      
Why would you want an open source Nest? Is there a market demand for something which is uglier, harder to use, takes more time to install, and works worse? Do you also build your own toasters or automobiles?
36
mistakoala 4 hours ago 0 replies      
That webpage killed my laptop. Presumably the video that did it? So thoughtful of them to play it automatically.
37
serf 11 hours ago 0 replies      
Those videos were way distracting.

Also the firmware definition bugs me.

neat product/concept tho.

38
blueskin_ 12 hours ago 0 replies      
Now Nest is part of Big Google, this is amazing. Time to add this to my projects list.
39
levlandau 11 hours ago 0 replies      
This looks pretty interesting. Definitely heading over to github to look in more detail.
40
baldajan 9 hours ago 0 replies      
very ugly, but very cool (I do like the wood finish though)
41
meerita 10 hours ago 1 reply      
3D printing comes to my mind.
7
US Supreme Court declines to hear appeal by patent troll inc.com
445 points by dded  3 days ago   85 comments top 10
1
grellas 3 days ago 7 replies      
A few thoughts:

1. The Supreme Court declined to hear an appeal by Soverain from an adverse ruling by the Federal Circuit Court of Appeals that had determined the Soverain "shopping cart" patent to be invalid on grounds of obviousness.

2. The Federal Circuit's holding by a 3-judge panel had been remarkable and had shocked patent lawyers generally in that the parties before the court had not even raised the issue on appeal as a ground for invalidating the jury's verdict below. The court raised the issue on its own, concluded that the patent was obvious and invalid, and gave judgment for Newegg in spite of the fact that the jury at the trial court level had found that Newegg infringed.

3. This particular patent had been the original shopping cart patent, dating back to 1994 (well before Amazon began) and it had had a formidable history by which its holder had gotten massive licensing fees from major players over many years for the privilege of using online shopping carts on the web.

4. It is easy to say today that everyone knows what the concept of a shopping cart is and that anyone could have come up with the idea of applying that concept to online shopping. That is all well and good but consider this: not only had this patent passed muster as being non-obvious with the USPTO on its original filing but it had also been found to have been non-obvious on two separate patent re-examinations before that same body and by a string of U.S. district court judges before whom the issue had arisen. In other words, Newegg faced a huge challenge on this issue (the legal standard required that it be able to prove that it was obvious by "clear and convincing" evidence, which is often a tough standard to meet) and this is why Amazon and virtually all other major other online retailers had long since caved and agreed to pay royalties for use of the patent. In the patent community, the Soverain patent was seen as rock solid and one whose shopping cart idea was deemed far from obvious. The top judges and lawyers in the nation, not to mention the USPTO, had all so concluded. The chances of upending it seemed slim to none. And, as noted, even the parties themselves had not raised the issue on the key appeal as a ground for potential reversal. Thus, everyone was stunned when the Federal Circuit reversed the judgment against Newegg on that ground, invalidated the patent, and threw the case out.

5. All that said, when Soverain petitioned the Supreme Court for review of the Federal Circuit's decision, it was trying to undo what it perceived as an injustice done to it as a private litigant ("this is so unfair to us and to our valuable patent"). However, from the Supreme Court's point of view, the kind of petition filed by Soverain is to be granted, and a case heard, only when it has significance far beyond whatever impact it might have on any private litigant. The Court's role in hearing such discretionary appeals is to step in and decide important questions of federal law or to determine who is right when the various lower federal appellate courts may have reached conflicting decisions on such points of law in way that cries out of definitive resolution by the highest court. The Court will not hear cases merely because they might have been wrongly decided unless some such extraordinary factor exists. Thus, in denying Soverain's petition, the Court did nothing more than say that this particular petition did not present important issues of the kind that warranted its attention. It did not validate the Federal Circuit's reasoning or analysis. It did not weigh in against patent trolls. It did not add its authority to the fight against frivolous patents. It simply did what it does on over 99% of such discretionary petitions: it used its discretion to deny it. The legal significance of its decision goes no farther than that.

6. Is Soverain a patent troll that deserved this outcome? Well, its CEO had been a law partner at a major law firm (Latham & Watkins) and the company's business was clearly driven by a legal licensing scheme that had little or nothing to do with active business operations or innovation. It had simply acquired the original company that had come up with the patent back in the day. So, it is a troll if you want to call it that or it is not if you want to use some different definition. But this distinction does underscore how difficult it becomes to analyze patent issues simply by placing labels on the parties. The problem with modern software patents is that too many are too easily granted over trivial "innovations" and this has given vast incentives to those who would package them into shakedown licensing ventures and thereby gum things up for true innovators. It is a situation that calls for action by Congress to rein this in. Otherwise, every party trying to defend itself will find itself, as Newegg did, having to go to extraordinary efforts at massive expense to avoid claims of infringement. Very few litigants can do that and, indeed, Newegg is to be commended for fighting this all the way against tough odds. Let us only hope that systemic fixes can help correct the problem so that this is not the only way available for dealing with such patents. Whatever else this system does, it hardly promotes true innovation.

2
kalleboo 3 days ago 2 replies      
I had to use a secret browsing window to read this without signing up, so here it is for anyone else who has trouble loading the page:

--------

Chalk one up for the enemies of patent trolls: The Supreme Court on Monday threw out a request for trial from alleged patent troll Soverain Software.

The case, called Soverain Software LLC. v. Newegg Inc., is one of three such cases the Supreme Court is expected to consider this year. While the Court will likely hear the remaining cases, which deal with finer points of patent law, its dismissal of Soverain speaks to the potential frivolousness of its claims.

Soverain acquired the rights to numerous pieces of code tied to the online shopping cart, developed in the 1990s. In recent years, Soverain has gone on a litigious tear, suing more than two dozen companies including Amazon, Nordstrom, Macy's and Newegg, an online retailer, which all use shopping carts for internet sales.

Soverain had some success suing on the state level, where a Texas jury awarded the Chicago-based company $2.5 million in damages against Newegg. However, Soverain lost on appeal last year in U.S. District Court for the Eastern District of Texas, which ruled the shopping cart patents owned by Soverain were too general.

Patent trolls typically acquire rights to fallow or soon-to-expire patents with no intention of using the patent. Often patent trolls set up shell companies whose only assets are the patents, which means they have no real revenues or assets. Their sole purpose is to harass small businesses, which usually settle rather than pay for extended and costly litigation.

Patent law was originally written to protect the patent holder, making it easier for the patent holder to prevail in court. For the patent infringer to win, rather, the defendant must prove exceptional circumstances--namely that the patentee acted in bad faith and made baseless claims. This is hard to do. While the patent holder can be awarded "treble damages," or three times the damage claimed, the most the infringer can ever collect is attorney fees.

The remaining cases before the Supreme Court will deal with these finer points.

Congress is examining legislation that would fight patent trolls and their frivolous lawsuits by making them liable for court costs, should they lose their cases.

Small businesses mounted 3,400 legal defenses in 2011 for patent cases, a 32 percent increase over the prior year, according to a research paper from 2012 by Boston University law professors James Bessen and Michael J. Meurer. That cost to small companies was about $11 billion in 2011, also a 32 percent increase over the prior year.

The total median awards to trolls is now nearly twice as high as those to legitimate patent holders, whose median reward fell about 30 percent to $4 billion, according to a 2013 report by PriceWaterhouseCoopers.

3
motbob 3 days ago 2 replies      
"While the Court will likely hear the remaining cases, which deal with finer points of patent law, its dismissal of Soverain speaks to the potential frivolousness of its claims."

I don't think this is accurate. The standard that the Supreme Court uses to decide whether to take cases is not "is this frivolous." Soverain v. Newegg would have to meet a pretty high standard in order to be granted appeal.

I think the author of this piece is reading into this denial way too much. The norm is for appeals to be denied. To be more precise, less than 5% of appeals were granted over a recent one year period. http://dailywrit.com/2013/01/likelihood-of-a-petition-being-...

4
vanderZwan 3 days ago 5 replies      
Good news, but the last sentence of the article made me curious:

> The total median awards to trolls is now nearly twice as high as those to legitimate patent holders, whose median reward fell about 30 percent to $4 billion, according to a 2013 report by PriceWaterhouseCoopers.

I was wondering how they estimated this, so I checked out the report:

> We collect information about patent holder success rates, time-to-trial statistics, and practicing versus nonpracticing entity (NPE) statistics from 1995 through 2012.

> Damages awards for NPEs averaged more than double those for practicing entities over the last decade.

Note: PWC does not use the word "patent troll" - that is entirely the interpretation of the article.

So, just to play the devil's advocate: are NPEs by definition patent trolls? I can't think of a counterargument, but maybe someone else can?

EDIT: Thanks for the enlightening examples so far!

5
dded 3 days ago 4 replies      
I'm encouraged that patent trolls are getting knocked. But my fear is that patent law will hit such a state that only large corporations can wield them. If I'm a small patent holder, and I'm liable for court costs if I lose a suit, then it becomes far too risky to defend my patent against a corporation that violates it.
6
ck2 3 days ago 2 replies      
What did it cost Newegg to litigate that?

Does the troll have to pay legal fees?

Hope Newegg can remain price competitive.

7
csbrooks 3 days ago 0 replies      
I worked on shopping cart software for the web in 1996, and the company I worked at, Evergreen Internet, had been around a while before that. I wonder if anything we did constitutes prior art.
8
shmerl 3 days ago 0 replies      
I hope TQP troll will be busted as well. When will the Supreme Court process that case?
9
incogmind 3 days ago 1 reply      
I think the best way out of these things is make software patents invalid after a short period- like 10 years.
10
revelation 3 days ago 1 reply      
I guess this is why HN mods edit titles on submissions (although the original title is just as terrible). The Supreme Court did not side with anyone; they denied a petition to the court, which is the case for the vast majority of petitions.

If they did accept this particular petition, this would not mean that the Supreme Court sides with the patent troll and the world is doomed; it simply means that the case deals with a contested issue where clarification by the Supreme Court is widely sought.

8
The Great Firewall of Yale 162.209.96.128
382 points by shaufler  3 days ago   126 comments top 39
1
zaidf 3 days ago 7 replies      
I thought my school was bad but reading this makes the administration at my school look like angels. When I launched a similar service at UNC Chapel Hill, the IT dept blocked requests from my server to theirs for scraping latest data.

They claimed I was creating excess load, which is silly because if they really did the math, given how many people were using my service I was probably saving them resources.

2
jahewson 3 days ago 4 replies      
There is no way that a valid copyright claim can be made over the underlying data because it is a statement of fact. Such a work is not eligible for copyright protection.
3
Tossrock 3 days ago 1 reply      
I don't think blocking a specific set of IP addresses constitutes deep packet inspection. If they were reading the payload contents for strings matching the CourseTable site, that would qualify.

Still, this is a stupid move by Yale.

4
girvo 2 days ago 0 replies      
Frankly, if colleges receive public funds, they shouldn't be allowed to claim "copyright" on something like timetable information, in my opinion. Actual intellectual property, maybe, but this? Not a time table. That's just silly.
5
ojbyrne 3 days ago 2 replies      
"Universities are a bastion of free speech." LOL.
6
jamesk_au 3 days ago 4 replies      
One of the principal issues raised here - and not squarely addressed in the post or the article to which it links - is the extent to which average subjective ratings of courses and professors should be permitted to dominate the decision-making processes of students.

Note that Yale's complaint included concerns over "the prominence of class and professor ratings", and the student developers' response was to remove "the option of sorting classes by ratings". Subjective five-point ratings can be useful in many contexts, but in the context of education they can also give rise to genuine pedagogical concerns about the way in which students choose their courses.

Looking at the screenshot in the post, it is not difficult to see that the pattern of enrolments might very quickly become skewed towards those classes with higher average evaluation ratings (whatever such ratings might mean). If that happens, it suggests that some students may be making decisions about the courses in which they enrol based principally on factors other than their interests, abilities and future career paths, or without critical thought. Whilst other factors are relevant, including those for which an average of subjective evaluation ratings might be a plausible heuristic, that does not mean those factors should be the primary or predominant factors.

Without seeking to defend or condone Yale's response, there is more to the story than the tale of student censorship presented in the post.

7
epmatsw 3 days ago 2 replies      
I'm sure no Yale student has ever heard of tethering and that blocking the site on the Yale network will effectively prevent very smart students from reaching this website.

You would think that the Yale administrators would know better than this.

8
dictum 3 days ago 1 reply      
I expect the official explanation to be something like "we cannot endorse an unofficial service that might give misleading information to our students."

Every censor does it from an honest desire to keep this terribly misleading information away from the unknowing masses.

I don't think Yale is blocking the service in a conspiratorial effort to stymie students, but from a not well thought out desire to babysit.

9
jlgaddis 2 days ago 1 reply      
It would have been really cool if the developers of this (really nice, AFAICT) site moved it to (or also made it available via) a Tor hidden service.

The students would regain access to their data (I realize that it has now been e-mailed to them) and it would be a great example of exactly how Tor can help "bypass" censorship.

10
nmodu 3 days ago 2 replies      
If I'm paying $58,000 to attend an institution (rather, if my family is sacrificing $58,000 for me to attend an institution...or,worse yet, if I am taking out $58,000 worth of student loans per year), I should be able to use a course listing service so that I can tailor my academic experience however I chose. THAT is how we open this debate, not with comments about who the proper copyright holder is or whether or not this constitutes as deep packet inspection.
11
shtylman 3 days ago 3 replies      
I run a similar service for other schools (courseoff.com) and I have run into this before. I bet what happened was their site failed to cache the course data or seat information and was thus making lots of requests to the Yale servers. To Yale it might appear like a DoS from this site.

Obviously I don't know for sure but I would venture to bet this block was more an automated response than malicious intent against the site.

12
Nanzikambe 3 days ago 2 replies      
If it were only deep packet inspection, the solution would be simply to prefix https:// and be done with it. As other posters have remarked, I suspect the article means an IP based block.
13
klapinat0r 2 days ago 1 reply      
To focus only on the actual website issue:

Could it be in order to govern the information, rather than "copyright" per say?

My thinking is that, from Yale's perspective, having a 3rd party (and especially a student) be the go-to source for course info might be a bad shift in power.

When it's all in good kind, it may not look bad, and even if it is well intended, there are a few problems that could arise:

- Bugs in crawling code causing some course information to be false, omitted or stale.

- Changes in OCI causing said crawler to keep stale data and fail to update.- Students complaining to Yale with wrong information.

all the way to the more paranoid:

- 3rd party maliciously falsifying information.

- Generel confusion as to which information is reliable, driving students to have a more, rather than less, difficult time finding and verifying class scheduling.

I'm all for net neutrality and strongly against censorship in all forms, but "playing devil's advocate" can't there be a somewhat "legitimate" reason to shut the 3rd party page off for Yale students?

15
jrs235 3 days ago 1 reply      
"They had contacted us warning that we were using copyrighted data" last I understood you can't copyright data or facts [in the US]. You can own copyright to a particular published format though. One can't copy and publish a phone book verbatim but you can certainly scrap a phone book for its data/facts and publish them in a different format.
16
ivanplenty 3 days ago 1 reply      
tl;dr -- the crux of the issue (right or wrong) is making the evaluation information too public. From the news story:

> "[Administrators' primary concern was] making YC [Yale College] course evaluation available to many who are not authorized to view this information,

> "[Administrators also asked] how they [the site operators] obtained the information, who gave them permission to use it and where the information is hosted."

Edit: Agreed, I don't buy these are the real reasons.

17
stormbrew 3 days ago 1 reply      
Something like this happened at the university in the city I live in. There was an apparently awful service for signing up for classes called BearTracks [1] and someone made a scraped version of it that was better called BearScat [2]. Eventually the university basically incorporated the better version into theirs (to, I understand, mixed results).

[1] https://www.beartracks.ualberta.ca/[2] http://www.bearscat.ca/

18
zamalek 2 days ago 0 replies      
> Universities are a bastion of free speech.

Incorrect - universities are now a business, nothing more. You can have your free speech so long as it makes the shareholders happy. Having students confused and lost (or being unable to chose the best education for themselves) is a fantastic way to have them repeat courses in the long run.

Tertiary education is no longer what it used to be. It is now exactly the same type of delusion that women face in terms of having to be slim; or consumers face in terms of having to have the latest iPhone or what have you.

19
dreamdu5t 3 days ago 2 replies      
What's the purpose of Yale censoring certain websites? I find it hilarious that people spend so much money to go to Yale, and some of that money goes to inspecting what they're browsing.
20
diminoten 3 days ago 1 reply      
Is the course listing software open-source? I'd like to do this for another school...
21
TylerE 2 days ago 0 replies      
I forwarded the link to a friend who works in the admissions office at Yale. Can't promise anything but she said she'd be asking some questions.
22
benmarks 3 days ago 0 replies      
The experience seems like fair preparation for the reality into which their charges will graduate.
23
thinkcomp 3 days ago 0 replies      
Harvard did this in 2003. It even went so far as to accuse me of using the word "The" improperly, in a copyright line where I properly attributed credit to "The President and Fellows of Harvard College," when http://www.harvard.edu at the time said the exact same thing (and apparently still does). I left Harvard early (with a degree), and then I wrote a book about it.

http://www.aarongreenspan.com/authoritas.html

Some things never change.

24
ballard 3 days ago 0 replies      
This is an unacceptable, naked abuse of power. Any education institution blocking any site on political or anticompetitive grounds flushes away any vestiges of ideals of free speech and open learning. The administration should have known better or it may find itself replaced for acting incompetently.
25
windexh8er 3 days ago 0 replies      
If anyone is curious that's a Palo Alto Networks NGFW block page. Yale is at least using some great hardware!
26
xerophtye 2 days ago 0 replies      
Wow. This really makes me appreciate what we had at my college. For nearly a decade now, the OFFICIAL portal for the university that lets students and teachers manage courses and assignments (submissions included), has been the one that was originally developed, and still managed by students. We have a webmasters club for that whose responsibility it is keep it up and running and add features to it as they see fit. The university has been nothing but supportive of this, including assigning it an yearly budget for hosting and other expenditures.
28
poizan42 3 days ago 1 reply      
If you actually go to http://coursetable.com you will be asked to login through Yale Central Authentication Service, which sends you to:https://secure.its.yale.edu/cas/login?service=http%3A%2F%2Fc...

I hope I don't give the administration any good ideas here, but I would seem that they have a much more efficient way to disable the site.

29
sgarg26 2 days ago 0 replies      
I understand that Yale and Harvard have a rivalry and compete for students. Out of curiosity, how might Harvard have handled a similar situation?
30
philip1209 3 days ago 0 replies      
Switch it to Cloudflare to obfuscate the source
31
arkinus 3 days ago 0 replies      
Note that this site is also accessible at http://coursetable.com
32
ballard 3 days ago 0 replies      
Has there been an official response?
33
zobzu 3 days ago 0 replies      
it's so disgusting that this stuff even happen.
34
Ihmahr 2 days ago 1 reply      
So MIT murders a student (Arron S.), Yale does some ridiculous censoring...

What's next?

35
takeda64 3 days ago 0 replies      
It looks like http://www.coursetable.com is filtered on WebSense.
36
lightblade 2 days ago 0 replies      
I'm surprised that we haven't DDOS them yet, lol.
37
songco 2 days ago 0 replies      
GFW don't show any "blocked" message, it just "reset" the connection...
38
robitor 3 days ago 1 reply      
"It threatens the very basis of academic freedom and net neutrality"

So pretentious, did a teenager write this?

39
epochwolf 3 days ago 2 replies      
This is not news. Most campus have filtering software and the university administration will use it to block websites that make them look bad.
9
Blackphone blackphone.ch
378 points by jorrizza  2 days ago   202 comments top 69
1
revelation 2 days ago 13 replies      
The privacy issue in smartphones isn't the freaking application processor running Android. Sure, that ones terrible enough.

But the actual problem is the baseband processor running completely non-free software, with an enormous attack surface and access to all the interesting periphery (GPS, microphone). There is not just opportunity to compromise your privacy, Qualcomm and others actively implement such features at the behest of governments and carriers.

Oh, and if you plug that enormous hole, you get to the SIM card, yet another processor that you have zero control over, but which has access to enough juicy data to compromise your privacy. I highly recommend everyone to watch a talk from 30C3 by Karsten Nohl, where he shows a live attack on an improperly configured SIM card that remotely implants a Java app on the SIM card which continuously sends your cell ID (your approximate location) to the attacker by short message (without notification to the application processor, e.g. Android or iOS):

http://www.youtube.com/watch?v=5B7XyVWgoxg

Carriers can do this today. (edit: that's a bit nonsensical, because carriers of course already know your cell id. Anyone with the ability to run a fake basestation momentarily (think IMSI catcher) can do this.)

2
joosters 2 days ago 4 replies      
Completely useless web page. All wooly 'feel-good' words and no hard, concrete information. So I guess we just have to take it on trust then?

Also, their privacy policy is laughable:

We turn the logging level on our systems to log only protocol-related errors - great!

the pages on our main web site pull in javascript files from a third party. This allows our web developers and salespeople to know which pages are being looked at - so instead of keeping your own logs, you are outsourcing this to a 3rd party with worse privacy policies, and who can now aggregate your website usage with other sites.

Why didn't they just keep logging on and get rid of the 3rd party bugs?

3
EthanHeilman 2 days ago 0 replies      
I'd really like a phone that had the following features:

* physical switches for GPS, WIFI, Radio, Camera, Mic, write/read access to disk (go diskless),

* a secondary low power eInk display that is wired directly into the hardware that shows when the last time GPS, mic, camera were turned on (and for how long) and how much data has been sent over the radio and read from disk,

* a FS which encrypts certain files with a key that is stored remotely. If your phone is stolen you can delete this remote key. The key is changed on every decrypt. You also get a remote log of all times this remote key was accessed.

* hardware support for read-only, write-only files,

* hardware support for real secure delete on the SSD,

* the ability to change all my HW identifiers at will (IMEI, SIM, etc),

* a log, stored on a separate SD card, of all data sent and received using a HW tap on the radio/WIFI. The log should be encrypted such that only someone with the private key can read it (public key used to encrypt an AES session key which is rotated out every 5 minutes). If you think someone has compromised your phone you can audit this log for both exploitation and data exfiltration. Since the log is implemented in HW, no rootkit can alter it.

4
buro9 2 days ago 3 replies      
Well, this is just a splash page and says very little.

It's in partnerships with http://www.geeksphone.com/ which is FirefoxOS based. But yet the Blackphone splash has an image of a phone with Android buttons.

They claim no hooks to vendors, so if it's Android I can't imagine this is going to carry the Play store.

I'd be interested in knowing how they will secure and make private the core functionality of being a phone and sending email and text, all of which are insecure.

On that, I'd speculate that this is just pre-loaded with Silent Circle apps, and maybe will be announced as having DarkMail and a choice of RedPhone.

But... there's no info at all really, so who knows what this is.

The only problem they really have to solve is the eternal question of: Is it possible to provide real security and privacy whilst providing convenience?

5
_wmd 2 days ago 2 replies      
As others have pointed out, the baseband is not your friend. Was thinking about this recently, and saw no reason why existing POCSAG (pager) networks couldn't be reused to provide a completely passive receiver. Imagine a phone where the baseband was off by default, unless attempting to make a call. Voicemail/e-mail summaries were broadcast encrypted via POCSAG, and generate notifications just like a new mail summary coming in via GPRS/3G would.

Obviously usability would suffer a little bit (mostly in huge latency when you actually wanted to make a call), but seems like very cheap phone could be built that integrated a pager, allowing complete disconnection from the 'active' radio network, avoiding location tracking by your cell provider, or similar evil tricks by third parties.

6
thecoffman 2 days ago 1 reply      
A site peddling a product that is supposedly about user control and privacy that won't even load without javascript...

The irony is almost too much.

7
Trufa 2 days ago 0 replies      
I agree with the fact that the website is still a little bit unspecific but this project is backed by Phil Zimmermann, he was the creator of PGP, it doesn't guarantee anything but it definitely means some smart people who are worried about privacy are behind it.
8
GrinningFool 2 days ago 0 replies      
https://www.blackphone.ch/hello-world/

I'm sure there's logic there - powering a very basic non-informative landing site with a WP installation that you took the time to customize, but not delete the default post and comment from...

But it certainly doesn't give me warm fuzzy feelings about the people behind this.

9
apunic 2 days ago 2 replies      
Android having the most granular permission system ever seen on any operating system is already the most secure operating system.

The biggest security hole next to the baseband processor and the SIM is the user who installs every app in seconds without checking permissions.

10
wavesounds 2 days ago 0 replies      
Anyone thinking of making a video to sell a privacy product to mass consumers should probably stay away from creepy music and women walking around in all black hoods. Instead go for soccer moms buying stuff with her credit card or librarians doing research for a school kid. Let's not make secure/private communications something weird and creepy but something normal that everyone does.
11
epaga 2 days ago 1 reply      
No mention of the thing being completely open sourced - or did I overlook something? If not, seems like something they should mention (I am assuming it IS open source?)...
12
darklajid 2 days ago 1 reply      
I'm weird enough to be interested in these kind of things, but the whole site is really .. just fluff. Ignoring that and focusing on the sparse details of the actual thing:

- High-End Android device

- Privacy features in the (custom) Android version

- "Secure communication builtin"

Again, I like the idea. But so far the details match CyanogenMod (with TextSecure for SMS, maybe XPrivacy on top)?

13
Duhck 2 days ago 0 replies      
I don't really feel like a slave, maybe I am under reacting here. I am pissed the NSA is collecting data, I am upset at all the recent revelations we have had about data privacy in the last 6-8 months, but I certainly don't feel like a slave.

These products should be advertised on theblaze and infowares.

Sure there is a need for better privacy, but I don't really care for the fearmongering...

14
joncp 2 days ago 2 replies      
Secure? They're rewriting the baseband, then? Color me skeptical.
15
c1sc0 2 days ago 2 replies      
How does this protect me from my carrier? No matter which phone I use they still need to record who I call for "billing purposes" and know which cell is closest to route my calls.
16
andyjohnson0 2 days ago 0 replies      
I know they are pre-launch and this is just a landing page, but it doesn't tell us much. Questions:

1. Is this just a stock phone with some privacy-orientated applications built-in, or is the OS and hardware contributing anything?

2. They seem to be using Android. AOSP or Cyanogenmod? Have they any work themselves to harden the OS? Are they using virtualisation?

3. Any closed binary blobs in there? What about the baseband firmware? (Does open source baseband firmware even exist?)

4. Whats the hardware like? Is it hardened in any way?

17
andyl 2 days ago 0 replies      
I think the Blackphone is a fantastic reaction to the problem of corporate and government spying. It will build awareness of privacy issues, and pave the way for other more secure offerings. A great first step.
18
sdfjkl 2 days ago 0 replies      
To even have the theoretical possibility of "privacy & security", both software and hardware must be fully open. And then there must be some way to check that the hardware and software you got in that box is actually the hardware from the spec, without extra chips. Those are pretty hard to accomplish.
19
runjake 2 days ago 0 replies      
I like Mike Janke and all, he's a nice guy. But, he has backed out of RSAC '14 yet [1]? I find it a tough sell to call yourself a privacy advocate and legitimize and fund RSA by speaking at their conference. It also doesn't help Blackphone's cause.

1. http://www.rsaconference.com/speakers/mike-janke

20
digitalengineer 2 days ago 1 reply      
"and anonymize your activity through a VPN."

iOS and Android support VPN but it needs to be manually activated each time, making it rather useless unless you're using some public wifi. If I understand correctly there is a possibility for large companies to integrate VPN but for the average guy it's rather useless if you have to activate it. If this phone has VPN really integrated that'd be great.

21
andyjohnson0 2 days ago 1 reply      
Renowned cryptographer believes his 'Blackphone' can stop the NSA

http://www.theverge.com/2014/1/15/5310710/phil-zimmermann-si...

22
josefresco 2 days ago 0 replies      
If I desire privacy would I buy a Blackphone, or would I buy another more common smartphone which I would then secure?

If you're "picked up" or detained and you have a Blackphone, or someone observes you using your Blackphone I doubt very much it would help your pricacy concerns.

If however you have a seemingly normal phone it might be overlooked and simply using it wouldn't raise suspicion.

My point is that this type of phone is more for the "regular" person who simply doesn't want to be monitored (as much) and not covert agents looking for a secure phone/platform for communication.

23
bosch 1 day ago 0 replies      
Does any one else find it odd a privacy centric phone's website won't load without scripts, cookies, etc? I would think they would have a text only version if items failed to load properly...
24
sifarat 2 days ago 4 replies      
I would hate to say this, but people here and there, are cashing in NSA fiasco. I would have loved it more, if this was more focused on 'features' than playing with people's emotions. this is valid for everything currently cashing-in NSA issue.

As for, NSA spying how exactly can this phone ensure 100% secrecy. Given a user would have to use the same apps, and above all, the carrier that other smartphone users use.

Point is, US Govt is hellbent on spying on you. And they will no matter what. Either change the US Govt, or suck it up. Nothing else is gonna work.

25
r0h1n 2 days ago 1 reply      
>> "Enabling revolutionary communications"?

Eh? Wouldn't "Enabling secure/private communications" be a better, albeit less grand, descriptor?

26
cyphunk 1 day ago 0 replies      
Will the browser be OSS? Will the mail app? Message app? Maps app? If the essential apps that constitute a "smart phone" are not open source, at least the defaults, it's really irrelevant.

Not to mention that none of the providers have the code to the baseband.

I could imagine a phone that treats the baseband as an untrusted entity and encapsulates everything running over it. This would require forcing SSL for all HTTP traffic, and using some standard for SMS and Voice encryption that is on by default when the recipient on the other end also has a supported device. For those that do not you're unencrypted SMS would be exposed at many hops even if they smartphone were full OSS and trusted, even to the baseband level. So silo'ing everything where possible is a valid solution with closed basebands.

27
tn13 2 days ago 2 replies      
How difficult is it really to make a truly open source phone ? All it takes is one dedicated hardware company and a software company coming together.

Hackers have built some amazing hardware in past and we all know about how open source communities have built some of worlds best software. Google, Apple etc. are building devices where they act as gatekeepers and charge us for all nonsensical stuff. If you make a website there are a gazillion ways to promote it but there is only one way to promote and app. Pay some advertiser and you are totally at mercy of Google or Apple.

Firefox has been doing the right thing so far but they seem to take too much time.

28
yetfeo 2 days ago 1 reply      
Mozilla could take great strides towards this type of phone if they cared. Integrate tor, Whisper Systems RedPhone and SercureText, HTML tracking disabled, etc. I'm surprised their Firefox OS looks and works so much like every other phone out there.
29
oh_sigh 2 days ago 0 replies      
I get the feeling this phone was designed by a marketing group, and not competent engineers. Unless they completely design every chip in the phone, including the SIM and wireless chipsets, the device will never achieve their stated goals.
30
blahbl4hblahtoo 2 days ago 0 replies      
Personally, if I were really worried about privacy I would use burners or get a lineman's handset. It seems like a smart device that you use all the time is going to have the same problems.

So, yeah you can encrypt the voice channel. That's great. You can send encrypted text messages. The people involved are serious cryptographers. All of it sounds good.

You have to ask your self though, what is it you are trying to do? Who is your adversary? Other people here have mentioned it, but what about apps on the phone? Facebook is still Facebook.

31
unicornporn 2 days ago 0 replies      
No Play store in this I hope. I'm currently running Cyanogenmod without Gapps and I'm wondering what this will offer me.
32
avighnay 2 days ago 0 replies      
Geeksphone is doing pretty impressive for a startup that they were launch partners for Firefox OS and now have roped in PGP founders for this project.

Were they successful in delivering on the Firefox phones?, Their website always says 'out of stock'. Blackphone seems to be ambitious too. Is it possible for a startup to sail these two boats?

Also I find it odd that the PR is always just before the Mobile World Congress (MWC) which happens in Spain, last year with Firefox OS and this year with Blackphone

33
fmax30 2 days ago 4 replies      
This maybe a bit off topic but,why did Switzerland get the .ch domain instead of china.China seems to have a lousy CN domain ,( which reminds me of cartoon network for reasons that are irrelevant here).
34
MWil 2 days ago 0 replies      
I thought it was funny, considering the top comments, that if I cntrl+F for "zimmerman" it takes me all the way to halfway down the page
35
rch 2 days ago 0 replies      
This is not the 'first' phone to do these things. I had an idea along these lines in 2003, and some searching turned up a German company that was already doing it. Somebody bought them a couple of years later, and I don't know what happened to the phone. This sure isn't the 'first' though.
36
whizzkid 2 days ago 1 reply      
With all the respect what they have done so far, I can't see any reason why this is securer than the other mobile phones..

With the latest NSA stuff, I came to conclusion that a true secure system can only be built under these conditions and just to put it out there, this is just my opinion;

- A computer company that manufactures their own hardware such as hard drive, ram, cables, network cards.

- An OS that is newly written and not based on any other existing operating systems.

- Building the whole system with INDEPENDENT hardware and software mentioned above.

- Keeping the mobile device's source code offline from Internet as much as possible

These are just the first steps on developing a secure system, then comes the mobile network architecture and encryption etc.

I admit, it is not an easy job but, trying to develop a secure system with "not secure" development tools is not the right way to go :)

37
huhtenberg 2 days ago 4 replies      
For a project concerned with privacy and anonymity the news subscription form is asking way too much.

Also, why is domain on .ch ?

38
djyaz1200 2 days ago 0 replies      
Will someone please tell them to remove the clips in their video of testing a white phone in the interest of brand consistency? Also this idea seems like a solid game plan for Blackberry? They could rename their company "Black" ala P-Diddy v just Diddy. :)
39
aagha 2 days ago 0 replies      
It's interesting that all the work being done on this "secure" phone is being done on non-secure hardware and networks. Presumably if interested parties think this is a threat, they can access all comms/data about this new phone, inject themselves where they see fit and compromise the final product.

Oh, and never mind compromising the people involved.

40
pieter_mj 2 days ago 0 replies      
True privacy on a smartphone can only be expected when software and hardware are 100% open sourced. This of course includes the source code for the 3 Os's that typically run on a smartphone.Anything that's running server-side cannot be trusted either. So we need client-side encryption/decryption as well.
41
bybjorn 2 days ago 0 replies      
Looks like there will be several players in this market - an alternative is Indie Phone; http://indiephone.eu .. If it ever ships it should be a better alternative privacy-wise as they are building everything from the ground up (their own OS instead of relying on Android, etc.)
42
elwell 1 day ago 0 replies      
While I see the reasoning, the name "Blackphone" just has too much of a racist connotation in America.
43
pessimizer 2 days ago 0 replies      
How usable is Android without a continual involvement with Google? If you have to be involved with Google, there's no point.
44
tinalumfoil 2 days ago 0 replies      
Does anyone else see this as ridiculous attempt to profit off the NSA leaks. The video is about scaring people into believing their being "enslaved" and are coming out with a device that has "never before before created" that is aimed at "for privacy-minded, security-minded people". It's filled with unrelated words like "neutrality", "all walks of life", "innovative thinkers" to make it seem legit.

There is no mention of the methods used by the phone to ensure privacy.

45
lispm 2 days ago 0 replies      
I stopped watching the video at 'Android'.
46
JoelJacobson 2 days ago 0 replies      
Would it be possible to do the encryption outside of a normal phone, via some AD/DA converted plugged into the standard 3.5mm-headphone minijack?

I started a thread to discuss this idea:

https://news.ycombinator.com/item?id=7066792

47
BuildTheRobots 2 days ago 0 replies      
Love the idea of a GSM handset that believes in protecting my privacy, however all their features seem to revolve around a secured Android OS.

Does anyone know if the actual baseband/wireless side has been designed with security in mind? -for example I'd love to be warned when I'm connected to an A5/0 "encrypted" GSM network, but I haven't been able to find a handset build in the last decade that's willing to warn me.

48
ilovecookies 2 days ago 0 replies      
Isn't the problem more connected to the hardware and the fact that most people are already willingly using tons of applications who are giving information about you to companies like google (maps) twitter, facebook etc. If you install the apps with consent on your phone, and those apps have access to the linux or ios kernel runtime and syslogs then you're basically fucked from start.
49
viseztrance 2 days ago 0 replies      
I would personally be interested if they would provide security updates over a long period of time.
50
blueskin_ 2 days ago 0 replies      
Please not another long scrolling page without any real info... shame, I might have wanted one if they had provided any specs or technical details at all...
51
linux_devil 2 days ago 0 replies      
One should be concerned about privacy and digital footprints , but more or less it depends on how many people are looking forward to adapt this concept. People still use Gmail and facebook .
52
dblotsky 2 days ago 0 replies      
"You can make and receive secure phone calls; exchange secure texts; exchange and store secure files; have secure video chat; browse privately; and anonymize your activity through a VPN."

People. It's really secure, private, and anonymous, ok?

53
jlebrech 2 days ago 0 replies      
nowhere near as secure as a burner phone purchased in cash.
54
naithemilkman 2 days ago 1 reply      
Isn't this kinda moot if you're using any services that is domiciled in the States?
55
muyuu 2 days ago 0 replies      
I loved it when they asked for my full name to keep me informed.
56
caiob 2 days ago 0 replies      
Funny how there's a twitter link at the bottom.

Jokes aside, I think it's a great initiative, looking forward to see what comes out of it.

57
sgarrity 2 days ago 0 replies      
They should probably work on the mixed-content SSL warnings on their own website. It's obviously not related to the security of the phones, but it doesn't instill much confidence.
58
sidcool 2 days ago 0 replies      
Is it an Android phone?
59
heldrida 2 days ago 0 replies      
The phone image is missing. Check "images/teaser_site/img03.jpg", css #phone style.css line 396

Thanks

60
junto 2 days ago 0 replies      
> Use the apps you know and love.

Ok, so how do they stop Facebook et al from abusing our contact lists and location data as they do on existing smart phones?

61
arj 2 days ago 0 replies      
Unless they have some really special hardware in this, I don't see how its that much different than running cyanogenmod + secure applications on top, such as textsecure.
62
drjacobs 1 day ago 0 replies      
Ouch, don't try this one on a slow connection.
63
pattle 2 days ago 0 replies      
The website doesn't really tell me anything about the phone.
64
dandare 2 days ago 0 replies      
I am not getting it, how do you prevent the carrier from knowing where you are if you sign up to it with your number?
65
skuunk1 2 days ago 0 replies      
Too bad they couldn't get the url blackphone.sh

;)

66
blackphace 2 days ago 1 reply      
Their trailer seems a little too "inspired" by this First ELSE promo video from 2009: https://www.youtube.com/watch?v=ZHghZnOH8dA
67
higherpurpose 2 days ago 1 reply      
Since NSA/FBI can reroute shipping boxes and install malware in them - do they have any plans against that?
68
n008 2 days ago 0 replies      
Just get an old Nokia feature phone
69
hekker 2 days ago 0 replies      
It would be nice to order Chinese food anonymously with this phone. Looking forward to the release!
10
Yale students made a better version of its course catalog. Yale shut it down washingtonpost.com
364 points by zt  1 day ago   112 comments top 38
1
tikhonj 23 hours ago 3 replies      
Some Berkeley students developed a similar service called Ninja Courses[1]. This lets you browse through classes, order textbooks and also shows ratings. (Although all the ratings are submitted by Ninja Courses users, I believe.)

It can even automatically build a schedule for you by choosing lectures and sections that don't overlap, optimizing based on user preferences. For example: do you want more morning classes, more afternoon classes, more gaps, less gaps, some days off... This automates away a rather tedious part of choosing your schedule--something I haven't seen in other similar tools (although I haven't looked too closely).

Instead of shutting it down, the university used it to build an official Schedule Builder[2]. The official version doesn't have rankings, but exposes other interesting information--in particular, grade distributions.

Since then, Ninja Courses has expanded to a bunch of other UC campuses as well.

Just thought I'd share a nice success story to counterbalance most of the others :).

[1]: http://ninjacourses.com/

[2]: https://schedulebuilder.berkeley.edu/

2
orf 1 day ago 5 replies      
My University released a web based timetable system that was absolutely shocking - it was an ASP.net based site with the kind of unfriendly interface you expect from a 1990's era intranet site (hint: utterly utterly horrible, it produced one timetable PER MODULE rather than a single combined timetable and every time you selected a module it would refresh the page). I was hungover the day before term started and rather than use that system I hacked together an easier to use alternative from my bed using python + flask in literally 60 minutes and released it[1].

The administration went crazy when hundreds of students started using it and asked me to take it down. While its not as impressive as something made at Yale it seems to be a common theme, I wonder if other universities suffer the same issues. Could be a market opportunity.

[1] http://timetables.tomforb.es/

3
mgkimsal 23 hours ago 2 replies      
Not at all surprised. "Big campus" is another entrenched "big industry", and disruption is hard. I'm in touch with students every so often that have essentially the same ideas - "let's make it easier for students to do XYZ", where XYZ always involves campus data and/or integrating with campus system. It's damned near impossible, and the reasons are many. Some are valid, some are invalid (obviously, these are my own personal views, nothing more), but the main takeaway is as with most problems, the core issue is not technology, it's politics.

Students who have these "we could change the world!" ideas rarely understand that they're not the first person to think of idea X, and that the issue is who you know who can pull the right strings.

4
obblekk 1 day ago 6 replies      
Academia, and the Ivy League in particular, prides itself on being rigorously open, on providing broad access to all knowledge to advance intellectualism. In this regard, academic institutions often attack governments and corporations that intend to limit its free speech, but on the rare chance that this spirit of openness works against the institution, they take a hard liner approach to censoring it.

The basic reason why Yale doesn't want this information easily accessible is because it would force them to acknowledge that many of the staff are brilliant researchers and absolutely lousy teachers.

This notion that thinking freely about tough problems, and having the ability to pass your knowledge down the generations go together isn't true in the hyper-competitive academic environment today. If a professor doesn't publish, he/she can lose tenure, but if a professor doesn't teach well, he/she can have more time to publish.

I think the best universities to learn are those where the professors truly want to teach. Where their research isn't an end in itself, but a way to inspire questions for another generation. Yale seems to be asserting that it's not in this category.

5
bonemachine 5 hours ago 2 replies      
Officials also expressed concerns that the site was making course evaluation information available to individuals not authorized to view the information.

Sounds like they have perfectly valid concerns, actually.

When it comes to protecting restricted (and potentially rather damaging information) -- which also happens to be the University's property, anyway -- it's generally prudent to suppress the source, and ask questions later. Taken out of context, even a single vindictive or poorly composed review can be fatal to an instructor's career. Failing to take action to suppress their unauthorized distribution can also invite nasty lawsuits.

So this doesn't seem to be about "the man" stepping in and quashing student creativity and initiative (as much as some may prefer that narrative). If anything it seems that Yale did the right thing, in this case.

6
thetwiceler 13 hours ago 2 replies      
Let's not blow this out of proportion. They scraped private data (that required a Yale login) without permission and accidentally made it available to a wider audience than intended.

Yale has historically been very supportive of these things. A couple of years ago, they acquired another very slick course catalog website, Yale Bluebook, for a good chunk of money [1].

[1] http://yaledailynews.com/blog/2012/08/01/university-acquires...

7
x0054 1 day ago 1 reply      
How about open sourcing the tool. Release the tool as a self hosted solution, and let any one who wants to host it, host it. Or just use for their own purposes. I am not sure if that could get them in trouble, as I am not sure what crazy provisions Yale has in their student honor code. To be safe I would obfuscate my connection to the open source tool, if I were a student.
8
acangiano 1 day ago 5 replies      
My money is on a professor being pissed off about poor reviews received.
9
sheetjs 1 day ago 0 replies      
Were there any updates since the last discussion http://news.ycombinator.com/item?id=7060261

(http://162.209.96.128/)

10
kriro 16 hours ago 0 replies      
Back in the day when I was a student, some other student built a php based support forum with uploads for stuff you wrote during lectures and tests ordered by professor and ratings, discussion about lectures and test prep etc.

It was the de facto meeting and exchange place for all things related to that degree and was great value, I skipped quite a few classes that had good material and opted for self study. Must have saved me lots of hours. The most interesting thing was that there wasn't much pure leeching. Most people that used it went out of their way to help new students, provide lessons learned etc.Some professors actually had accounts (sometimes "undercover", was always fun when they got "exposed")

The guy started it when he started his BA (equivalent) and wrote his MA thesis about it and then the side faded away when he graduated (but by then there was other stuff available anyways)

Wasn't sophisticated at all but man was it useful. Talk about identifying customers and fixing their problems :)/old story

11
DanBC 18 hours ago 2 replies      
Sometime ago a man made a better version of the Odeon's (a UK cinema chain) website. His version was accessible but also mu h easier to use. People could use his site to buy tickets from Odeon. They shut it down and stuck with their terrible site. I'm on mobile and finding links is frustrating, but here are a few.

http://www.ntk.net/index.cgi?b=02003-07-25

He did some other sites too and as far as I know they all got taken down pretty quickly.

It's interesting to see some history of web scraping and how much people protect their, well, i don't know what is being protected there.

12
dasil003 17 hours ago 1 reply      
With all these negative anecdotes I thought I'd share a counterpoint.

In 2001 at my first professional job I was the web manager for the student unions at the University of Minnesota Twin Cities campus. At the time there was an old unmaintained ColdFusion app for student group registration. Having recently been dabbling in PHP for the unions websites, I proposed writing a new student group registration site from scratch in PHP. My manager approved, I gathered requirements from the stakeholders in the office for student activities and 3 months later shiny new mysql-backed PHP registration system.

They also did cool things like let me open source the custom CMS I wrote and push forward with a standards based HTML template while the rest of the University was still on a standardized but antiquated table-based template.

I realize now that I've been incredibly lucky with the people I've had above me in every single organization I've worked for in the last 15 years.

13
bertil 1 day ago 0 replies      
One of the author posted on Reddit that they had been contacted by the administration and their project was being re-considered.
14
sammcd 21 hours ago 0 replies      
Did the same thing at my college. We crawled their course catalog. They blocked our IP.

Our big addition was being able to sort classes by time and day, the university ended up adding that feature, so I guess we got what we wanted?

15
don_draper 18 hours ago 1 reply      
Will someone please disrupt Academia? Tenured professors don't want you to see that many suck at teaching so any website that brings that to light is shutdown.
16
sirkneeland 5 hours ago 0 replies      
I'm sorry, was someone under the impression that Yale or any other institutions of higher education are progressive, cutting-edge institutions employing best practices in IT or cost management?

These are reactive, hidebound institutions that have been able to go for at least a century without transformative reforms.

They're rather overdue for it.

17
Fuxy 16 hours ago 1 reply      
Typical American double standard. We promote free speech but don't you dare use it in a way we don't approve.

Is that really free speech then? I'm not free to say/do anything you don't approve of.

And since when is it ok to threaten disciplinary action on a student that didn't do anything wrong just provided a service students desperately need.

It's like me threatening to shoot your family if you don't sell me your house.

The second one is clearly illegal this one for some reason is ok can someone enlighten me?

We're talking about these people's future in both situations?

18
smsm42 17 hours ago 1 reply      
So Yale students learned an important lesson - some of the staff cares much more about looking good than about helping students, and the administration of Yale would not be on the student's side in this case, because students come and go and the teachers stay.
19
swombat 18 hours ago 2 replies      
Ah, embracing innovation the British way.

I recall that 6 months before one of those "date the person right next to me" sites picked up in the US, something similar was launched by some students in a UK university (a pretty high-profile one, though I can't remember which one... LSE? Imperial?). Anyway, long story short, the university IT department shut it down on the grounds that it was not appropriate use of IT facilities.

Six months later, the same launched in the US and grew insanely fast.

20
mathattack 5 hours ago 0 replies      
Is the issue privacy, or making it too easy to compare faculty ratings with one another? In general faculty and class ratings are not popular with tenured faculty who view greatness as synonymous with citations. Tenured faculty also are fighting the "dumbing down of classes to improve ratings". (I'm not sure if this is a reality or not at places like Yale)
21
rcfox 23 hours ago 0 replies      
I had a similar experience at my university (University of Waterloo), which I discussed some time ago: https://news.ycombinator.com/item?id=3682163

In my case, it was internship postings instead of a course calendar, and I was actually "punished" for it.

22
xacaxulu 22 hours ago 0 replies      
These brothers will be successful in spite of Yale. A story like this definitely lends some cred when it comes to hiring time.
23
CalRobert 16 hours ago 0 replies      
The more I reflect on my time at university, the more I realize it was an utter and complete scam. Fortunately I was lucky enough to attend junior college for a year and a half after graduating, took classes that I actually wanted to take instead of whatever BS was prescribed by my program, and didn't deal with manipulative twits for professors who held me hostage, forcing me to do their research for them before they'd let me graduate. The sooner academia dies, the better. I certainly won't shed any tears.
24
izietto 14 hours ago 0 replies      
[XKCD] University Website http://xkcd.com/773/
25
jimbokun 11 hours ago 0 replies      
My simple question, what has the Yale administration gained by doing this?

Now the Washington Post has picked up the story, making them look petty and childish. Does whatever point they are trying to prove offset the PR damage?

26
nicholaides 23 hours ago 0 replies      
I and some friends made a similar piece of software for our university. They reluctantly started emailing us CSV dumps of the course schedules but eventually stopped. It's surprising how resistant to technology a university with a reputation for engineering can be.
27
kmfrk 19 hours ago 0 replies      
This bring back memories. I did the same thing at my university way back when - especially back when people still didn't bother to develop for iPhones.

I imagine they were afraid that people would get the wrong information, which is fair in a way, but if Yale's website was anything like my university's, then the website would far outweigh the alternative.

28
Cyph0n 17 hours ago 0 replies      
I actually built a similar, albeit simpler, course scheduling website for students of my university, UAEU. I however couldn't improve it because I feared that it might catch the eye of one of the higher-ups and lead to my expulsion. So I've left it as is. Quite a few students are using it, which is more than enough for me.

Here's the site (no domain yet):

http://nameless-shore-4042.herokuapp.com/

29
zoowar 7 hours ago 0 replies      
Just give us the data and let us decide which application presents it in a form we can consume efficiently.
30
mikekij 21 hours ago 0 replies      
Rename it "Stanford Bluebook+", give Stanford a 4 year license, and transfer.
31
thinkcomp 21 hours ago 0 replies      
Seriously, reading this article it's almost as if they're deliberately re-enacting Harvard's response to houseSYSTEM (which included course reviews, scheduling features, and of course, The Facebook) from 11 years ago. It really says something about institutional behavior. There must be some kind of Independent Thinking Students Emergency Playbook they hand out in elite university administrations.

As then-President Summmers said on September 15, 2005, We are a community that is committed to the authority of ideas, rather than to the idea of authority. What a perfect line to summarize their utter hypocrisy.

http://www.aarongreenspan.com/authoritas.html

32
edelans 17 hours ago 0 replies      
We used a quite similar tool (although much more MVP) in my business school in France (ESSEC): we would share a google spreadsheet where we would comment and rate our professors and classes. Although the administration knew about it, they wouldn't accept it officially, which is quite disapointing in my opinion. There are more than 10 years of data in there: the link is transmitted to every promotion.
33
darkhorn 16 hours ago 0 replies      
Similar web application but it was not punished http://www.metutakvim.com/
34
rvac 18 hours ago 0 replies      
Once you understand that these universities are more or less (publicly or privately owned) corporations, their behavior starts making more sense. As far as I know, many of them actually turn a profit.
35
0ptical 20 hours ago 0 replies      
Oberlin did this, too - but the department supported it and I got credit for working on it. https://oprestissimo.com/
36
dzink 1 day ago 3 replies      
Why not just limit access to validated students?
37
jamdavswim 21 hours ago 0 replies      
Say anything you want, as long as it's positive.
38
iaygo 16 hours ago 0 replies      
Is lecture attendance compulsory at Yale?
11
Super successful companies samaltman.com
363 points by dko  2 days ago   145 comments top 40
1
kevinalexbrown 2 days ago 0 replies      
Mediocre founders try to hire people for the parts that they don't like. Great founders just do whatever they think is in the best interest of the company, even if they're not "passionate" about that part of the business.

I don't often like to follow people, but when I do, this attribute matters to me. I'm not sure, but I suspect that it's a signal that someone wants to build something great more than accumulate accolades. In the latter case, I help someone get rich and famous (which is fine!). But in the former, I'm a part of something awesome.

I find it a little difficult to articulate why this affects me so deeply, but it resonates with me, and talented people I know.

In general, I suspect these attributes are important if for no other reason than talented people you might be recruiting will be on the lookout for them.

2
carsongross 2 days ago 9 replies      
C'mon now, kids.

While there is some laudable-if-extremely-conventional wisdom in here, almost every point suffers from either survival bias ("Successful companies succeed by being successful.") or from having obvious counter-examples ("Was Steve Jobs a nice guy?"), or both.

3
exit 2 days ago 12 replies      
> - The founders are nice. I'm sure this doesn't always apply, but the most successful founders I know are nicer than average. They're tough, they're very competitive, and they are ruthless, but they are fundamentally nice people.

ruthless, rooTHls/, adjective

1. having or showing no pity or compassion for others.

what does "ruthless" mean nowadays in valley newspeak? or did i miss a memo about "nice"..?

4
aashaykumar92 2 days ago 0 replies      
Another important trait Ron Conway highlighted at Startup School that really stuck with me was that Founders of 'super successful companies' can careless about other distractions, especially the media. He highlighted the example of Ben Silbermann and how he used to reject several interviews so as not to lose focus on Pinterest.

So although it's a hybrid of two traits mentioned in the article ("They are obsessed with the quality of the product/experience" and "They don't get excited about pretending to run a startup"), it is still one that should be there by itself.

5
tvladeck 2 days ago 3 replies      
To all those people screaming "survivorship bias", isn't that what this whole post is about? Like, explicitly in the title of the thread?

Of the companies that became very successful, what were some common traits? That's what this post is about.

"Yeah but you only looked at the successful companies."::smacks head::

6
argumentum 2 days ago 0 replies      
Great article ... Some of the points might seem obvious in hindsight, but are rarely followed in practice. Perhaps it is because they are so obvious as to be somewhat invisible, or they are repeated so often as to be ignored.

I thought one of the less obvious, and especially interesting, insights was the following:

Another way this trait shows itself is "right-sized" first projects. You can't go from zero to huge; you have to find something not too big and not too small to build first. They seem to have an innate talent for figuring out right-sized projects.

I wonder how much of this talent (which I'd call a "knack") can be acquired from experience.

I noticed in myself an instinct for categorizing projects as "right time"/"right place". By "instinct" I don't mean I'm particularly good at this, I may be really bad. I mean that I have a feeling, which I can imagine as some sort of neural pattern recognition algorithm. I'd guess everyone has this same feeling .. to what extent can it be tuned into a "knack"?

7
tomasien 2 days ago 1 reply      
"They don't get excited about pretending to run a startup"

That's the biggest difference between me now and starting my first company - I was ultra-excited to call myself "CEO of startup" the first time around, this time around we don't even have titles because we don't give a shit. We have a CEO because you need to know at whom the buck stops, but we've even discussed using that interchangeably depending the situation (decided against it) because of how thoroughly we don't care.

I don't think the way I felt the first time around was bad or the reason we failed - but it was a pretty good signal, and I even knew it at the time.

8
ritchiea 2 days ago 0 replies      
This started out with some boilerplate SV platitudes (obsess over your product, obsess over talent), but got quite good by the end. The concrete items, particularly "They make something a small number of users really love" and "They don't get excited about pretending to run a startup" really resonate with my experience and are things I think everyone should keep in mind when running a company.
9
edw519 2 days ago 0 replies      
Funny, seems like I always narrow my list down to this 1 item:

"They are obsessed about their customers' success."

Everything else is a byproduct.

10
calbear81 2 days ago 0 replies      
I think a lot of the points are great but when we talk about super successful, I can think of some great startups that grew organically but also a lot of operationally efficient companies that knew how to build businesses at scale with a combination of organic growth, massive marketing, and smart PR. For example, one of Google's largest advertisers in spend is Booking.com (owned by Priceline). They have built an efficient machine that can acquire users/customers in an efficient manner and in the process capture 48% market share of online hotel bookings in Europe and help make Priceline a $60B market cap company.
11
_sentient 2 days ago 2 replies      
Great points. I would also add: They have serious intestinal fortitude.

You can possess all of the other traits, and you're still pretty much guaranteed to run into numerous points in the life of a company where you're staring into the abyss of imminent failure. The ability to withstand that kind of pressure is probably a prerequisite for highly successful founders.

12
normloman 2 days ago 2 replies      
SURVIVORSHIP BIAS. SURVIVORSHIP BIAS. SURVIVORSHIP BIAS.
13
oskarth 2 days ago 1 reply      
All I can think about reading the comments (and the article, for whatever reason) is pg's advice to sama and how it didn't seem to help at all here: https://news.ycombinator.com/item?id=6843726
14
jusben1369 2 days ago 0 replies      
Good stuff. Only nitpick is the "partnership" comment. I actually think too many startups are too insular and not comfortable working with other companies early on. However perhaps here he meant the "hit it out of the ballpark" partnership type.

Please add to "They don't get excited about pretending to run a startup" - they're not on Twitter tweeting cliche's around vision/team/culture/design/customer love all day. That's a big red flag to me.

15
vzhang 2 days ago 5 replies      
Number one reason you didn't mention - they are lucky.
16
inthewoods 2 days ago 0 replies      
I think there are two important areas he doesn't touch on:1. Timing: The most successful companies are also the ones that are usually at a particular point in time when an opportunity exists. Too early, you fail, too late, you fail as well (for different reasons).2. A scalable idea: The most successful companies find product or service ideas that scale hugely. These ideas are pretty rare. More often, people find ideas they like and can execute but turn out to be ideas that scale only so far. Sometime it turns out that they've actually founded a services business and didn't know it. More often, they find an idea that can scale to, say, $5m in revenue but then has trouble scaling beyond that. Billion dollar ideas likely represent a very small portion.
17
Jormundir 2 days ago 2 replies      
Is there any evidence backing up these claims? I can think of exceptions for just about every one of these points.
18
tdumitrescu 2 days ago 1 reply      
I like this one: "They respond to emails quickly." It always amazes me just how many would-be "founders" are unbelievably flaky, miss appointments, drop off the face of the earth for days or weeks at a time...
19
aryastark 1 day ago 0 replies      
This article is literally advocating a cargo cult. I've seen sharks jumped, but never this high and this obvious.
20
lpolovets 2 days ago 0 replies      
"Great founders are execution machines." That's a great summarizing quote.
21
sillysaurus2 2 days ago 0 replies      
This is an excellent list. Thank you, Sam, for putting it together.
22
saumil07 1 day ago 0 replies      
Sam Altman is a great writer. He was also Founder/CEO of Loopt. I have to wonder why this post doesn't relate the traits back to his work at Loopt so the lessons learned can be more contextualized.

For example, are there areas where he was a mediocre founder by his own definition? Were there times when he was mediocre and then became great? What did it take to go from mediocre to great?

I'm surprised that the essay is so general in nature when there's a wealth of specific (and maybe more valuable) cases that could have been shared even after respecting privacy of individuals involved, etc.

23
semerda 1 day ago 0 replies      
I like Sam's blog posts. They are short and punchy. Always leaving me charged with energy. Good stuff. Keep it up Sam!

"They have a whatever-it-takes attitude." - This is such a powerful trait that it puts the "big dreamers" to shame and separates the Wannapreneurs from Entrepreneurs. Anyone is capable of dreaming, talking big, generate ideas et al.. but few are capable of executing them by doing whatever it takes to turn that dream into reality. Ha, it reminds me of the "never give up frog poster".

24
tsunamifury 2 days ago 0 replies      
These are great foundational traits, which are common among noble-failure and successful startups I've seen.

You still need to add in the resources (money, energy, charisma) to fund sustained hard work, market resonance, and luck to reach super success.

25
dclara 2 days ago 0 replies      
Wow, so many traits!

I noticed this one is not quite easy to make:

"*They grow organically. And they are generally skeptical of inorganic strategies like big partnership deals and to a lesser extent PR. They certainly don't have huge press events to launch their startup. Mediocre founders focus on big PR launches to answer their growth prayers."

Most startup companies are looking for big partnership deals with PR support intensively. But they are focusing on building customer base. It's really not easy.

26
wslh 2 days ago 0 replies      
He misses one point: live and fund your company in US. There are more successful Internet companies in US than abroad.
27
hoboerectus 2 days ago 0 replies      
* They can bench press twice their body weight.

* They are sublime swordsmen.

* They revere the supreme commander.

28
bsirkia 2 days ago 0 replies      
I would add they were also a bit lucky (whether with timing or virality or some other factor) at some point in their lifecycle.
29
pbreit 2 days ago 1 reply      
"Charging customers early" is actually the opposite. Few of the big internet successes charged early (google, eBay, PayPal, Facebook, yahoo, etc.).

And, frugality is good in the beginning but after you prove yourself, you have to step on the gas.

30
tschellenbach 1 day ago 0 replies      
With many of these points I completely agree, I also believe that there's data to back them up. The following however:

- the generate revenue very early on in their lives- they keep expenses low

Is in direct contrast with many of the most successful startups. As far as I know, Google, Facebook, YouTube, Instagram, Skype don't fit this criteria.

31
zephyrnh 2 days ago 1 reply      
"They generate revenue very early on in their lives. Often as soon as they get their first user."

Is this true?If "super successful" can be understood as "the biggest tech IPOs of the last 15 years", then I think that Google, Facebook, LinkedIn & Twitter would be at the very top of that list. I guess it depends on how we define "very early".

32
mathattack 2 days ago 0 replies      
I like the list but what are all the source companies? (And in the spirit of survivor bias, failed comparison companies)
33
drelihan 2 days ago 0 replies      
Did you have a list of extremely successful companies you were looking at specifically when you wrote this? If so, I'd be interested in seeing that list to compare
34
quadrangle 1 day ago 0 replies      
tl;dr what makes a company great is how it is run greatly
35
samishamlet 1 day ago 0 replies      
It turns out that pattern matching on PG's essay style does not make one's essays as insightful..

All I can say is: the only factor really worth a damn is luck. Unfortunately you can't control luck, the only thing you can do is increase your luck surface. From my experience, nothing on that list actually does that - what increases luck is: hard work, value of idea, connections and personality. Re-order at will.

36
LeicaLatte 2 days ago 0 replies      
We are prone to saying super while talking. That's ok.

That many supers in writing? Sorry, but that's bad writing.

37
desireco42 2 days ago 0 replies      
No examples, just slogans.
38
kimonos 2 days ago 0 replies      
Nice! I see some helpful information in here. Thanks for sharing!
39
higherpurpose 2 days ago 1 reply      
> *They grow organically. And they are generally skeptical of inorganic strategies like big partnership deals and to a lesser extent PR. They certainly don't have huge press events to launch their startup. Mediocre founders focus on big PR launches to answer their growth prayers.

Has Google+ written all over it.

40
drdiablo 1 day ago 0 replies      
this is a test of fun
12
Public speaking is tough speaking.io
342 points by FredericJ  3 days ago   106 comments top 39
1
nostromo 3 days ago 23 replies      
Here's two pieces of public speaking advice nobody will tell you about, but actually work.

1) Beta-blockers. Ask your doctor.

2) Alcohol. Obviously, be careful with this. :) But having a drink really will take the edge off. This works better when giving a toast as a best man than it does at work. It could probably work at a conference too.

Other than this, for a big talk or pitch, I just practice until I'm blue in the face, then I practice some more. If you experience a fight or flight response, your brain cannot think straight, but you can fall back on something that has become rote long enough for you to regain your footing.

After 30 seconds or so, your body will start to calm down, you just have to make it through that 30 seconds without pulling a Michael Bay. http://www.youtube.com/watch?v=_tqRyzTvNKE

Ask HN: I was thinking the other day, someone should make an Oculus Rift app that is just a giant conference room of people staring at you. People with stage fright could use this to practice public speaking and hopefully improve.

2
beloch 3 days ago 3 replies      
Everyone probably has some good advice for public speaking. Here's my #1 piece:

Slow the fuck down!

You don't "win" at public speaking by getting more words in. In fact, you'll likely lose your audience by going a mile a minute. It makes perfect sense, but it's still hard to do. You can practice your talk in private a hundred times and it'll be X minutes. You can present your talk to colleagues and co-workers and it'll be X minutes. Then, when you get in front of a room full of strangers, the adrenaline will hit, you'll go into manic-caffeine-squirrel mode, and you'll blast it out in X/2 minutes! Some people deliberately make their talks too long, knowing they'll finish early if they don't. This is a mistake. They're just cramming too much material into the time allowed and will shell-shock their audience. Slow the fuck down!

The method by which you slow the fuck down is going to be somewhat personal. Different things work for different people. Personally, I do a lot better if I've gotten to know even just a few people in the room a tiny bit. If I can get a few people (hopefully in the front row) into the colleague-zone, I can focus on them during the talk and ignore the strangers.

3
bane 3 days ago 1 reply      
I'm a so-so to "good" public speaker. I used to be a terrible public speaker. I'll probably never be a great orator or Steve Jobs, but I'm pretty happy with my presentation skills. In group settings, I'm often the one chosen to give the public presentation.

Some things that improved me:

1) My university undergrad CS program required a semester of public speaking. Everybody hated it. It's probably one of the top 3 most important classes I took. If you're in a school that doesn't require it, take it as an elective.

2) I had a teaching job for a few years. Getting points across day in and day out, and trying to drag a class along of people at very different learning speeds teaches you very quickly how to project and enunciate so people can hear you well. Watching the faces of, and talking to, the people in the back rows becomes a very important speaking tool.

3) To deal with stage fright, I learned to mentally "not care" about giving the talk. It's hard to explain, it doesn't mean "not caring about doing a good job", it just means to adopt a viewpoint of detached apathy. Before I learned how to do this, even small stumbles would send me into a panic state which only made it worse ending with an avalanche of stutters and tied tongues. Detached apathy turns those little stumbles into such unimportant things that I don't even know they happened until I listen to a recording of my talk or see myself in a presentation.

4) Practice your speech. Because it's important to look up every once in a while in order to project. Practicing your speech helps you do that, instead of looking down into your note cards or your script. I don't practice it relentlessly like Steve Jobs or President Obama. 2 or 3 runs through is usually good enough for most of my purposes. But it helps you keep your focus on not caring.

5) Practice giving speeches. I haven't done it, but I've heard lots of good things about Oration societies like Toastmasters. In my case I got plenty of practice while teaching. But for those people who don't have that option, this is a great option. Nothing gets you used to the routine of giving speeches like giving speeches.

4
reuven 3 days ago 1 reply      
I have been speaking professionally for a number of years now. In a given week, I'm probably speaking 2-4 full days (minus lunch and breaks), teaching various programming languages and technologies. I also give talks at conferences and user group meetings.

I remember very, very well when I had to give a talk oh-so-many years ago, while doing a student internship at HP. I flubbed it big time, and left the room saying to myself and anyone who would listen that I disliked public speaking, and was bad at it.

I'm not quite sure when things changed, but I think that it had a lot to do with my attitude. Instead of worrying about whether people would like me or believe me, I instead concentrated on trying to teach people something they didn't already know, and have a good time in the process.

If I'm enjoying myself while speaking, then the odds are good that the people in the audience are enjoying themselves, too.

If I've learned something interesting, then the odds are also good that the people in the audience will find it interesting, too, and will be glad that I'm sharing it with them.

Again, I'm not sure when my attitude changed, but when I get up in front of an audience now, I feel like I'm there to have a good time. Of course, I don't want to flub things, and there are times when I worry about that more than others. But for the most part, it's a matter of thinking, "Hey, everyone here has the same goal -- to enjoy themselves and learn something."

As others have written, your enjoyment will be enhanced significantly if you prepare. I'd even say to over-prepare. You probably need to know twice as much as you will actually say in your talk, so that you can speak naturally and reasonably about the subject. Try to outline your talk as a story, with a beginning, middle, and end. In technical talks, the story will often be something like, "Here's a problem. Here's a solution. Here are some examples of the solution in use. Here's where the solution fails. Questions?"

Don't worry about your slides too much. Yes, they should be high contrast. Yes, they should be easy to read. But I think that people worry way way way too much about colors, fonts, and images, and not enough about the actual SPEAKING. You want people to be engaged with what you're saying, not with what's on your slides... and that's going to happen if you have interesting things to say.

Above all, be yourself. There are oh-so-many examples (in real life, and also in movies and on TV) where people are told that they should open with a joke, and so they tell a ridiculous joke that no one finds funny, including the presenter. If you're naturally funny, or are willing to have people not laugh at your jokes, then go for it. If you're a serious kind of person, then be serious. (Although it's always better if you can be somewhat silly, in my book.)

5
hawkharris 3 days ago 0 replies      
Public speaking became much easier to me once I recognized that all good speeches follow a concrete formula.

It's kind of like writing. You wouldn't pick up a pen and start scribbling a lengthy essay without considering its structure.

Similarly, effective public speakers follow a pattern not necessarily the same formula, but a formula. For example, Bill Clinton likes to...

1) Begin with a personal, visual anecdote about a specific person or small group. (e.g. A family walking miles to collect water.)

2) Relate the small example to broader theme. (e.g. Poverty is a big problem.)

3) Weaving that broader concept into the theme of the speech.

Another thing to remember is that while speeches share a structure with writing, they are not written articles. The biggest difference, I think, is that people are not capable of processing as much information.

While repeating yourself in a written piece is often bad form, most public speakers repeat key phrases to keep the audience focused. Listening is usually harder than sitting down to read.

6
saurik 3 days ago 1 reply      
To some extent the point I want to make I'd similar to the one made by reuven elsewhere in this thread, but I think it is still different (and maybe shorter? we'll see ;P) enough to still post. (OK, after writing, this failed at my goal of being shorter ;P.)

So, I also do a lot of conference speaking, albeit nowhere near as much as reuven. I remember in high school, public speaking was terrifying. By the end of college, I was giving one of the graduation speeches.

The difference was not me becoming better at making arguments or telling stories or being prepared or building slides or really anything about what I said on stage: the difference is that I felt at home there.

In essence, I had the fear of public speaking that many, if not most, people have. This fear is mostly about people watching you and judging you. You are concerned about where they are looking and what you are doing: it paralyzes you.

It had very little, however, to do with what you are doing in front of everyone: you could be on stage being told "eat breakfast as you would on a normal day" or simply a lunch meeting where you are standing due to lack of chairs while everyone else is sitting.

I don't feel, therefore, like helping people present is the solution. I will say that it might try to ease the person's anxiety enough to consider doing it once, but that isn't why they are afraid: I am not afraid of bungee jumping because I think I'm going to die due to the cord breaking, I'm afraid of bungee jumping because even looking at a photograph taken from a high-up location makes me curl into a ball.

These fears can be so bad that they aren't obviously fixable (phobia-level fears can be like that). In my case, I likely have acrophobia (heights), but as something of a "class clown" when I was much much younger, I can't ever claim to have had glossophobia (public speaking). My fear was mild, and I tackled it.

I want to be very clear, though, that there is a difference between "preparation" and "lack of fear": if you told me to go stand on stage right now in front of a thousand people, I'd be happy to do that. I would be willing to try to entertain them. I might fail, but I don't mind anymore.

I might thereby recommend more doing something structured that tales away all of the "things you can do wrong" variables entirely before bothering with trying to prepare those away: take an acting class. You are told exactly what to say, you have a director guiding your movements, and on the show day a perfect performance can be identical to the previous day. You don't have to worry if what you are saying sounds stupid: you have no choice in what to say.

(That said, I wouldn't "recommend" it strongly, as I think a lot of these shortcuts in hindsight by people who have defeated something others find hard are missing the point of what made it work for them: that you probably just need to be doing it, constantly, for long enough, to make it easy. This is similar to the "monad tutorial fallacy" in my mind.)

Then, when your fear of being in front of people is gone, maybe the preparation isn't even that big of a deal: if you are comfortable, the audience will be comfortable, and you can "get away with" a lot more on stage.

I mean, preparation is great, but "public speaking is tough" is not because "writing slides is tough" or "answering questions is tough", it's simply tough because "public anything is tough"... you answer questions every day in the hallway: you don't need more preparation to do that on stage, you just need less fear (which again: isn't easy).

7
ctdonath 3 days ago 1 reply      
As an introvert, I have no problem talking in front of a large group. I thrive on one-on-one conversations where each person has an opportunity to talk thru long complex interesting thoughts without interruption. Speaking in front of a large group is exactly that: I get to talk at length on a favorite topic, at whatever level of detail I choose, to someone who is interested in what I'm saying and will not interrupt; that I'm doing this with 10,000 individuals at once is just being efficient about it.

Helps that I've decided that if I'm going to be wrong, I'm going to be definitively wrong.

8
EliRivers 3 days ago 1 reply      
As someone in your audience, I beg you, please do not tell me what you're going to tell me, then tell me, then tell me what you just told me.
9
chops 3 days ago 0 replies      
I've given a handful of talks at miscellaneous user groups ranging from 5 minute lightning demos to one way-too-long-but-there-is-too-much-to-cover-in-45-minutes talk about Erlang types (I felt bad it was so long).

While I'm the last guy to walk up to a stranger and strike up a conversation, and I break out in cold sweats preparing to cold-call prospects for my business, I've always had this thing about public performing, whether it be speaking, playing and instrument, or even (gasp) singing.

I'm not sure of the psychology of it all, but it feels like the pressure of presenting, combined with a strong fear of being viewed a failure gives way to a certain comfort zone in presenting. And once up there for a minute or two, I notice that I quickly find myself firing on all cylinders (probably from the adrenaline), and then everything from then on becomes quite natural for me (even if my natural presentation style comes across a little neurotic).

Anyway, that's my anecdotal contribution to the public speaking discussion.

10
drblast 3 days ago 0 replies      
Don't think of it as public speaking, think of it as a performance.

You wouldn't go try to perform a play without scripting it and memorizing your script first, nor should you do that with your presentation. Once you do that you can ad-lib and it will seem natural. Even the off-the-cuff jokes aren't really off-the-cuff.

And go twice as slow as you think you should, and pause a lot. When people get nervous they talk faster and don't realize it. If you're nervous your perception of time will change and small pauses seem like an eternity. Slow down and force yourself to break for five seconds between "paragraphs" and you'll be way ahead of most people.

11
bedhead 3 days ago 1 reply      
I had never spoken publicly, as in a featured speaker in front of a large gathering of strangers. I had spoken in front of everyone at my old company (80 people) but that was the closest I came to public speaking, and since I knew everyone it didn't count. I remember freshman year of high school having my stomach in knots when teachers would call on me. I just had that nervous personality. Want to know how nervous I'd get in public with everyone's attention on me? I almost fainted at my wedding - at the altar. The priest had to cut the ceremony in half to accommodate me. To this day people make fun of me for it (I feel bad for my wife).

A couple months ago, I surprisingly got asked to be a speaker at a pretty large and prestigious conference in town. It was at a large venue with over 1,000 attendees, some of whom are important to impress for various reasons. It was a great opportunity so I accepted, knowing that this could be a problem.

Anyway, I rehearsed my 10 minute speech ad nauseum, I could do it in my sleep. Every little last verbal tic, joke, everything. I knew I'd still be nervous. I wanted to be so good that I could do it on autopilot and hopefully be more confident. I got on stage, lights shining brightly, and took a seat as the host read a brief introduction about me. While he was doing this, I was so nervous that I thought I was either going to vomit or faint, or some horrible combination of the two. I was literally telling myself not to puke over and over again. My stomach was tossing and my head was spinning...I could barely breathe.

He finishes his intro and I start my talk, visibly nervous. Then a funny thing happened. About 20 seconds in, something clicked and I just thought to myself, "Why are you nervous? You know this stuff cold. You got this." And wouldn't you know it, from there on out I killed it. I dunno, it was weird, I instantly became as relaxed as I am with my friends and delivered a great speech. I had tons of great jokes, kept everyone really engaged, and I think even delivered an interesting idea to the audience. By the time it was over I was actually disappointed it was over since I was having so much fun. I got tons of superlative-filled compliments afterwards and was really in shock about it all.

I dont know what the moral is. Just have fun I guess. Know what you're talking about and the rest will sort itself out.

12
Theodores 3 days ago 2 replies      
Just wing it. Seriously.

Why is it that so few schools teach children how to speak in public?

It is not difficult, all you need is a debating society.

I am fortunate enough to have gone to a school where the debating society was the thing to do. Even on a cold winter with snow outside two hundred or so of the thousand at the school would show up, of their own accord and without anyone telling they had to go. To be voted by your peers onto the committee for the debating society was the ultimate in status. Our debating society made public speaking a fun thing to do.

As well as being able to propose/oppose a motion from the stage with a self-prepared speech it was also possible to learn how to listen, ask questions from the floor and respond to points made.

So, when I left school, I had a head start. I had spoken in front of a crowd on two hundred or so occasions from a very safe sandbox. In my adult life this experience has been invaluable. I know about what happens if one is not totally prepared. I know what happens if one is over prepared - i.e. reading instead of talking. I know about posture and how to make meaningful eye contact with a sea of faces. However, most importantly, I knew that public speaking was a desirable thing to do, a privilege.

If anyone reading this has kids and their kids are not involved in a school debating society, think about it. Get together with the school and a few teachers and sell them the idea of a debating society. Get someone charismatic - a head teacher who has to present in front of all the kids - to make the debating society the most important thing he/she does. Your local posh school will have a debating society, visit them, learn how they do it and steal their procedures and organisational structure.

Then, if you are lucky and the school debating society kicks off and becomes the thing to do, your child should grow up to be a darned good public speaker. What they will learn from that will help them no end. If they also end up knowing a subject inside and out at some stage of their adult life they should be able to literally wing it without having to use any of the silly suggestions presented on this thread (betablockers - you must be kidding!!!).

13
bigd 3 days ago 0 replies      
I've a talk in 30'.

Another suggestion should be "do not read suggestions on how to do talks right before giving one".

after a life in academia, what I usually suggest is:like your topic, keep it easy, and reharse, reharse, reharse.

14
yodsanklai 3 days ago 0 replies      
I used to be really scared when i had to give "important" talks, especially in English which isn't my native language. I was so anxious that I couldn't even work the days before. I remember my first professional talk. My mouth was so dry that talking was difficult. (tip to beginners: take a bottle of water).

Interestingly, I had much less problems when I was presenting somebody else's work.

The thing that really helped me was benzodiazepines (e.g. Xanax). I took them a few days before until the day of the talk and I felt much much better. I know these drugs get a bad press, but in my case, they really helped. The side effets is that they tend to make you sleepy, but it didn't really affected me.

Now, I'm certainly not a great speaker, but I don't have any problems with public speaking.

15
pmiller2 3 days ago 0 replies      
Between grad school (teaching, seminar talks, etc) and other occasions, I've spoken in front of groups of 3-300 people hundreds of times. I have no idea if I'm all that good at it, but at least I'm comfortable with it. :-)

The biggest trick for me is realizing that talking in front of a group is different from talking to one person, but talking in front of a small group is not that different from talking in front of a medium or large group. Under 5 or so people is still pretty much an intimate/conversational atmosphere in my experience, but going from 5 or 10 up to 50, 100, or 300 is pretty much all the same. The only real difference is the amount and type of projection equipment involved.

Depending on the specific scenario, there are other things I try to keep in mind (e.g. I found that between 0.5 and 1.5 slides per minute worked well for a seminar talk in grad school), but abstracting away the size of the audience in my mind is the one that's paid me the biggest returns in reduced anxiety. Now if I just had a way to make sure the A/V equipment always worked, I could make a crapload of money. ;)

16
julienchastang 3 days ago 0 replies      
Public speaking has caused a great deal of distress, panic, and anxiety for me in the past. To remedy this situation, I joined a local Toastmasters club. They are located literally all over the world, and there is probably one in your area. I cannot say enough good things about Toastmasters. Through frequent, repeated public speaking exposure, over time, you become desensitized so you don't feel as panicked. And your speaking skills improve as you have to give speeches on a regular basis. I completely disagree with comments that suggest this problem can be solved through drugs or alcohol. I had ONE stiff drink before an important talk, and I completely hated the feeling while I was speaking.
17
Codhisattva 3 days ago 4 replies      
Practice at Toastmasters meetings.
18
pessimizer 2 days ago 0 replies      
Public speaking terrifies me. I seem to do alright if I follow five rules:

1. Don't bail and run out of the room screaming.

2. Don't ramble. Don't leave your outline for an anecdote or further explanation - trust your outline to be good. If you have to meander because you did your outline at the last minute and you know it kinda sucks, if you then meander while meandering, you've lost the game and no one remembers what you were talking about.

3. Don't "umm," "right," or "ok." before and after anything you say.

4. Don't laugh at your own jokes (at least don't do it before you finish getting them out.)

5. Remember that you don't look as nervous as you feel.

19
treenyc 2 days ago 0 replies      
I have enzyme issue with Alcohol, but I believe that is a nice trick.

From personal experience, and from someone who had tremendous problem with public speaking to someone who performed very well at a toast master event in NYC without any preparation. I can say quite a few things on the subject.

One thing is for sure. We are all afraid of other people. No matter who we are. It is just that fear get expressed in different ways. Some people are being shy and passive, while some are being aggressive and over-confident. Until we discover who we really are. Using tricks (power point) and strategies (drink alcohol/weed) will not take us far.

What made the most difference in my process is some ontological training like this leadership course. The course doesn't really say that it will help you with public speaking. Just that you will leave the course

"Being a Leader and Exercise Leadership Effectively as your own natural Self-Expression"

Nothing more, nothing less.

However, the course has nice side-effects, like public speaking.

The course is NOT cheap, but I consider it worth more than my college degree. Next one is at Singapore. FYI, I have no financial tight to the course or University.

http://beingaleader-singapore.com

20
alan_cx 3 days ago 1 reply      
I assume there are different reasons for people fearing public speaking. But, FWIW, my thing is to really and fully know the subject you are talking about. For me, the nervousness comes from the fear of being found out in some way. So, I find that if I know my subject, Im quite happy to waffle on to who ever wants to listen, but if I know or think the audience might know more than me and be able to some how show me up to be some sort of fraud, Im a bag of nerves.

I dont know if that works for anyone else, but my theory is that the nerves come for the fear of somehow looking a fool, and that becomes less likely the more you know about what you are talking about.

21
jccalhoun 2 days ago 0 replies      
As someone that has and is teaching public speaking my number one tip: sound like you care.

I can't tell you how many terrible speeches I've sat through where the person was saying "this is really important and means the world to me" but sounded like they didn't care at all.

Number two: don't write out ever word of your speech. It is public speaking not public reading. Being able to read a text out loud without sounding like you are reading is a skill and you should learn to speak from notes/outlines first because that is easier to sound like you are talking with us rather than at us.

22
anildigital 3 days ago 0 replies      
23
wturner 3 days ago 0 replies      
The easiest way to speak publicly is to actually believe in what you're doing and talking about. The audience then becomes kind of like an omnipresent pressure that keeps you going.

If you aren't 'locked into' what your talking about then nothing will save you. I know from personal experience.

I also heard a talk that if you imagine the audience as 'prey' such as small rabbits or chickens then it becomes easier as it takes power away from the flight or fight aspect.

24
AhtiK 3 days ago 0 replies      
Exhale as deeply as possible and keep it this way as long as you can. After that breathing restores with first few rapid big inhales. Restarting your breathing this way is also restarting your brain in a way so the thinking becomes calm. Works every time.

Another tip is to eat 1-2 bananas half an hour before the event and maybe a glass of fresh orange juice. Banana works as a natural beta blocker reducing anxiety. While on stage, plain water, no juices..

25
chaz 3 days ago 0 replies      
Plant your feet and square your shoulders to the audience. Walking around is ok, too. But slouching and shifting your weight from left to right can hurt your confidence as well as hurt the way your confidence is projected. You'll develop your own more natural style over time.
26
city41 3 days ago 0 replies      
blatant plug: I'm working on a website aimed at increasing social skills and one "track" of the site will be for improving public speaking -- http://metamorf.us
27
mebassett 3 days ago 1 reply      
Say someone is a mediocre-to-decent public speaker already. How does one "level up" to be a really great public speaker? I've thought about a speech coach or class, but I don't know anyone who has had any success with this who could recommend where to find a good one.
28
re_todd 3 days ago 0 replies      
I went to a doctor, and he gave me beta blockers, which helped a lot.

Another thing that helped is reading forums like this where many people admit how nervous they are. In speech class, everyone seemed to do relatively well, so I was under the impression that I was the only person in the world that gets nervous during a speech. Just knowing that other people get nervous has helped me handle it better.

You can also take your contacts out or glasses off so you cannot see people clearly, which also helps a little.

I've also noticed that my anxiety attacks usually happen before the speech, not usually during it, and they only last a few minutes. Knowing that they will not last forever has also helped me.

29
eflowers 3 days ago 0 replies      
What I've learned is that 20 minutes in, you're hour is up.
30
peteri 2 days ago 0 replies      
For talks to user groups where I'm generating new slide decks and demos one piece of advice I was given was reckon on around 1 hour of prep for each minute of speaking time. The successful stuff that I've done seemed to match this.

Also for a one hour time slot you'll probably actually want around 40 minutes of material allowing time for introductions and a Q&A session at the end.

31
aniketpant 3 days ago 0 replies      
Nobody mentioned Speak Up. It's a wonderful community of people where everyone helps each other out in planning and preparing for talks. It's been slightly inactive recently, but every mail gets an assured response.

Link: http://speakup.io/

32
cmbaus 3 days ago 0 replies      
Here are couple ideas I've written on the topic: http://baus.net/i-don%27t-like-public-speaking/

I did quite a bit of public speaking in the past couple years and it gets easier over time. I think the best advice is prepare, prepare, prepare.

33
janogonzalez 3 days ago 0 replies      
Shameless plug, here it is my own advice regarding conference speaking: http://janogonzalez.com/2013/12/02/conference-speaking-how-t...
34
gumby 3 days ago 1 reply      
To me there are different scales of public speaking or presenting.

I actually have no problem presenting to 500 people (the largest audience I've had): I just talk, and try to make some eye contact. There are always a few friendly faces.

Presenting to up to a dozen people is no problem for me: I can adapt (speed up / slow down, skip over stuff, dive deep, repeat, whatever) depending on how the people react.

But there's an excluded valley of somewhere between one and three dozen. I feel weird just presenting as I would to 500 people, yet it's too big to get the intimate preso treatment. When I have presented to a group this size it has almost always fallen flat.

35
Kerrick 3 days ago 0 replies      
Another great resource: We Are All Awesome! http://weareallaweso.me/
36
ismaelc 3 days ago 0 replies      
If you have something exciting to talk about, public speaking is not such a chore (a joy in fact). The challenge is having content that's easy to make exciting.

If that's not possible for you, then try to get excited of the fact that you're out there to excite the hell out of something mundane. Surprise your audience.

Being in that state of mind alone should knock out the jitters.

37
crimsonalucard 3 days ago 0 replies      
The only way a phobia can be conquered, if it can be conquered at all, is through repeated exposure.
38
hakanson 2 days ago 0 replies      
Where can I submit a pull request to remove the F-word from these otherwise great tips, or do I need to fork. One could consider "dropping the F-Bomb" against many conferences code of conduct pertaining to "harassment includes offensive verbal comments." Also, as we try and mentor more youth to code, including school age girls, is this the persona we ware marketing?
39
gre 3 days ago 1 reply      
Tell them what you are about to tell them, tell them, tell them what you told them.
13
Net neutrality is half-dead: Court strikes down FCCs anti-blocking rules arstechnica.com
324 points by shawndumas  3 days ago   126 comments top 23
1
mjmahone17 3 days ago 1 reply      
What the court is saying is that, if the FCC refuses to classify broadband providers as common carriers, then, because they neither receive the same protections as common carriers nor have the same responsibilities, they can't be regulated as if they were common carriers.

The FCC could change their rules to treat broadband suppliers as common carriers. However, that's something that big-name broadband providers don't seem to want, as it would reduce their freedom of operations.

2
saalweachter 3 days ago 7 replies      
Note that the DC Court of Appeals is the one that the Filibuster Crisis was all about. According to the Wikipedia, it still(!) has three vacancies, and the Senate Republicans have spent the last ~N months preventing any of the Obama administrations nominees from being confirmed to the Court.

These things matter.

3
loup-vaillant 2 days ago 1 reply      
We could make all the laws we want about Net Neutrality, it wouldn't change the fundamental flaw that made this problem possible in the first place: too much centralization.

I hear that the US, there are only 2 ISPs: one of the big 2, or the little local one. In France, we have about 4. At the other end, we have Google, Amazon, but most notably we have YouTube and Netflix.

Clearly the market is not efficient. Why do we have so big players in the first place? Why do we tend to have only the big players?

Because of the infrastructure. In the way the internet is distributed, artificial economies of scale and barriers to entry favour the big ISPs (this is clearly the case in France, I suppose the US is the same). And, we have asymmetric bandwidth, which kills peer to peer exchanges. If people were allowed to host servers at home, there would be no need for things such as YouTube, Blogger, or Facebook (search engine are still a thorny problem, though).

If we got rid of this over-centralization, it would be harder to discriminate your bandwidth in the first place. Net Neutrality would be the default, instead of something we have to fight for.

4
sologoub 3 days ago 2 replies      
Definitions from US Code Title 47:

"(1) Advanced communications servicesThe term advanced communications services means(A) interconnected VoIP service;(B) non-interconnected VoIP service;(C) electronic messaging service; and(D) interoperable video conferencing service."

"(11) Common carrierThe term common carrier or carrier means any person engaged as a common carrier for hire, in interstate or foreign communication by wire or radio or interstate or foreign radio transmission of energy, except where reference is made to common carriers not subject to this chapter; but a person engaged in radio broadcasting shall not, insofar as such person is so engaged, be deemed a common carrier."

"(24) Information serviceThe term information service means the offering of a capability for generating, acquiring, storing, transforming, processing, retrieving, utilizing, or making available information via telecommunications, and includes electronic publishing, but does not include any use of any such capability for the management, control, or operation of a telecommunications system or the management of a telecommunications service."

I'm not a lawyer, but consider myself well grounded in tech and telecom, but reading these definitions I'm kind of at a loss. In common law, my understanding is that a "common carrier" is someone that makes transport services available to the public. These can be physical, such as shipping a crate, or technological (telecom) in nature. By that inference, transporting packets of information is essentially same as transporting normal packages.

Unfortunately, the "by wire or radio or interstate or foreign radio transmission of energy" is so period-specific that one could argue that it doesn't apply and the (24) Information Services is so broad and vague, it could practically be applied to anything.

One interesting bit, which makes me think that there is hope, is the definition of advanced communications, that include both VoIP and messaging services. Sadly, their definitions are not that broad...

5
gdubs 3 days ago 1 reply      
My knowledge of Anti-Trust laws dates back to elementary school, but how is it legal for the companies that maintain the infrastructure to be in the content game as well, when other content providers can't compete on favorable pricing for bandwidth?
6
jacobheller 3 days ago 0 replies      
Here's the full text of the opinion on Casetext: https://www.casetext.com/case/verizon-v-fcc-3

We'll be getting some leading net neutrality scholars and lawyers to annotate the doc, so check back later today for interesting, in-depth analysis along-side the key passages in the case.

7
adricnet 3 days ago 0 replies      
So, the court agrees with many others that the FCC needs to re-label cable companies as communication common carriers before regulating them as common carriers. I guess that's good?

Is it a difficult thing technically or only politically for the FCC to change their minds / admit they did this wrong in the first place?

What is the downside of treating the cable networks as communications media?

There are some thoughts on that here, though note the source: http://www.attpublicpolicy.com/government-policy/the-fcc-hav... .

8
declan 3 days ago 1 reply      
This is a duplicate of another thread started an hour earlier: https://news.ycombinator.com/item?id=7057495
9
anExcitedBeast 2 days ago 1 reply      
I know this isn't a popular opinion, but I think we need to reconsider accepting precedent that we're OK with the US government regulating the Internet (without legislation, no less). With the concern about surveillance, copyright abuse, DMCA/Computer Fraud and Abuse Act abuse, and content regulation (in the case of the UK and China), is this really the solution we're looking for? Neutrality is a real problem, but this seems like throwing the baby out with the bathwater.
10
pasbesoin 3 days ago 2 replies      
It is obvious that they are, de facto, common carriers.

Give up the lobbyist payola, reclassify them, and introduce some real competition to my now more frequent than annual Crapcast price bumps (or significant humps, as it were).

(And in my case, this is primarily for Internet, although basic cable comes along as a quasi-freebie -- it costs, but then a discount on the combined package largely or totally negates that cost.)

Otherwise, you can bet I'm not voting for either major party, next time around.

As a consumer, I find that the only way to defeat this bullshit, is to stop paying for it. If I had an alternative to Crapcast in my neighborhood, I'd take it. (I don't count AT&T, because for a lonnngggg time they refused to deploy high speed Internet here, and because their policies and behaviour are just as bad. As well, they've personally screwed me in a prior location, as I've commented before.)

11
kolbe 3 days ago 2 replies      
Intuitively, I would have thought that this would be horrible news for content providers/distributors, and great news for wireless carriers. However, today, Google, Facebook, Amazon &etc are flying, while Verizon and AT&T are falling.

Does anyone in the industry know what this is all about, and what importance this decision really has on the future of mobile?

12
shmerl 3 days ago 0 replies      
Why can't they start classifying ISPs like common carriers?
13
unethical_ban 3 days ago 0 replies      
Am I experiencing deja-vu? I fee like many of these comments (and their responses!) are exact reposts from earlier submissions about this very same topic.

  "The court is saying the FCC needs to reclassify providers"    "The Republicans are holding up nominations"    
so on so forth.

14
angersock 3 days ago 1 reply      
Anyone who is interested in a really good overview of 20th century telco policy should read The Master Switch by Tim Wu (http://www.amazon.com/The-Master-Switch-Information-Empires/...).

It goes over the transition from telegraph to telephone to internet, talks about the rise of media conglomerates, and basically explains how we're in the mess we're in today. Quite an enjoyable read, especially when learning about the differences between old and new styles of monopolies.

15
wahsd 3 days ago 0 replies      
What I wish would happen is that organizational forces were focused on breaking up ISP monopolies over the pipes. Essentially, building a firewall between infrastructure and content. It would create a market...remember that thing we think controls America's destiny...that would lead to faster bandwidth and also ISPs that offer free, open, and protected services.
16
tomrod 3 days ago 0 replies      
The economist in me is happy, as this allows for greater investment incentives on the part of ISPs.

The FLOSS advocate in me is sad, as this is a compromise that I don't want to see go away.

17
hrjet 2 days ago 1 reply      
How much should the rest of the world care about this?

If a web service is hosted, say, in Europe and is being consumed by a customer also in Europe, will they be affected? AFAICT, they shouldn't.

18
VladRussian2 3 days ago 0 replies      
>In its ruling against the FCCs rules, the court said that such restrictions are not needed in part because consumers have a choice in which ISP they use.

in theory vs. in practice

19
the_watcher 3 days ago 1 reply      
Does anyone have a good solution for this argument? I find myself wildly sympathetic to both sides of it. Is there any way to decentralize internet access in the future (something like what the utopian ideal of solar powering your home would be for electricity)?
20
draggnar 2 days ago 0 replies      
Perhaps this means the other shoe will fall. Will local regulations making it easy for municipalities or other actors to set up their own ISPs?
21
nitrobeast 3 days ago 0 replies      
Quote from the linked article, "(net neutrality rules) forbid ISPs from blocking services or charging content providers for access to the network." But that is confusing. ISPs are already charging content providers for access to the network. Netflix and Google need to pay for their bandwidth.

Actually, web neutrality means the ISPs should treat all data in their network equally.

22
Grovara123 2 days ago 0 replies      
Why is this not a bigger deal?
23
pearjuice 3 days ago 1 reply      
How can something or someone be half-dead? Life is a binary thing, you are either YES (1) alive or NO (0) dead. I fail to comprehend how any respectable (tech) journalist would call something "half-dead". It implies there is a state between being alive and death when this is clearly not the case.
14
Why 'Her' will dominate UI design even more than 'Minority Report' wired.com
308 points by anigbrowl  3 days ago   206 comments top 30
1
aegiso 3 days ago 16 replies      
Here's the thing that bugged me throughout the movie: once AI's progressed to the point where it can rival a human, all bets are off. Nobody needs to work again, ever -- not even to maintain or develop the AI's, since they can, by definition, do that themselves, with infinite parallelizeability to boot.

What does "design" even mean in a world where everyone on earth can basically have an arbitrarily large army of AI's in the background designing everything in your life, custom-tailored for you?

For this reason I don't see how the world in the movie could possibly exist. Not because the technology will never get there, but because once it does virtually all aspects of society that we take for granted go out the window. So imitating any of this design is a silly pursuit, because once you can make it there's no reason to.

I should go re-read some Kurzweil.

2
mrmaddog 3 days ago 3 replies      
I have not yet seen "Her", but this strongly reminded me of Ender's communication with Jane from the "Ender's Game" sequels. One of the most interesting facets to their conversations is that Ender could make sub-vocal noises in order to convey his pointsshort clicks of his teeth and movements of his tonguethat Jane could pick up on but humans around him could not. It is the "keyboard shortcuts" of oral communication.

If "Her" is really the future to HCI, then sub-vocal communication is a definite installment as well.

3
jasonwatkinspdx 3 days ago 3 replies      
I once read a quip in an interview with a sci-fi author. He said something like: "No one writing about the present day would spend paragraphs explaining how a light switch works." It's easy for sci-fi to fall into the trap of obsessively detailing fictional technologies, to the determent of making a vivid setting and story.

Edit: I'm not saying that sci-fi shouldn't communicate some understanding of the future technology or shouldn't enjoy engaging in some futurology. Just that it's difficult to do in an artful way.

4
kemayo 3 days ago 2 replies      
>>> Theos phone in the film is just thata handsome hinged device that looks more like an art deco cigarette case than an iPhone. He uses it far less frequently than we use our smartphones today; its functional, but its not ubiquitous. As an object, its more like a nice wallet or watch. In terms of industrial design, its an artifact from a future where gadgets dont need to scream their sophisticationa future where technology has progressed to the point that it doesnt need to look like technology.

This article really makes me think of the neo-Victorians from Neal Stephenson's Diamond Age.

...which is kind of funny, because in many ways Snow Crash exemplifies the other ("Minority Report") style of design the article talks about.

5
scotty79 3 days ago 5 replies      
Voice is horriblly slow medium of transfering information. I read because it's faster than listening to an audiobook. It's not scannable. You can't skip through the unimportant parts with one thought as you can do when you look at things.

You can listen to a single voice stream at a time so when AI talks to you you are more cut off from the people around you than when you look at our phone. ...unless exchanging glances is more important than what people are actually trying to tell you when you happen to look at the screen.

6
w-ll 3 days ago 2 replies      
OT: But if you get a chance, watch [1] Black Mirror. There is 2 seasons of 3 episodes. skip the first episode maybe? but I liked it because that* could happen tomorrow. Where as the other shorts are in a somewhat see-able future.

I feel like Spike Jonze was inspired by a few of the episodes. Her was still an amazing movie.

1. http://www.imdb.com/title/tt2085059/

7
sourc3 3 days ago 2 replies      
Saw the movie this past weekend and thought it was really good. I didn't like it just because it has awesome voice driven OSes or endless battery life devices, but because it portrays a current trend we are experiencing; hyper connected loneliness.

The more people are "digitized" and tethered to their devices, the more they seek some human connection.

Don't want to ruin the movie for those who haven't seen it so I won't comment on the ending. However, I urge the HN crowd to check it out. It's one of the best movies I've seen in a while.

8
snowwrestler 3 days ago 1 reply      
Does Minority Report dominate UI design? I think it has dominated the movies' potrayal of future UI, but that is not the same thing.

I think if you look at the actual UIs being designed and sold today, their clearest entertainment ancestor is Star Trek the Next Generation.

9
altero 3 days ago 4 replies      
I wish futurist would just drop speech recognition as holly grail. Speech has lot of flaws, is horribly unprecise and non private. I think neural interface has better future.
10
mratzloff 3 days ago 1 reply      
I found the technology in Her to be natural and elegant, all things considered.

Actually, the most improbable thing in the movie is that this guy had the equivalent of a $40,000 a year job and rented such a fantastic apartment.

(Also, that the website BeautifulHandwrittenLetters.com would be successful with such a clunky domain name.)

11
aaron695 3 days ago 0 replies      
As usual a fictional movie uses a imaginary amazing far future backend with a 'new' UI and people seem to think it's the UI that's the great bit.

Minority Report was never about the UI, it was the software that allowed the gestures find the info. It would have been equally amazing and quick with a mouse and keyboard.

This is a common trick when people demo new hardware. Somehow that internet mirror knows exactly what to show you in the morning by magic, but you think it's the physical internet mirror that's amazing when you watch the demo.

12
jkw 3 days ago 4 replies      
Can someone explain how Minority Report dominated UI design? (serious question)
13
njharman 3 days ago 0 replies      
Making technology "invisible" is missing the point and wrong tack to take. It's not that tech is hidden. It's that tech has become so ubiquitous, accepted, and integrated that we no longer notice it or think of it as "tech". Which combines social changes, refinement of technology, and time (as in, new generation has to grow up not knowing life before smartphones for example).
14
leephillips 3 days ago 0 replies      
According to the article, the movie depicts a near-future where "a new generation of designers and consumers have accepted that technology isnt an end in itself". Do people the the present regard technology as an end in itself? I had no idea. Anyway, I'm a big Jonze fan and want to see this.
15
danso 3 days ago 1 reply      
Does anyone still re-watch TNG episodes and find that the queries they do to be profoundly limited in power, other than the feature of having the universe's knowledge to query across?

If UIs are taking cues from entertainment, they might act as a nice bridge, but are just as likely to be stifling

16
JVIDEL 3 days ago 2 replies      
From the UX standpoint the problem with Minority Report (MR) is that when you compare it with the tech we had in 2001-02 its completely INSANE, while Her is actually building on top of something we already have

Point in case 12 years ago we didn't have ANYTHING close to the UX in MR, and even today we don't. Any consumer-available motion tracking and gesture recognition is still not comfortable to use in a professional way (ie: for work) as it was in the movie, but voice recognition is much much better than it was in 2002.

Basically Her is like Siri or any other decent voice assistant, but MR is like...........what? kinect? nah, wii? yeah right, leap? yeah right! I can picture tom cruise losing all tracking the moment he rotates his hand...

17
skizm 3 days ago 1 reply      
Minority Report technology is garbage. That much hand waving and moving around gets tiring after about 5 minutes. In no way does that UI beat a keyboard and mouse or an xbox controller depending on context.
18
krazybig 3 days ago 0 replies      
The question of how AI will integrate with our society and economy is a fascinating one. We often make the mistake of assuming that an AI will be similar to a human just faster or smarter, but that misses some of the key distinctions of an AI versus biological intelligence.

One of the most striking is the ability to radically alter the substrate and operation of an AI system.

Because of the emergent nature of intelligence, I suspect that many AI instances will be raised like children, tested and validated for specific environments and then large portions of their consciousness could be frozen to prevent divergence of their operational modes. AI systems could also incorporate self-auditors, semi-independent AIs which have been raised to monitor the activities of the primary control AIs. Just as we involve checks and balances in corporate or national governance, many AIs may be composite entities with a variety of instances optimized for different roles.

This will be desirable since you may not want a general AI intelligence acting as a butler or chauffeur. Do you really want them to be able to develop and evolve independently?

Of course this just scratches the surface. AI will take in us in directions we can not dream of today.

19
sirkneeland 3 days ago 0 replies      
So this is how Apple gets disrupted. A future in which devices go from the central component, the obsession, the grabber of our attention, to dumb (if not invisible) terminals to a massive omnipotent cloud.
20
jotm 3 days ago 2 replies      
I haven't seen the movie, so I gotta ask - do those glasses have built in displays? Cause that seems like the near future and a better one than just vocal communication...
21
platz 3 days ago 0 replies      
All the comments here debating whether AI in the movie would. What about the topic of the article, design?
22
wooptoo 3 days ago 0 replies      
While I was reading this I couldn't stop thinking how much it converges with the ideals of calm computing http://www.ubiq.com/hypertext/weiser/acmfuture2endnote.htm
23
ecoffey 3 days ago 0 replies      
Reminds of the Human-AI relationship in this series : http://en.wikipedia.org/wiki/Counting_Heads
24
marc0 3 days ago 0 replies      
I see quite some discussion about UIs and whether they should be audio based or rather visually oriented etc. For a really futuristic intelligent device (call it OS, robot ...) I would drop the idea of "the UI" at all. Rather I would imagine such a system to be intelligent enough to provide a suitable way to exchange data depending on the situation and the task.

There are times when "it" listens to my words and answers verbally. At other times I just want "it" to read what I wrote on my sheet of paper and interpret it. Or I want it to follow my eye movements, or read command off my lips. And it's not just a collection of UIs, but it's a flexible UI that adapts its protocols permanently (sometimes twinkling of an eye has huge information content, sometimes not).

25
solnyshok 3 days ago 1 reply      
started reading that article, but then got carried away with thoughts, what if AIs were designed to make humans's life nice and pleasurable and romantic. That could work until 2 humans fell in love with one AI. What's next? Give each a clone?
26
trumbitta2 3 days ago 1 reply      
I'm uncomfortable with the idea of a computer system solely based on speech recognition, without a keyboard or other input devices, as the one depicted in the article.

How about people who can't speak or hear?

27
zequel 2 days ago 0 replies      
" he realized, isnt a movie about technology. Its a movie about people"

That quote, from the article, could be applied to every apocalyptic, zombie and robot movie. It's not about the [X], it's about how people react to [X].

28
tempodox 3 days ago 0 replies      
Can we PLEASE stop posting this pointless Wired infotainment crap?
29
frade33 3 days ago 1 reply      
pardon my ignorance to technology, is this even hypothetically possible to create AI intelligent enough to be at par with humans or even more?
30
abhi3188 3 days ago 0 replies      
any idea when this movie is releasing in India?
15
AMD launches Kaveri processors aimed at starting a computing revolution venturebeat.com
294 points by mactitan  3 days ago   187 comments top 39
1
pvnick 3 days ago 5 replies      
Among other things, this has lots of applications for molecular dynamics (computational chemistry simulations) [1]. Before you had to transfer data over to the GPU, which if you're dealing with small data sets and only computationally limited is no big deal. But when you get bigger data sets that becomes a problem. Integrating the GPU and the CPU means they both have access to the same memory, which makes parallelization a lot easier. If, as someone else here said, AMD is partnering with Oracle to abstract the HSA architecture with something more high-level like java [2], then you don't need to go learn CUDA or Mantle or whatever GPU language gets cooked up just for using that hardware.

I'm personally hoping that not only will we get to see more effective medicines in less time, maybe some chemistry research professors will get to go home sooner to spend time with their kids.

[1] http://www.ks.uiuc.edu/Research/gpu/

[2] http://semiaccurate.com/2013/11/11/amd-charts-path-java-gpu/

2
ChuckMcM 3 days ago 2 replies      
This reaffirms for me again that we really need AMD to keep Intel from falling asleep at the wheel. I was certainly intrigued by what I saw in the Xbox One and PS4 announcements and being able to try some of that tech out will be pretty awesome.

It is fascinating for me how FPUs were "always" co-processors but GPUs only recently managed to get to that point. Having GPUs on the same side of the MMU/Cache as processors is pretty awesome. I wonder if that continues though what it means for the off chip GPU market going forward.

3
pron 3 days ago 3 replies      
AMD is doing some interesting work with Oracle to make it easy to use HSA in Java:

* http://semiaccurate.com/2013/11/11/amd-charts-path-java-gpu/

* http://www.oracle.com/technetwork/java/jvmls2013caspole-2013...

* http://developer.amd.com/community/blog/2011/09/14/i-dont-al...

* http://openjdk.java.net/projects/sumatra/

It is intended that the GPU will be used transparently by Java code employing Java 8's streams (bulk collection operations, akin to .Net's LINQ), in addition to more explicit usage (compile Java bytecode to GPU kernels).

4
amartya916 3 days ago 1 reply      
For a review of a couple of the processors in the Kaveri range: http://www.anandtech.com/show/7677/amd-kaveri-review-a8-7600...
5
AshleysBrain 3 days ago 2 replies      
I have a question: Previous systems with discrete GPU memory had some pretty insane memory bandwidths which helped them be way faster than software rendering. Now GPU and CPU share memory. Doesn't that mean the GPU is limited to slower system RAM speeds? Can it still perform competitively with discrete cards? Or is system RAM now as fast as discrete-card bandwidth? If so does that mean software rendering is hardware-fast as well? Bit confused here...
6
bvk 3 days ago 2 replies      
The comparison is hardly disingenuous: the i5 may not be given Intel's highest branding designation, but it is an enthusiast processor and only a slight step down from the top-of-the-line i7-4770k, lacking only hyperthreading.

And this is completely irrelevant, since the i5-4670k ships with Intel's highest integrated graphics option for desktop chips, which is what is being compared to the A10-7850k.

At the moment AMD's processors can't compete with Intel at the high end. It makes no sense to berate a company for not doing what it can't.

7
networked 3 days ago 0 replies      
This is an interesting development indeed. In light of http://images.anandtech.com/doci/7677/04%20-%20Heterogeneous... I wonder if we'll soon see a rise in cheap, low-power consumption dedicated servers meant for GPU-accelerated tasks (e.g., for an image host to run accelerated ImageMagick on to resize photographs). Do you think this would be viable in terms of price/performance?

And in case you were, like me, wondering about how much the new AMD CPUs improve on improve on their predecessors' single-thread performance you can find some benchmarks at http://www.anandtech.com/show/7677/amd-kaveri-review-a8-7600....

8
tommi 3 days ago 2 replies      
Kaveri means 'Buddy' in Finnish. I guess the CPU and graphics are buddies in this case.
9
GigabyteCoin 3 days ago 3 replies      
Any initial insights as to whether this new CPU/GPU combo will play any nicer with linux than previous AMD GPUs?

Setting up Catalyst and getting my ATI Radeon cards to work properly in a linux setup is probably my least favorite step in setting up a linux computer.

10
anonymfus 3 days ago 3 replies      
11
ck2 3 days ago 2 replies      
AMD needs to die shrink their R9 chip to 20nm or less and put four of them on a single pci-e board.

They'd make a fortune.

12
transfire 3 days ago 2 replies      
Hey, they finally built an Amiga-on-a-chip!
13
dmmalam 3 days ago 0 replies      
This could be an interesting solution for a compact steambox, essentially very similar to the hardware in the ps4 & xbox one, though I wonder if the lack of memory bandwidth would hurt performance noticeably.
14
malkia 3 days ago 3 replies      
Old ATI chips were named Rage. Kaveri seems to be a river in India.... but it would've been much more cooler if it was named Kolaveri, which according to my poor translation skills must mean Rage in Indian (or one of it's dialects - possibly tamil).

And then there is the song... :)

15
jjindev 3 days ago 0 replies      
"AMD says Kaveri has 2.4 billion transistors, or basic building blocks of electronics, and 47 percent of them are aimed at better, high-end graphics."

This sentence would have been so much better off if they'd just punted on the weak explanation of "transistor" and left it to anyone unsure to look it up.

16
jcalvinowens 3 days ago 2 replies      
This is interesting, but my experience is that Intel's CPU's are so monumentally superior that it will take a lot more than GPU improvements to make me start buying AMD again.

Specifically I'm dealing with compile workloads here: compiling the Linux kernel on my Haswell desktop CPU is almost a 4x speedup over an AMD Bulldozer CPU I used to have. I used to think people exaggerated the difference, but they don't: Intel is really that much better. And the Haswells have really closed the price gulf.

17
fidotron 3 days ago 1 reply      
This is great progress, and the inevitable way we're going to head for compute heavy workloads. Once the ability to program the GPU side really becomes commonplace then the CPU starts to look a lot less important and more like a co-ordinator.

The question is, what are those compute bound workloads? I'm not persuaded that there are too many of them anymore, and the real bottleneck for some time with most problems has been I/O. This even extends to GPUs where fast memory makes a huge difference.

Lack of bandwidth has ended up being the limiting factor for every program I've written in the last 5 years, so my hope is while this is great for compute now the programming models it encourages us to adopt can help us work out the bandwidth problem further down the road.

Still, this is definitely the most exciting time in computing since the mid 80s.

18
Torn 3 days ago 0 replies      
> It is also the first series of chips to use a new approach to computing dubbed the Heterogeneous System Architecture

Are these not the same sort of AMD APU chips used in the PS4, i.e. the PS4 chips already have HSA?

According to the following article, The PS4 has some form of Jaguar-based APU: http://www.extremetech.com/extreme/171375-reverse-engineered...

19
sharpneli 2 days ago 0 replies      
This looks really cool. However it suffers from the same issue as their Mantle API suffers from. The actual interesting features are still just hype with no way of us accessing them.

Yeah the HW supports them but before the drivers are actually out (HSA drivers are supposedly out at Q2 2014) nothing fancy can be done. It'll probably be at end of 2014 until the drivers are performant and robust enough to be of actual use.

20
rbanffy 3 days ago 2 replies      
Are there open-source drivers or will the driver builders have to reverse engineer the thing?
21
vanderZwan 3 days ago 2 replies      
Here's something that confuses me, and maybe someone with better know-how can explain this:

1: The one demo of Mantle I have seen so far[1] says they are GPU bound in their demo, even after underclocking the CPU processor.

2: Kaveri supports Mantle, but claims to be about 24% faster than Intel HD processors, which are decent, but hardly in the ballpark of the type of powerful graphics cards used in the demo.

So combining those two, aren't these two technologies trying to pull in different directions?

[1] Somewhere around the 26 minute mark: http://www.youtube.com/watch?v=QIWyf8Hyjbg

22
grondilu 3 days ago 0 replies      
The A-Series APUs are available today.

It's nice to read a tech article about a new tech that is available now, and not in an unknown point in the future.

23
higherpurpose 3 days ago 1 reply      
I wish Nvidia would join HSA already, and stop having such a Not Invented Here mentality.
24
codereflection 3 days ago 0 replies      
It's really nice to see AMD getting back into being a game changer.
25
hosh 3 days ago 3 replies      
I'm a bit slow on the uptake ... but does this remind anyone of the Cell architecture? How different are those two architectures?
26
annasaru 3 days ago 0 replies      
Nice name. A majestic river in South India.. https://en.wikipedia.org/wiki/Kaveri
27
rch 3 days ago 0 replies      
> the power consumption will range from 45 watts to 95 watts. CPU frequency ranges from 3.1 gigahertz to 4.0 gigahertz.

I was fairly dispassionate until the last paragraph. My last Athlon (2003-ish) system included fans that would emit 60dB under load. Even if I haven't gotten exactly the progress I would have wanted, I have to admit that consumer kit has come a long way in a decade.

28
jsz0 3 days ago 1 reply      
The problem I see with AMD's APUs is the GPU performance, even if it's twice as fast as Intel's GPUs, both Intel & AMD's integrated GPUs are totally adequate for 2D graphics, low end gaming, and light GPU computing. Both require a discrete card for anything more demanding. IMO AMD is sacrificing too much CPU performance. Users with very basic needs will never notice the GPU is 2x faster and people with more demanding needs will be using a discrete GPU either way.
29
dkhenry 3 days ago 1 reply      
So we finally get to see what HSA can bring to the table.
30
belorn 3 days ago 1 reply      
Will the APU and graphic card cooperate to form a multi-GPU with single output? It sounds as it could create a more effective gaming platform than a CPU and GPU combo.
31
devanti 3 days ago 0 replies      
Hope to see AMD back in its glory days since the Athlon XP
32
erikj 3 days ago 0 replies      
The wheel of reincarnation [1] keeps spinning. I hardly see anything revolutionary behind the barrage of hype produced by AMD's marketing department.

[1] http://www.catb.org/jargon/html/W/wheel-of-reincarnation.htm...

33
lispm 3 days ago 0 replies      
So the next computing revolution is based on more power hungry chips for gamers?
34
adrianwaj 3 days ago 0 replies      
I wonder how well they can be used for mining scrypt.
35
imdsm 3 days ago 1 reply      
How do I get one?
36
ebbv 3 days ago 2 replies      
All of Intel's recent mass market chips have had built in GPUs as well. That's not particularly revolutionary. The article itself states "9 out of 10" computers sold today have an integrated GPU. That 9 out of 10 is Intel, not AMD.

The integrated GPUs make sense from a mass market, basic user point of view. The demands are not high.

But for enthusiasts, even if the on die GPU could theoretically perform competitively with discrete GPUs (which is nonsensical if only due to thermal limits), discrete GPUs have the major advantage of being independently upgradeable.

Games are rarely limited by CPU any more once you reach a certain level. But you will continue to see improvements from upgrading your GPU, especially as the resolution of monitors is moving from 1920x1200 to 2560x1440 to 3840x2400.

37
higherpurpose 3 days ago 1 reply      
> AMD now needs either a Google or Microsoft to commit to optimizing their operating system for HSA to seal the deal, as it will make software that much easier to write.

I'd say this is perfect for Android, especially since it deals with 3 architectures at once: ARM, x86, MIPS (which will probably see a small resurgence once Imagination releases its own MIPS cores and on a competitive manufacturing process), and AMD is already creating a native API for JVM, so it's probably not hard to do it for Dalvik, too. It would be nice to see support for it within a year. Maybe it would convince Nvidia to support it, too, with their unified-memory Maxwell-based chip next year, instead of trying to do their own thing.

38
X4 3 days ago 0 replies      
Want to buy, now! Can someone give me a hand at choosing a motherboard or something that allows using about 4 to 8 of these APU's?
39
noonereally 3 days ago 1 reply      
"Kaveri" is name of one of major river in India. Must have involved ( or headed) by Indian guy.

http://en.wikipedia.org/wiki/Kaveri

16
Project Euler projecteuler.net
291 points by gprasanth  3 days ago   134 comments top 35
1
jboggan 3 days ago 5 replies      
The best technical interview I ever had involved picking a random Project Euler problem in the hundreds and pair-programming our way through it. The CTO wrote his version in Python and I worked in Perl . . . he was astounded mine ran 8x faster.

The same company also had regular hack night where everyone drinks a lot of Tecate, agress on a Project Euler problem and a language no one knows, and races. Fun times.

2
habosa 3 days ago 1 reply      
I can't adequately express how great of a resource Project Euler is to someone learning about programming.

The way I learned to code was working my way through Project Euler problems in Python, eventually getting to a score of about 55 before I was at the point where I decided to try making "real" programs like Android apps.

When you learn to code people tell you that X or Y is bad for performance, and you should do A or B instead. The problem is that most beginner-type programs run in a few milliseconds and there is no way to see the performance either way. When you're doing a PE problem, a performance tweak can change your answer from a 1-minute runtime to a 1-second runtime. That's something anyone can appreciate, and it lets you experiment with performance on interesting math problems.

Another advantage of Project Euler is it makes you realize just how powerful a computer can be in the right hands. These are problems that nobody in their right mind would try to solve by hand, but they're so tractable with programming knowledge. That was a very exciting realization to me and it pushed me towards a career in software.

3
b0b0b0b 3 days ago 5 replies      
I love project euler, but I've come to the realization that its purpose is to beat programmers soundly about the head and neck with a big math stick. At work last week, we were working on project euler at lunch, and had the one CS PhD in our midst not jumped up and explained the chinese remainder theorem to us, we wouldn't have had a chance.
4
FigBug 3 days ago 0 replies      
I was really into Project Euler when I had a job where I didn't have to do anything. I've solved 122 problems. Now I work for myself and don't have the time, as well I solved all I was able to solve. I last solved a problem in 2009 I think.

It's fun, I encourage everybody to do a few. Get past the easy ones at least.

5
henrik_w 3 days ago 0 replies      
Another good one for (more general) programming problems is Programming Praxis: http://programmingpraxis.com/
6
mixedbit 3 days ago 0 replies      
I love project Euler. A nice way to improve programming skills in a new language is to go through others solutions in the same language after you solved a problem. This allows to break bad habits. Say you are a C programmer learning Ruby or Lisp, 'C-ish' approach will often seem the most straightforward, but will rarely be optimal and idiomatic in the new language you are learning.
7
asgard1024 3 days ago 2 replies      
I solved about 80 of them, then my interest waned a little. But I wonder, are there any hints or recommended reading for the harder ones? Some of them I have no idea how to even start working on..
8
gaius 3 days ago 2 replies      
A dozen Project Euler solutions in a given language can be an excellent pre-interview candidate screening technique. Quite simple to check for plagiarism too, within reason.
9
dmunoz 3 days ago 0 replies      
A lot of good links to similar sites in this comment thread.

I enjoy Project Euler, but as with many people slowly got annoyed by lack of specific mathematical knowledge as opposed to programming. One thing I believe would really help with this would be a resource that discussed the problem in the abstract. As an example, for most of the programs that rely on using primes, whether it be iterating them (e.g. first 1M primes) or the unique prime factorization of a number, discuss the known algorithms in pseudocode. Perhaps this is a bit much, as I would be satisfied with just knowing the words I need to go find resources for myself. This is what I tend to do anyway after I have taken a fair stab at a problem: "Oh, I'm doing prime factorization. I wonder if there are better algorithms than I have used." Indeed, one resource for this is the forums that are made available after the problem is solved.

Some might see this as ruining the fun, but I would personally have more fun and solve more problems if this was available.

10
datawander 3 days ago 1 reply      
To be honest, I'm shocked this is on the front page as this website has been out for years and already notably mentioned, but I guess it's good to recycle very important websites for those who haven't heard of it.

My favorite problems is 98. This problem, along with the Sudoku one at 96, require much more careful programming than some of the others due the drastically fewer number of people who solved it compared to the surrounding problems.

11
bradleyjg 3 days ago 0 replies      
These are a lot of fun to do, especially in a new language you want to play with. However they are as much an exercise of your math skills (mostly basic number theory and combinatorics) as programming. One thing I'd suggest is that you pick an algorithm reference and stick with it, if you google anything too specific you will come across one of the many sites where people have blogged about thier solutions.
12
Karunamon 3 days ago 0 replies      
I'm trying to go through this with Ruby right now and having a lot of fun. Being a bit rusty on basic algorithms and higher algebra has not helped much, though.
13
blacksmythe 3 days ago 0 replies      
If you are not challenged by these problems, here is an alternative that I found considerably more difficult:

http://www.spoj.com/

14
captn3m0 3 days ago 0 replies      
Tangentially related: I made a pseudo-terminal web interface to Project Euler called CodeBot[1]. You can view problems, submit solutions, and do much more (some *nix commands work) in your browser. Its even open-source[2] on GitHub

[1]: http://codebot.sdslabs.co.in/

[2]: http://github.com/sdslabs/codebot

15
kozikow 3 days ago 0 replies      
In my opinion it may be better to do practice SRMs/Codeforces contests instead of project Euler. Topcoder rank imo tends to mean more, since it is timed. If someone says "I solved x problems on site X" you can't say if he done it in days or weeks of effort. If someone says he's red on topcoder you can say he's awesome.
16
kylemaxwell 3 days ago 0 replies      
For those interested I keep a list of these sorts of things at https://github.com/technoskald/coding-entertainment.
17
ahuth 3 days ago 1 reply      
There's only one problem for me with Project Euler. Eventually, the problems become more about coming up with the mathematical algorithm you need to solve it.

That may be what you want. However, a lot of these are outside my math knowledge/ability, without really expanding my programming ability.

18
doughj3 3 days ago 2 replies      
Project Euler is great but as others here have said it is very math focused. Can anyone share other programming challenge sites? I saw one the other day here on HN in a comment but can't find it again. The only thing I remember is the problem I checked out was a kind of AI / pathfinding for a "floor cleaning robot" and code was submitted directly in the page.

[Edit] Just found it going through my history: https://www.hackerrank.com/

19
donquichotte 3 days ago 3 replies      
Problem that has been solved by the smallest number of people (31): http://projecteuler.net/problem=453
20
selectnull 3 days ago 0 replies      
I love it, although I found I lack math knowledge to really be good at it.

I enjoyed solving a few of those problems using SQL, that was fun.

21
yankoff 3 days ago 1 reply      
Project Euler is great. Another one, but more algorithm and CS oriented: hackerrank.com
22
JakeStone 3 days ago 0 replies      
I always love this site for when things get a little slow and I think I could use some relaxation.

Then I remember that I only took a little bit of math, so then there's the research, the papers to read and decipher, the code to write, and I finally solve the problem and swear I'll never come back.

So, yeah, I just finished a batch of problems last week so I could get a couple of ego badges just within reach. 75 down, 379 to go!

23
lquist 2 days ago 0 replies      
If I see a github with 30+ solved Project Euler problems, 99% chance it becomes a hire.
24
aezell 3 days ago 0 replies      
I always liked this set of challenges/riddles, though it is directed at Python specifically. I appreciated that it forced you to deal with some Internet-related programming tools and concepts.

http://www.pythonchallenge.com/

25
wanda 3 days ago 1 reply      
Weird, I was just talking about this earlier when someone asked for productive activity on train journeys to/from work.

I used to do these problems years ago when I was still a student and later when commuting to London. I did as many as I could on paper before trying to program solutions. I'll have to log in sometime and finish the few I missed.

26
elwell 3 days ago 2 replies      
How does it work? Do you submit code or just input your answer as a number?
27
prothid 3 days ago 1 reply      
This site is great fun to tinker with a new programming language.
28
elwell 2 days ago 0 replies      
How does a site this old get so many upvotes? whatever, I guess it's worth bringing back into the collective consciousness.
29
careersuicide 3 days ago 0 replies      
Here's a little side project I've been working on for a few months: https://github.com/seaneshbaugh/rosetta-euler/

I've been a little busy lately so it's been neglected somewhat. Why is Prolog so hard?

30
sricciardi 3 days ago 0 replies      
I used it to learn the basics of F# and solving algorithms using a functional approach.
31
Sgoettschkes 3 days ago 0 replies      
Learning haskell with ProjectEuler right now. It's great and after solving it, one can always look up the forums and improve the own code or learn different ways to implement the solution!
32
veritas9 3 days ago 1 reply      
On CodeEval.com we have over 126+ executable programming challenges in 18 languages :)
34
yetanotherphd 3 days ago 0 replies      
The best and worst thing about project Euler is the binary feedback they give you: either you pass or you fail.

On the one hand, it is a good lesson in how hard it can be to write correct code.

On the other hand, real world problems aren't black boxes where you try an integer until you get the right one. Problems with multiple tests needed to pass (like topcoder) are much more realistic.

35
jbeja 3 days ago 2 replies      
I will start this with python.
17
Mother sen.se
282 points by rkrkrk21  3 days ago   192 comments top 76
1
nostromo 3 days ago 8 replies      
Wait, is this real? It seems like commentary on the current zeitgeist, not a real product.

> Mother. Mother knows everything.

> She's like a mom, only better.

> Sense: the meaning of life

edit: I see they are based in France, so perhaps the branding didn't translate well.

2
michaelwww 3 days ago 7 replies      
First I've heard of it, but I had the same reaction as Cringley. Maybe it's an age thing.

"Imagine v1 of Big Brother's -- or NSA director Keith Alexander's -- most inflamed fever dream: a sensorbot shaped like a Russian nesting doll wearing a Hindi-cow smile. Then terrifyingly name it "Mother" and build it specifically to monitor as many facets of your personal life as it can. Are you schvitzing yet?"

http://www.infoworld.com/t/cringely/sense-mother-may-i-the-m...

3
devindotcom 3 days ago 3 replies      
I played with this at CES. The "mother" bot is basically just a router. The little things only sense motion, and when I asked the lady said they had no plans to add any other types of sensitivity - temperature, moisture, light, current, etc. Compared with the other 'internet of things' kits out there battling for visibility, this one doesn't seem original or more useful, only visually striking. The tags are also pretty big for what they do. A useful thing for $50 maybe to buy once, but really doesn't seem like a worthwhile 'ecosystem' to buy into in any big way.

Also, I was unhappy to learn upon close inspection that the face is a sticker.

4
cromwellian 3 days ago 2 replies      
The way the thing is filmed with the smiley face and lighting up eyes, I could easily imagine a sci-fi horror film being based around it. :)

More seriously, the idea of using cheap motion trackers to track usage of things in the home is very interesting.

When Google acquires this, it'll make the Nest complaints pale in comparison. :)

5
CodeMage 3 days ago 7 replies      
That was a really poor choice of a name. It took me less than 10 seconds to start hearing Pink Floyd's "Mother" [1] in my head. Once that started happening, I just couldn't stay objective while looking at the pitch.

[1]: http://www.youtube.com/watch?v=p0HrrR9QDQU

6
vertex-four 3 days ago 2 replies      
As a young person who wants to remember to take her pills, to cut down on her soda consumption, to track how much she exercises (and maybe turn it into a game of walking further every week), and no doubt some more that I can't think of right now, this product sounds like it'd be amazing.

The video is a brilliant marketing asset. It showed me some very real problems of mine, and how it could help me solve them (by tracking things that I want to, and gamifying them).

The only issue is cost. As a young, single person, 166 is prohibitively expensive. It's likely not worth it for me. Is it worth it for people with families and kids? If they had 166 to spend, could they find something more pressing to spend it on?

7
pcurve 2 days ago 2 replies      
We all may be suffering from a case of Fortune 100 CEO syndrome. That is, we all wish our lives are so busy and important that we need personal assistants managing our lives.

So we buy these products that make us feel more important. It documents what we do, and it tells back our story through a dashboard, in an autobiographical way, as if we're some kind of celebrities.

But are we that important?

8
MartinCron 3 days ago 3 replies      
Just yesterday I posted a quasi-luddite rant about how these smart devices and services are infantalizing.

And now they're naming one Mother? I can't tell if I should feel vindicated or offended.

9
Jun8 3 days ago 2 replies      
Same French company that created the successful Nabaztag rabbit and then couldn't cope with the traffic. I had my wife buy me one of those for Valentines Day (stupid, I know) and after trying to do something useful with it and getting frustrated I tossed it somewhere in my cube where it remains to this date.

Apart from the super bad naming and Branding, this is another reason for me to stay away from this mother rabbit.

10
fab13n 2 days ago 1 reply      
I haven't been so creeped out by an ad for quite some time.

This looks like a solution desperately looking for a problem; that is, unless your problem is "I want to spy every single step of everyone in my family".

And seriously, "mother"? Do they even ever had one, to be that much off-mark? The very first quality of anything motherly is to be human; this is a wireless log collection system. Call it a "warden", or at best "coach", if you really need an anthropomorphic comparison.

11
cracell 3 days ago 1 reply      
Cool product but very creepy branding. Might be ok to keep the name Mother but shouldn't be emphasizing it as a "mother" on the site at all.
12
gjm11 3 days ago 0 replies      
This is the single creepiest thing I have seen in the last month.
13
dmazin 3 days ago 1 reply      
God, the future is so fucking weird.
14
dictum 3 days ago 0 replies      
Mothers watch their sons out of love and genuine care for their wellbeing. Mine did a bad a job and that's why my next sentence will be bitter:

If a company wants to make me use a telescreen, they might as well make it a suppository.

15
aray 2 days ago 0 replies      
I'm surprised it doesn't have wireless. Places I've lived always have the router tucked away in some inaccessible closet.
16
hrktb 2 days ago 1 reply      
They would have called this little sister, they would avoid so much of tastelessness surrounding their current branding...that put apart, what it's doing is already 50% doable by current smartphones + an arm band eventually (alarm , podometry, sleep control), and the other things it's trying to solve doesn't seem to be solved in a reliable enough fashion.

You'll have to update your coffee capsule count every time you buy them. Buying new packs when the opening the last set of capsule is ny far the easiest way to manage I think.

You'll have to put the sensor on every bottle you drink.

If you care that much about toothbrushing, buy an electric toothbrush. A timer will be integrated telling you when you pass the 2 pr 3 min mark.

Central temperature management would need a programmable device anyway, you'll basically need a Nest I guess.

At the end, It doesn't seem so easy to use in practice, it will forward every life information to an external server, and half of what it does is better solved an other way.

17
nilkn 3 days ago 1 reply      
> we reinvented mothers

> Mother knows everything (in red text at that)

> She's like a mom, only better

The branding of this is either creepy or crazy. Maybe it's a bit of both. But I'm certainly not going to forget it, and the idea itself seems pretty interesting.

18
atmosx 2 days ago 0 replies      
Apart from the privacy issues, which the community already raised here, there's another fact that bothers me: Applications do can not discipline yourself for you.

I have tried many applications which should increase by productivity, sleep quality, or you-name-name-it. I don't recall a single one that managed to do so in the long run (most not even in the short run...).

So either one is open to change and that has little to do with technology or... You're toasted anyway. But even when you decide to change for yourself and not because a notification tells you to do so, these technologies become time consuming and troublesome to use. Of course they look nice on TV and ads, but in real life, most of them are frivolous IMHO.

19
woofyman 3 days ago 0 replies      
It may be an age thing, but it find it creepy and useless. I haven't needed a Mom since I left home at 18.
20
sheraz 2 days ago 0 replies      
More technology where none is needed. How about just being responsible and accountable for our own actions?

  Want to improve ___ about yourself?   Just do it(tm).   Get it done(tm).   Do what it takes. 
I don't need devices and a dashboard to tell me I'm winning at life.

Fuck this arrogant and stupid product.

21
notlisted 2 days ago 0 replies      
I like the concept. Surprised so many here are bugged by the marketing and/or the technology. Maybe you don't have kids (yet)?

Above all, I like the neat interface of the apps (mockups?) and simplicity of the cookie sensors. No charging nonsense, because they measure and buffer stuff but don't transmit. 1yr battery life. 10 day memory.

Sure, I'd love to see additional, more advanced cookies that would require charging, e.g. with built-in LED or vibration (reminding me when I enter or leave, though my phone could serve that purpose), Data, GPS (though my phone could serve this purpose, need an app that intercepts an SMS after presence is detected to auto-upload GPS data), multi-mother stuff (one at work, one at home), integration with home automation systems (someone below mentioned frequencies indicate zigbee), Zapier/IFTT support and above all some sort of data input/output API so I can import my own data points.

By the way, $222 for a mother and 4 sensors seems quite affordable to me.

The only thing that prevents me from pre-ordering a set is that the NaBazTag history doesn't exactly instill much confidence in the makers' ability to support this thing in the long run; I also wonder where the data is stored and if it's remotely future-proof (data import/export/backup).

22
buro9 3 days ago 1 reply      
The sync module reminds me of the Nabaztag ( http://en.wikipedia.org/wiki/Nabaztag ) and I wondered whether Mother was going to have signals and indicators so that you didn't have to use a mobile device for insight.
23
ameswarb 3 days ago 2 replies      
Their tagline "Mother knows everything" is terrifying.
24
state 3 days ago 0 replies      
I like how open-ended this thing is. I wonder if the market is actually ready to move beyond domain-specific sensor hardware and in to something broader. The aesthetic isn't quite my taste, but I'm very curious to see how their users react.
25
tomphoolery 3 days ago 0 replies      
This is fucking creepy. But like most creepy things, the idea is also kinda neat. :)
26
RutZap 2 days ago 0 replies      
I want everything Mother knows/finds out, to be stored locally (i.e. on my pc, not in the cloud), to be kept secure, private and I want to access it at any time from anywhere.... can Mother do that? I don't think so.

Still pretty good but as long as there isn't a privacy promise that would satisfy the basic security principles (Confidentiality, Integrity, Availability) I don't see it as a successful device.

27
anonu 3 days ago 1 reply      
I think this is really cool and definitely brings us closer to the Internet of Things. I don't think I would have anthropomorphized the system by calling it "Mother" and putting an eerie LED smiley face on the base station.

I can't seem to find any technical info on the "cookies". Are they similar to the technology in the Fitbit Flex, ie Bluetooth Smart coupled with some sort of accelerometer. If that's the case, do the cookies need to be charged every week. This remains the single massive downside to widespread adoption of such devices.

28
zxcvvcxz 2 days ago 0 replies      
As I saw it popping up, I thought "wow that looks like a sex toy." Freudian slip, whoops.

"Sense Mother is at the head of a family of small connected sensors that blend into your daily life to make it serene, healthy and pleasurable."

You never know.

29
jds375 3 days ago 1 reply      
Seems like a pretty cool product. They have an amazing design and a beautifully done website too. Only thing I am a bit concerned about is the price. It costs 222 USD for a base unit and 4 cookies (sensors).Here's a video from CES2014 about it http://www.youtube.com/watch?v=024OPHSgOqo
30
cm2012 2 days ago 0 replies      
To me, this is pretty awesome and not at all creepy (22). A friendly UI and ease of use for life tracking? Yes please. Attach it to barbells to track workouts.
31
thatthatis 2 days ago 0 replies      
What does it do?

I scrolled to the bottom of the page expecting some kind of explanation of what it is and why I would want it or what need or want it solves. Nothing that I could find.

32
nathan_f77 3 days ago 0 replies      
This is horrible marketing. Seriously, who came up with this creepy design and name.
33
carls 3 days ago 1 reply      
This seems to be a herald to the situation described in the poem All Watched Over By Machines Of Loving Grace (1967) by Richard Brautigan.

  I like to think (and   the sooner the better!)   of a cybernetic meadow  where mammals and computers  live together in mutually  programming harmony  like pure water  touching clear sky.  I like to think  (right now, please!)  of a cybernetic forest  filled with pines and electronics  where deer stroll peacefully  past computers  as if they were flowers  with spinning blossoms.  I like to think  (it has to be!)  of a cybernetic ecology  where we are free of our labors  and joined back to nature,  returned to our mammal  brothers and sisters,  and all watched over  by machines of loving grace.
And yes, incredibly creepy.

34
Geee 2 days ago 1 reply      
What's going on in here? I don't get the negativity. I think the branding is great, and I realized the function of the product immediately. Also I think they presented it in a funny way (programmable mother). It's obvious that it doesn't 'know everything', that was a joke. It has simple sensors and you can collect data from those, there's nothing scary about that. The product is interesting, but however not useful at least for me.
35
EdZachary4 3 days ago 0 replies      
They need the companion "Father - Common sense" to tell you not to waste your money on nonsense like this.
36
avighnay 2 days ago 1 reply      
This thread is a good example why name matters. If the same product was given any other name, perhaps it would not have been noticed that much. A set of motion sensors with a central comm hub.

The makers perhaps thought that the name 'Mother' would evoke care and love in the minds of their users. To their agony, it is revealing in the thread that though most people love their mom, they really do not want to be a 'watched over' kid.

I guess it gives all of us that creepy feeling of guilt as kids when we stole from the cookie jar and kept turning our heads with fear of being caught by mom :-)

Note to self: keep relationship names away from product names, too much friction ;-)

37
girvo 2 days ago 0 replies      
So, Defcon last year had a talk where they hacked things (including a Bunny ostensibly for watching your baby in another room) like this. And it was super easy. I wouldn't out this anywhere near my house or life.
38
rglover 3 days ago 0 replies      
Will it send me a notification that says "don't disappoint mother" if I forget to do something?
39
samolang 2 days ago 0 replies      
Interesting concept. Simplify the sensors as much as possible and do all of the work in the software. I'm guessing they have profiles that allow you to determine how a sensor's data is interpreted. I wonder if they allow you to define custom profiles or at least have access to the raw data.
40
Yetanfou 3 days ago 0 replies      
Apart from all the other emotions which this plastic big sister evokes, I wonder what it is that makes so many of these startups reach back to the crib when it comes to branding their products. From this bastardized matryoshka doll through Snapchat's Miffy-like ghost to Twitter's tweety to just about half the iconography on tablets and smartphones, they all have one thing in common: the more infantile the logo and/or branding, the better it is. Is this idiocracy at work or are they all following some celebrity psychomarketeer's edict about successful marketing to the attention-span deficient generation?
41
pnathan 3 days ago 0 replies      
Like other people: it's an interesting idea, but the branding is dystopian.
42
dennisz 3 days ago 0 replies      
If you scroll down far enough, you get to the 'technical details', where the device is described as 'a white mother'. I just found that funny, haha.
43
jawr 2 days ago 0 replies      
Did anyone notice this:

"Cookies immediately send everything they capture to the nearest Mother."

For me, that's a bit of a scary statement considering how intimate the product is meant to be in someone's life.

44
owenversteeg 2 days ago 0 replies      
I personally think that I don't need to spend $222 on something that seems to be minimally useful. I don't need to monitor how often I brush my teeth, how often I drink coffee, how often I water the plants, and how often I take medication.

For the things on the list that are somewhat useful (like sleep logs + a pedometer + temperature) I have a 1975 pedometer/calculator combo that's worked fine since the day I got it, a notepad, and an infrared thermometer that is a thousand times cooler.

I think the only people that will buy this are people that want Google Analytics for their life.

45
xianshou 3 days ago 0 replies      
Who knows you better than your mom?

From this marketing, I'd answer...Big Brother.

46
paul9290 2 days ago 0 replies      
This looks really cool, though I had no idea what this product did based on their bloated landing/homepage.

Just a few picture examples with blurbs of text & a demo video would suffice rather then an infographic type of website.

47
Roelven 2 days ago 0 replies      
Happy to see the guys behind Violet are not giving up. I believe they're on to something but the branding / language choosing is indeed poor. Whatever they launch with now will surely be extended, I'm hopeful that they've learned a great deal with the Nabaztag (which I've owned one back in the days).
48
webXL 2 days ago 0 replies      
Cheese.... wait for it... E!

And can I see the product without having to hunt down my country in a gigantic freaking select box first?? Isn't it fundamentally the same in every country?

49
themoonbus 3 days ago 0 replies      
I was hoping for news about an Earthbound sequel, and instead I got this weird little smiling pod.
50
Houshalter 2 days ago 0 replies      
I can't wait to have an AI virtual assistant that can keep track of and manage all these mundane statistics for me.
51
dlsym 2 days ago 0 replies      
> Mother. Mother knows everything.

I guess it's "Big mother is watching you" then.

52
jnardiello 3 days ago 0 replies      
Beside the branding thing, i've lost my fitbit one in less than 2 weeks. How long till i lose one of the cookies? Dongles are not for me.
53
forgotprevpass 3 days ago 1 reply      
Does anyone know how the signals are being sent from the cookie to Mother? The company mentioned in a CES video that they werent using the traditional bluetooth, wifi, etc.
54
jrochkind1 2 days ago 0 replies      
Are they TRYING to scare me?
55
Shtirlic 2 days ago 0 replies      
Looks like this was done before in Green Goose project in 2011 http://www.engadget.com/2011/02/23/green-goose-sensors-monit...
56
protez 2 days ago 0 replies      
This is one of the most horrendous brandings I've ever seen. Maybe, the horrible branding is intentional, but it's damn too creepy for sane users who dare using their product.
57
treenyc 2 days ago 1 reply      
Very nice, however unless all the hardware and software are open sourced. I will not use it in my real life.
58
75lb 2 days ago 0 replies      
advertising slogan: "It's not enough to pipe every move, call, text and click you make from your smartphone? Mother's sensors are cute, small and funky. Collect more data for US spies today!"
59
MiWDesktopHack 2 days ago 0 replies      
kill it. kill it with fire.this product collects the kind of personal data that should not be handed to third parties. too ripe for abuse. too much insight into your existence.a scary Orwellian nightmare.
60
TeeWEE 2 days ago 0 replies      
So when I'm living in the Netherlands I cannot continue? (its not in the list)
61
sifarat 2 days ago 0 replies      
I don't how I feel about this, right after watching the 'her' trailer. I am speechless.
62
xname 2 days ago 0 replies      
Watching the video. First I liked it..... Then I hated it. It's too much. I don't want that kind of life. I don't want Mother to watch at me everyday every moment.
63
nfoz 2 days ago 0 replies      
Have these people ever seen a mother before?
64
ilitirit 2 days ago 0 replies      
What does this gadget do?
65
Houshalter 2 days ago 0 replies      
Can people please come up with better names for things, especially not common English words?
66
dpweb 2 days ago 1 reply      
Expected Pink Floyd link..
67
mikegriff 2 days ago 0 replies      
Hmm, Ireland doesn't exist. I guess they don't want me to get one, or find out about it.
68
grumbles 2 days ago 0 replies      
I got confused for a second and thought I was still reading 'The Circle' while reading this page.
69
lcasela 3 days ago 0 replies      
The son could have easily tricked the sensor.
70
meandyounowhere 3 days ago 1 reply      
Concept is stupid as FK. Why you need sensors just to know some basic stuff such as taking pills, tracking health etc. You can use app also. All they are doing is using sensor( motion sensors in particular) and send message to your phone. So why would I spend $222 for something where I could just it with $10 reminder app ?
71
pyrocat 3 days ago 0 replies      
...creepy...
72
elwell 2 days ago 0 replies      
Doesn't look ready.
73
indigromer 2 days ago 0 replies      
I know I'm an unfortunate case but as someone with a recently deceased mother I do not like this one bit.
74
diu_9_commerce 3 days ago 0 replies      
Bad name - I hate the fact that mother knows everything.
75
michaelrhansen 3 days ago 0 replies      
makes me want to go bowling
76
rambojohnson 2 days ago 0 replies      
ech, creepy.
18
PSD to HTML is Dead teamtreehouse.com
279 points by nickpettit  3 days ago   162 comments top 46
1
wpietri 3 days ago 6 replies      
And thank goodness. If anybody knows where the grave is, I'd like to go piss on it.

As somebody who long ago did print design, I totally get why designers would want pixel-perfect control. It is awesome, but you get that in print because you are physically manufacturing an object and sending it to people. The web was device independent from the get-go. It wasn't your paper anymore; it was their screens. There were a couple of designers I came close to beating to death with their own Pantone books because they refused to get that.

Sadly, the desire for pixel perfection led to trying to force every single user on the planet to conform to the designers' weaknesses and fetish for control. For example, every Flash intro in the world. Or all of the goddamn fixed-width "experiences" that were either too wide for what users wanted their window to be or so narrow that acres of space were wasted. An approach that surely looked fine in presentation to executives, but much less well for actual users.

The great improvements in CSS have definitely helped. But I think the major changes have been the the explosion of form factors (small laptops, giant desktop monitors, tablets, phones) and the rise of a generation of designers for whom the web is a native medium. The old paradigm got harder to force at the same time there were plenty of people who were thinking in a new way.

Planck wrote, "A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it." Design, like science, proceeds one funeral at a time. So goodbye, PSD2HTML, and let's quietly put a stake through its heart so it never returns.

2
reuven 3 days ago 6 replies      
It drives me totally batty to work on projects in which the designer assumes that their only responsibility is to provide a PSD file, which the developers will then turn into HTML and CSS.

I want to work not just with designers, but with Web designers, who intimately understand the workings of HTML, CSS, some JavaScript, and the implications for different browser sizes and versions. Web designers speak HTML/CSS natively, taking these limitations and issues into account when they're creating their designs. And if something needs to change, they can change the HTML/CSS that was created. If the designer only knows how to work with Photoshop, every change to the site requires a great deal of additional work and communication.

I've sometimes remarked that a designer who uses Photoshop, but who doesn't know HTML and CSS, is like a photographer who refuses to actually touch the camera, and instead tells someone else how to aim, focus, and shoot. (And yes, I'm aware that TV and movies work this way; the analogy is far from perfect.) I want to work with someone who lives and breathes Web technologies, not who sees them as just another type of output. I'm glad that this blogger made this point, and has indicated that while Photoshop might once have been acceptable, it no longer is.

3
IgorPartola 3 days ago 7 replies      
Rant to follow:

So I have done a fair share of PSD to HTML, PSD to WordPress theme, PSD to application web GUI, etc. rewrites. I generally have no problem with the concept of this, and got quite good at this. However, there are some real pet peeves that keep coming up in this workflow, that are really driving me crazy. If you are a designer working with a developer, and you happen to read this, at least please consider it the next time you produce a PSD:

First, PSD's that assume text length. For example, if you have three call-out boxes with a title and some text to follow, don't assume that the title will always be one line and the text will always be the same length. Instead, figure out what this will all look like when you do have very uneven amounts of text. Do we center it vertically? Do we abbreviate it?

Second, PSD's that don't assume a responsive design. Sure, working directly in the medium (HTML/CSS) would solve this, but you can still provide some direction here. Tell me how the columns should be laid out. Which parts of the site should expand/collapse with size, which parts can be hidden, etc.

Third, and this goes without saying, but clean up the PSD layer names and groupings. Layer 1, Layer 2, etc. is not a great convention for this.

Fourth, show me the unusual cases. I know the clients always want to focus on the prominent pages, like the home page, the product listing, etc. Those are important, give me those. But also give me what a form submission error looks like. Or what a 404 page looks like. Or an empty shopping cart. Or pagination. Or a table that's wider than the viewport would normally allow.

Fifth, consistency. It sucks for the developer, and I'd argue it sucks for the user, to have every page use a slightly different set of CSS rules for headers, paragraphs, lists, etc. Best case scenario here is to give me a style guide I can trust. I know it's two different documents you now need to maintain, but honestly this is the biggest help you can give me.

Sixth, show or describe to me the interactions and workflows. A simple shopping cart can become a giant minefield of interpretations of what the design is supposed to convey.

Seventh, and this is a bit meta, but don't walk away from the design before a single line of HTML/CSS is written. This is bad because there will be questions about interactions, etc. If first I have to email your boss's boss to try to see I can ask you a simple question, the process is broken and I will not recommend working with you again.

Eighth, if you do promise to deliver sample HTML/CSS, for the love of good, do this well. I have recently had the misfortune of having HTML/CSS/JavaScript delivered to me for a large site redesign by a big name web design agency. I was very excited about this, especially since these guys said they would use Bootstrap as the foundation for this so that we would have all the benefits of that framework built right in. I got the files, opened them and OMG. It did include Bootstrap, but in name only. After that declaration, it instead included a completely custom column system that was just slightly incompatible in sizes with Bootstrap's. It also used none of the same class names even where it made sense, etc. Needless to say, I had to re-write all of their CSS from scratch, and re-adjust lots of the Bootstrap variables to accommodate their column system.

</rant>

Great designers are worth their weight in gold. The above highlights that the waterfall process of design -> develop does not work. Instead it should be design -> develop/design/develop. If you cannot step outside of Photoshop that's fine, but if you want to be efficient, you must know the final medium, which is the web.

4
bbx 3 days ago 0 replies      
I'm currently redesigning a backend interface, and it's the 1st time since I've started my Front-End career (7 years ago) that I'm not using Photoshop at all. I'm just using Bootstrap, Sublime Text, and Chrome.

For many projects of course, it won't be sufficient: clients want (and probably need) a stunning Photoshop mockup to provide feedback and boost their self-assurance.

But if you combine a simple CSS framework (even if it's just for a grid system), Chrome's inspector, a selection of Google Fonts, and some sense of "flat" aesthetics, you can come up with a more than decent, and sometimes amazing, design. Plus, it takes 70% less time, especially considering it's usable right now.

37signals mentioned this "skipping Photoshop" attitude in 2008 [1], but I never quite managed to put it into practice until recently.

[1] http://37signals.com/svn/posts/1061-why-we-skip-photoshop

5
Trufa 3 days ago 3 replies      
I'm a little bit confused of the workflow they are suggesting.

I'm a web developer with "good design taste" but I definitely can't design myself, I always pair up with a designer that does the PSD.

But of course this doesn't mean that when I see a navbar that has a gradient I copy a paste the image of the navbar in my website with a <img>, my job is porting this images to HTML, CSS and JS.

If you're actually putting images from the PSD, you're definitely doing it wrong, but in my case, I still need a highly detailed design that I can make a website, otherwise I have to design it myself, wireframes only get you that far.

When I'm working with a good designer, that knows about how the web works, I feel it's a great workflow.

6
rwhitman 3 days ago 2 replies      
I swear I feel like I've read a version this article once a year since the advent of CSS. This is a naively utopian vision of the future. The designer/developer is a very rare breed outside of the HN community. Most designers can't / won't write markup or CSS, and most developers are piss poor designers. The design->planning->building segmented workflow will always exist, as it has in all engineering disciplines since the dawn of human civilization.
7
dredmorbius 3 days ago 0 replies      
As a mostly back-end guy (systems, network, databases) who's dabbled in HTML and CSS, somewhat increasingly over the past few months (latest results below), I've taken a highly pragmatic approach to how I prefer sites styled:

Consider the screen as a sheet of paper on which you can 1) communicate your message 2) provide a UI, and 3) apply your branding. Modest amounts of logo / artwork, color palette, and styling touches go a long way. Other than that, it's a rubber sheet. There are no fixed dimensions.

Start with a basic HTML5 framework. body, header, article, aside, footer.

Put minimal elements above the fold. Your header, logo, and some basic navigation. Emphasize body text and / or UI.

You almost always want to design around the text. That's your payload. For interactive tools, controls layout should be clear, consistent, logical, and most of all provide enough space to meaningfully navigate options. For that last: size-constrained modal dialogs or their equivalents (pop-up menus, etc.) are strongly deprecated. Unless the user needs to see other content while performing input, that dialog should be front, center, and the principle screen element.

CSS gives you a whole slew of tools: special selectors, including :hover, :active, :first-child, :last-child, :nth-child, :nth-of-type, shadows, columns, and more. No, MSIE legacy doesn't support many of these. Fuck'em.

Stick to light backgrounds and dark fonts, with few exceptions. http://www.contrastrebellion.com/ is strongly recommended.

Think of your page in either ems or percentages, and almost certainly ems (scaled to your principle body font).

Provide a minimum page margin of around 2ems for desktops. For mobile, enough to keep text from flush with the edge of the screen, 0.25em typically. Don't crowd your text. I accomplish this by setting a max width (usually 45-60ems depending on context), and a 2em left/right internal padding. This provides a comfortable reading width but preserves margins in narrow displays.

Scale fonts in pt, or use relative/absolute sizing based on the user's preferences. I recommend "medium" for body text.

Other than image elements and logos, avoid use of px. Never mix px and ems (say, for line heights).

Rather than a traditional sidebar, use CSS column elements for your asides, which are then full-screen width. @media queries can toggle between 3, 2, and 1 column views.

If you've got to float elements, float right of text rather than left. This is less disruptive to reading. 0.5 - 1em padding or margins is usually appropriate.

For long lists, I'm growing increasingly partial to "li { display: inline;} or inline-block (the latter allows trick such as ":first-letter" but fails for wrapping.

Make modest use of dynamic elements. I'm generally not a fan of flyouts, automatically opening menus, etc., and they're among the first elements I nuke when modifying sites. Color shifts to indicate links and other dynamic elements, however, can be useful. Google's "Grid" is a notable exception to this rule.

Don't fuck with scrollbars. Allow the user environment defaults. Yes, Google+, I'm talking to you.

DO NOT USE FIXED HEADERS OR FOOTERS. Far too many displays are height-constrained, and robbing another 10-25% of the display with elements which cannot be scrolled offscreen is an insult. If you've got to fix something, put it in a margin. Do not fix ANYTHING for mobile displays.

CSS modification: Metafilter lite http://www.reddit.com/r/dredmorbius/comments/1v8fl5/css_adve...)

8
tomatohs 3 days ago 1 reply      
This article should be titled "the slice tool is dead."

The slice tool represents the direct transformation of raster image to website. We all know that this isn't possible anymore because of mobile, retina, etc.

But Photoshop and image editors still provide tremendous value to the web development process for mockups, image assets, colors, etc.

What this article is trying to say is that the process of turning a design into a website has become much more difficult. A PSD is no longer a final deliverable but the beginning of a conversation.

Now design needs to be functional. Instead of taking the static image you get from a PSD, you need to ask "What does this look like on mobile? What about huge resolutions? What if we don't have that content?"

The article suggests that this process will be improved by designing in the browser thanks to CSS3.

The truth is that the browser has just barely hit the minimum requirements to be able to make design decisions. Have you seen the Chrome color picker? It's alright for choosing a border color but final design work can not be done entirely in the browser just yet.

9
elorant 3 days ago 7 replies      
As a developer I hope that CSS would share a similar fate sometime in the not too distant future. Its freaking hideous, doesnt work as it should and in order to build any decent modern site you end up writing something like 5,000 LoC. Nine out of ten times I want to do something with CSS I prefer doing it with JavaScript.
10
efsavage 3 days ago 2 replies      
I disagree. In the hands of a competent web designer, photoshop is still the most expressive tool available. I've been bouncing PSDs with a designer for the past couple of weeks and I want him being creative and making something beautiful, not constantly worrying about how the images are going to get sliced up or sprited or what's svg and what's not. That's my job. So long as there is in iterative process in place where I can keep him within the bounds of reality, it all works out very well in the end.
11
danboarder 3 days ago 0 replies      
Photoshop may be dead as a starting point, but not quite dead as an intermediate step for customized template design. A workflow that works today for quick site turnaround in commercial web design is to screenshot a Wordpress or other CMS responsive template, bring that into Photoshop, drop in branding, color changes, and replace content to produce a comp for presentation to clients. It is still quicker to make design changes in this Photoshop intermediate phase. Once the design is signed off, it's fairly easy to customize the CSS in the original template and arrive at a branded site the client is happy with.
12
callmevlad 3 days ago 0 replies      
The pain of the PSD->HTML workflow, especially around responsive design, is one of the reasons we're working on Webflow (https://webflow.com). While Photoshop will have a critical role in web design for a long time to come, having to deal with multi-resolution elements is extremely tedious.

Also, Photoshop layer styles are way behind what's actually possible with CSS3 these days (multiple shadows, multiple background images, etc), so designers who have to implement a website end up doing their work twice. With a tool like Webflow, implementation work is part of the designer's workflow, so once something looks good on screen, it's actually ready to ship.

Granted, designers have to learn the base concepts of how content flows in a website (the box model), but I think that's a small price to pay for designing directly in the intended medium.

13
tn13 3 days ago 0 replies      
Thanks goodness. My life was hell when I was working for an Indian outsourcing giant where they made web application like an assembly line.

The designer were hired from school which taught only print media design. They made PSD mockups which arrived at frontend developers who would then make HTML out of it with dummy data.

For example say you are designing a charting app for a banking company, They would create pie chart in PSD and then ask the frontend devs to convert into HTML. So these people use to put those charts as image. When it arrived with us the backend team, we use to realize that this graph needs to be dynamic. If we use any other charting library it use to look ugly with overall design.

Not to mention if the webpage does not look pixel perfect in FF and IE it would go as a bug. Countless human hours were wasted in making corners round in IE.

The real interesting part was that, the baking giant did not give a shit about the design in first place neither about the browser compatibility. It was meant for their say 30-40 employees who could simply switch to FF if they did not like sharp edges in IE.

In the battle of egos between the designer and testers we were screwed.

14
at-fates-hands 3 days ago 0 replies      
I actually stopped working in Photoshop about two years ago when I realized you can prototype faster just by building a design from scratch in the actual browser.

It's so much faster than having a designer painstakingly mock something up in PS, then have me build it and realize a myriad of things that weren't apparent because we weren't looking at it in an actual browser.

15
sandGorgon 3 days ago 0 replies      
This whole post, and comments, sound extremely unrealistic. In an ideal world, things work as you would say - but in the real world, things don't work like this.

I'm not sure if any of you guys have seen the inside of a psd2html place - it is highly optimized with a hive mind around browser compatibility. I would say that best of the breed slicers leverage bootstrap, sass/less, etc and incorporate their experience inside it.

I would argue that the missing piece is not some new, magical way of doing things - but rather the interchange formats. For example designers don't use PSD grids that account for fluid layouts (FYI - I'm not even opening the can of worms that is responsive design). This makes it hard for slicers to deliver fluid layouts.

The search for the mythical designer + SASS engineer is very hard and very likely futile. In fact, my opinion is that you are starting the process incorrectly. I suggest to find a best of breed slicer, START the design process with them as opposed to a designer (get their recommended grids, etc) - then give the designer a set of constraints to work with. This should ensure your downstream workflows are smooth.

16
anthemcg 3 days ago 0 replies      
I am not here to say that web designers should create PSDs and just throw them over the fence.

But, I don't think most web designers really agree with this. I think this philosophy really tries to downplay visual style to practical problem solving and I believe they are both essential.

I can write competent HTML/CSS/JS, Frameworks etc. At least, I know enough to work with engineers and work effectively in my projects. For me using Photoshop isn't just about what browsers can and can not do. Its certainly, not just about pixel perfection or making a design ready to code.

Working with HTML is just clunky. Working with paper is too loose. I can think about how to build a design, plan it on paper but exploring visually is actually quite constrained by trying to do it with markup or just paper/wireframes. Photoshop represents an open environment where I can create anything I need from an illustration to a button and its powerfully close to what it will really look like. To some people that might sound like a clunky or wasteful step but I think it really helps.

For sure, I think Nick makes some great and valid points here. I agree, there are problems with the PSD process but direct prototyping and CSS frameworks just don't solve those problems.

I don't know, I feel like if in reality everyone used HTML to design, everything would look like Bootstrap and that would be acceptable.

17
ilaksh 3 days ago 0 replies      
I agree that PSD to HTML is generally now a bad idea that will make the task more difficult.

However, I believe that the idea of having an interactive design tool should not be abandoned so easily.

I believe that we should create interactive GUI design tools that support the back-end encoding.

I know that doesn't meld well with hand-coded and maintained approaches.

I believe that we can create design tools that output acceptable markup. But I don't think we have to.

I think that the business of writing code in order to layout a user interface is ludicrous. I do it, because thats the way most everyone does it these days. Most everyone also drives massive 5 passenger vehicles as the sole occupant that waste huge amounts of energy driving to and from work every day. Point being, just because that is the way people do things, doesn't mean it makes sense.

Programmers by definition write code. If you're not writing code, you're not a programmer. The problem is the definition of programming needs to be updated, since we now can create very sophisticated programming tools that have friendly user interfaces.

18
tomkin 3 days ago 0 replies      
I don't know what the author at Treehouse is doing, but I use the PSD as a visual representation of what I will end up creating as CSS/HTML/JS. Who was still seriously drawing grids and cutting out PNGs/JPGs?

The take away for many reading this article is going to be: Photoshop is not the way to design a website. The article does attempt to address this is not the case, near the bottom.

In the end, the author admits that you do need some design reference point (Photoshop, Illustrator, paper, etc). I do remember the days of cropping out many images, backgrounds, etc., but that was at least 6-7 years ago.

19
ChikkaChiChi 3 days ago 0 replies      
We don't live in a world where every web user is part of a majority of three monitor resolutions and web design has changed to accommodate that. Web sites need to scale properly and that cannot be done with raster graphics.

If you are using a raster program for anything other than mockups before you head into real design, you are doing yourself, your clients, and their customers a disservice.

20
wwweston 3 days ago 0 replies      
Well, as long as we're making controversial statements (those in the "____ is dead" usually are)....

I think Photoshop as a design/layout tool may have done more damage to front-end design/development productivity than Internet Explorer. And this article is just an indicator that there's a growing awareness of how.

Photoshop is an amazing raster image manipulation tool. But the dominant mechanics have always been about composing a series of fixed-dimension bitmapped layers (outside some shoehorned not-quite-layers-but-actually-layers there's really no other kind of entity to work with). For that reason there's always going to be an impedance mismatch between the tool and the web.

21
tlogan 3 days ago 2 replies      
What is the best HTML page design tool? I.e., designing CSS and HTML with minimum coding?
22
seivan 3 days ago 1 reply      
PSD to iOS as well. I just wish companies would stop wasting resources on photoshop goons and let the engineers who work with the platform & SDK design.
23
discordian 3 days ago 0 replies      
He may wish it was dead, but I can assure you there is probably more PSD to HTML work going on now than ever before.

First of all, it seems the author is not even opposed to the idea of mocking up a design in PSD. He just thinks that responsive design and advances in CSS have altered the process somewhat. OK, point taken, but this doesn't make the overall concept of PSD to HTML obsolete by any stretch of the imagination. The majority of designers will always favor mocking up their intended design in a program like Photoshop, and using that as a starting point for the development process. Responsive design just adds an additional layer of complexity, which may call for additional mockups.

I've heard people advocate prototyping concepts directly with HTML/CSS, but this is ultimately a rather inefficient way to work if you are a detail-oriented designer.

As far as the actual workflow changing and becoming more iterative, it completely depends on the context. Not everyone works at a company like Treehouse that has a team of in house developers and designers. Many website projects - the majority even - are the result of small businesses subcontracting the process out to various companies. It's not always possible for the designer and the developer to be in the same room. So as an ideal - sure, the designer should be involved throughout the process, but this doesn't always match the reality.

24
zx2c4 3 days ago 0 replies      
The work flow might be dead but... psd.js lives on!

http://git.zx2c4.com/psd.js/about/

    git clone git://git.zx2c4.com/psd.js
This is a neat project from `meltingice`.

25
mgkimsal 3 days ago 0 replies      
Yay. I'm surprised it was ever a thing, really. Maybe not surprised, but pissed off. We've all got our horror stories - I got a PSD with > 200 layers (4 layers for 4 rounded corners on a button - WTF). It was just crazy.
26
mratzloff 3 days ago 0 replies      
tl;dr Most browsers support modern CSS techniques that remove the need for image-based techniques, and mockup tools like OmniGraffle and Balsamiq make it easy to create layout drafts.
27
goggles99 3 days ago 1 reply      
Link bait warning. Author even admits in the comments thatwhat he means is "Going directly from a PSD to an HTML file is dead".

Link bait may get you more traffic in the short turn, but will likely just hurt you in the long run. Especially since lots of people think that he an idiot now.

Why? Who would have thought... Modern day web dev needs to be rendered to different sized screens and we have CSS3, more skills, and better tooling now.

Who does not know this already. I was baited and now he is hated (JK)...

PSD is still used quite commonly for conceptual purposes. Of course no one expects anymore (did they ever?) that it will be pixel perfect across devices ETC.

28
atomicfiredoll 3 days ago 0 replies      
"Everyones workflow is different and nobody knows how to make the perfect website. You should always do whatever is most effective for you and your colleagues."

Not to say that there aren't some valid points brought up, but this feels like dramatically titled click bait with a weak conclusion.

When I click a title like this, it's because there is an implication that a better process exists--I want to know what that process is! At best, it's only hinted at here.

I know teams that are using processes similar to the PSD oriented ones outlined in the article very successfully. I suppose that means that it's not dead for them, as it's effective.

29
lstamour 3 days ago 0 replies      
I'm surprised no one here's mentioned Photoshop CC's Generate function yet, especially given that it was written in Node.js: http://blogs.adobe.com/photoshopdotcom/2013/09/introducing-a...
30
Bahamut 3 days ago 0 replies      
As a frontend developer, I like having designers figure out the look of a page, and implement the look in a way that doesn't break what I've implemented. If they need help, I don't mind helping - in fact, I have a bit of design experience as well. However, it is not a good use of my time, so I don't do too much of the css.
31
Sssnake 3 days ago 0 replies      
Now perhaps in 10 years idiots in suits will finally stop demanding ridiculous pixel perfect control over website designs.
32
workhere-io 3 days ago 0 replies      
"X is dead" is dead. Just because you don't use X doesn't mean others don't use it.

A very large number of people who do web design for a living are much better at making their visions a reality using PhotoShop than HTML/CSS.

33
SkyMarshal 3 days ago 0 replies      
Well it should be dead, but like COBOL it'll be around a long time simply because there are tons of expert Photoshop designers who are much more productive with that tool than raw html/css and need their designs converted. I'm working with one right now, don't see it going away anytime soon.
34
joeblau 3 days ago 0 replies      
If you're looking for a fast way to extract images from your Photoshop file by layers/visibly/etc I highly recommend this software: http://getenigma64.com/

And if you're trying to extract gradients from Photoshop into CSS, SCSS, SASS: http://csshat.com/

35
jbeja 3 days ago 1 reply      
And would be glad if people stop making icons and UI sets in PSD, and start using a more portable format like Svg.
36
d0m 2 days ago 0 replies      
The concern is also about needing to hire two different people all the time. It slows down the workflow so much.. it's way easier to have one person in charge of it and being able to design, hack and tweak it as the projects evolve.
37
wil421 3 days ago 0 replies      
Tell that to people I work with, this is something I just did last week. I dislike doing it and I dont really agree with the camp that slices their page into images.
38
grimmdude 3 days ago 0 replies      
When I first read this article I didn't really agree with it, but after reading some of the comments on here I can understand where it's coming from.

I think the main issue is that the designer understand that it's more of a guideline on how the site should look. When they start getting nitty gritty about exact line breaks and page by page style changes is where it gets hairy and falls apart.

I don't think moving away from mockups is the answer if that's what the article is implying. Just a greater understanding of modern web abilities and standards is all that's needed from the designer.

39
supercanuck 3 days ago 0 replies      
So what is the replacement?
40
ctrl 3 days ago 0 replies      
+1 all these replies.As a designer first, I taught myself to code, Just as I taught myself how print medium works. These details are integral to the end product.

A web designer that doesn't understand code != a web designer.

I think Photoshop should be replaced with Illustrator. for the initial design phases.1) You can do wireframes in Illustrator, then build directly on top of that for design.2) Multiple artboards lets you layout multiple screen sizes/breakpoints.3) Resizing elements and keeping crisp edges is much faster.

41
j_s 2 days ago 0 replies      
Here is the direction designers should be heading:

http://www.sketchingwithcss.com/

42
leishulang 3 days ago 0 replies      
with tools like edge reflow, the human part of PSD to html is going to be dead. HTML/CSS/JS will become the assembly of the web that no one is directly programming with. Designers will keep working on psd and use edge-like tools to export html/js files, and coders will be using clojurescript/fay/coffeescript etc.
43
martianspy 2 days ago 0 replies      
I would like some constructive advice.I currently use sliced PSDs as part of page content workflow to get a one page flyer from InDesign onto the web.

I start with a one page PDF which was generated in InDesign for print. This one page flyer needs to be linked to products on an ecommerce site. The flyer changes weekly.

Currently I open the PDF in Photoshop, slice it, add the links and upload it into an iframe. It takes about 15-30 minutes to get if from PDF to live.

What would be a more efficient way for me to convert this PDF to clickable web content? I don't want to spend more time than I currently do on it.

44
mtangle 3 days ago 0 replies      
But picture is a good start to show what kind design you want And yes in many accessions some designers are tooooooo pitchy about their psd.
45
bluemnmtattoo 3 days ago 0 replies      
stone and chisel is dead
46
thomasfoster96 3 days ago 0 replies      
Hurrah!
19
Announcing The Matasano/Square CTF matasano.com
271 points by alepper  1 day ago   62 comments top 25
1
tptacek 1 day ago 4 replies      
Go easy on us for a bit; this is a fresh deploy in a new environment and it's bound to be janky. The odds of it not completely asploding tonight are pretty low.

Happy to answer questions.

One obvious question we haven't answered: how long will it be up for? Answer: a'unno. Until it gets boring? Or super expensive? We're in no rush to shut it down. I've never understood why awesome CTF events are so eager to shut down.

Here's what it looks like:

http://twitter.com/tqbf/status/423992147155509248/photo/1

We're on Freenode #uctf if you want to bug us live.

In case you're interested: this is a very small Rails app talking to an emulator we wrote in Golang that exports an HTTP/JSON interface.

2
haberman 21 hours ago 0 replies      
I've worked a lot in assembly, but never tried to exploit anything before. This is like crack.

I've always been meaning to try out a few public problem sets of this sort that a couple university security courses publish. Theirs are often based on a VM image with various binaries inside them that you are supposed to exploit. While those would probably have the advantage of being slightly more relevant to my everyday work (since they're running a Linux OS/environment more like the one I regularly use and program for), this contest makes it much easier to get over the hump of setting everything up, and of course the points/competition aspect is highly motivating.

Nicely done. :)

3
haberman 1 day ago 1 reply      
PSA for people like me who aren't security specialists: "shellcode" (in the survey) does not mean "a shell script", it means this: http://en.wikipedia.org/wiki/Shellcode
4
gibybo 21 hours ago 3 replies      
Can someone explain what "cmp.b@r13, 0x2400(r14)" does, and/or how I would find out myself?

I looked through the manual and saw the instruction 'cmp' tests the two parameters for equivalence, but I don't see 'cmp.b' anywhere. I found the TI manual for the instruction set which happens to say something about '.B' being a byte operation, but I'm not sure if that's related.

The tutorial explained that @r13 uses the value in memory for the address in register 13, but '0x2400(r14)' is really confusing.

I was thinking it just meant register 14, but that doesn't seem to be the case because the comparison fails when r14 (which is the value 0x0000 at this point) and @r13 match. I thought it also could mean the literal value '0x2400', but the comparison still seems to fail when it should match, and that doesn't explain the r14 in parenthesis at the end of it.

5
orthecreedence 20 hours ago 0 replies      
I used to jump at hacking challenges in high school, blowing off homework and in the summers staying up til 5am.

Nowadays I see these and I have to practically tie myself to the ship's mast to not drop everything I'm doing and sink days into it. I couldn't resist with Stripe's CTF but had to quit after I got busy. This one's especially hard because I love tinkering/building embedded devices.

I guess the point is please, please keep this running as long as you can so I can have a crack at it when I'm not working day and night =].

6
jwise0 1 day ago 2 replies      
During the tutorial, it seems like the memory dump goes blank for me (it loses its scroll bars, and loses its contents) -- http://i.imgur.com/ta9iykd.png

This is Firefox 25.0~b1+build1-0ubuntu0.12.04.1, on Ubuntu 12.04. I'll try it on something more modern when I get home.

Looking forward to it!

7
IgorPartola 22 hours ago 1 reply      
Dean Pelton: Agnes, cancel all my appointments.

Agnes: What appointments?

Dean Pelton: ...Wishful thinking.

Damn. There goes my weekend.

8
neur0mancer 1 day ago 1 reply      
The lock (fake) manual is available here:

https://microcorruption.com/manual.pdf

9
darklajid 4 hours ago 0 replies      
Encountered quite some bugs with FF (Aurora here), most prominently the highlight wasn't updated when I moved a step forward.

Suggestion on top of that: It would be really nice to grab the whole 'firmware' and dump it to a local .hex file. If that isn't allowed for obvious reasons/by design: Fair enough.

10
gibybo 20 hours ago 1 reply      
Possible bug in the score board:

In most cases, it won't let me see the scores for levels that I haven't beaten yet. Presumably this is because seeing the input size and min cpu cycles would be a strong hint about how to solve the level.

However, when viewing a particular user's profile (e.x. https://microcorruption.com/profile/294), it shows their completion stats for the level you are currently on, despite having not beaten it yet. The levels after it are still obscured, though.

11
strags 23 hours ago 2 replies      
FYI - On level 3, the "okay" button is obscured by the page footer, and can't be clicked on one of my machines. Scolling down doesn't help, the button remains obscured by the page footer which scrolls with the page.
12
nardi 21 hours ago 0 replies      
And then it turns out that this was a massive Mechanical Turk.
13
midas007 20 hours ago 0 replies      
14
richadams 23 hours ago 0 replies      
I encourage anyone who's avoiding taking part because they don't know assembly or reverse engineering to at least give the tutorial a try, you might just surprise yourself!

Having not really touched assembly that much before, I found the tutorial to be an excellent introduction. I'm now battling with level 4 after thinking I wouldn't even get past the first level.

Great work Matasano & Square!

15
Veraticus 20 hours ago 1 reply      
This is embarrassing, but... I'm totally stuck on puzzle #2 (Sydney) since the cmp doesn't seem to match up with what's in memory, and I bet that the puzzles won't get any easier from here! Is there a good resource, trove of documentation, or excellent book for those of us who would love to learn how to do this stuff? The tutorial great, but it was definitely pretty basic.
16
adsche 10 hours ago 0 replies      
This is awesome, I'm having a lot of fun reactivating my assembler knowledge.

Also, I want to compliment you on the interface, my laptop broke yesterday and I'm doing this on a borrowed Acer A500 tablet without any serious problems.

17
voltagex_ 1 day ago 0 replies      
This is great. I hope at least the UI is open-sourced at some point - it's really clear and it'd be good for other reversing tutorials.
18
busterarm 22 hours ago 0 replies      
I was all excited with myself for passing the first one after the tutorial and then couldn't get past the second...though I've been drinking.

This is very cool and I think would be "even more awesome" if there were a separate version that provided a tutorial for the skills required for each level.

For us 'tards. :P

19
dcwilson 22 hours ago 0 replies      
If anyone is worried about participating because they don't understand this domain very well, don't be. The tutorial is very useful, and the interface is generally very nice. Check it out.
20
quantumpotato_ 23 hours ago 1 reply      
I went through the tutorial. Seems very cool. Disappointed that "password" worked though - I thought it would show me how to read the value my input was compared to instead of just matching the string (and lucky guessing).
21
redshirtrob 17 hours ago 0 replies      
This is fun. For those looking for more details on the instructions and addressing modes, check out the MSP430 User's Guide:

http://www.ti.com/lit/ug/slau049f/slau049f.pdf

22
spydum 1 day ago 0 replies      
I put my credit card number in, and it didn't give me an account. Did I do it wrong? Why isn't there a padlock on the signup? Ohh nooo...
23
cpher 1 day ago 1 reply      
As someone completely inept in this niche, I'm looking forward to the results. I hope you share them. You'll probably have to dumb them down for us commoners.
24
banachtarski 1 day ago 0 replies      
Just did the first one. This is a lot of fun. Great work!
25
smoyer 1 day ago 0 replies      
This would be a great way to crowd-source the cracking of a digital lock ... I hope you're opening the warehouse that contains all the NSA's secrets (every HN story has to have comments that reference the NSA or they'll be thrown into the dead-pool).
20
Tell HN: Server Status
263 points by kogir  1 day ago   117 comments top 26
1
barrkel 1 day ago 2 replies      
By tolerating the loss of two disks, do you mean raidz2 or do you mean 3-way mirror?

Raidz2 is not fast. In fact, it is slow. Also, it is less reliable than a two way mirror in most configurations, because recovering from a disk loss requires reading the entirety of every other disk, whereas recovering from loss in a mirror requires reading the entirety of one disk. The multiplication of the probabilities don't work out particularly well as you scale up in disk count (even taking into account that raidz2 tolerates a disk failure mid-recovery). And mirroring is much faster, since it can distribute seeks across multiple disks, something raidz2 cannot do. Raidz2 essentially synchronizes the spindles on all disks.

Raidz2 is more or less suitable for archival-style storage where you can't afford the space loss from mirroring. For example, I have an 11 disk raidz2 array in my home NAS, spread across two separate PCIe x8 8-port 6Gbps SAS/SATA cards, and don't usually see read or write speeds for files[1] exceeding 200MB/sec. The drives individually are capable of over 100MB/sec - in a non-raidz2 setup, I'd be potentially seeing over 1GB/sec on reads of large contiguous files.

Personally I'm going to move to multiple 4-disk raid10 vdevs. I can afford the space loss, and the performance characteristics are much better.

[1] Scrub speeds are higher, but not really relevant to FS performance.

2
makmanalp 1 day ago 4 replies      
The trend I'm noticing is people mentioning that if only HN was moved to <insert-cloud-provider>, problems would go away.

Instead of doing that, they probably dropped a bit more than a thousand dollars on a box, and are probably saving thousands in costs per year. This is money coming out of someone's pocket.

This site is here, and it's a charity, being provided free of cost, to you. Who cares if HN is down for a few hours? Seriously? Has anyone been hurt because of this, yet?

3
whalesalad 1 day ago 2 replies      
There's a lot of tuning that can be done on a ZFS setup to improve performance. I'm not a pro, so others will have more feedback and knowledge, but some things off the top of my head to get you started:

Add a flash memory based (SSD) ZIL or L2ARC or both to the box. That'll help improve read/write performance. I believe the ZIL (ZFS intent log) is used to cache during writes, and the L2ARC is used during reads.

You might want to look into disabling atime, so that the pool isn't wasting energy keeping access times on files up to date. Not sure if this is relevant with the architecture of HN or not. This can be done with

    zfs set atime=off srv/ycombinator
Finally, ZFS needs a LOT of memory to be a happy camper. Like 3-5GB of RAM per TB of storage.

I actually think you'll probably have a lot of fun with ZFS tuning, if that's the problem with news.yc. FreeBSD's page is pretty detailed: https://wiki.freebsd.org/ZFSTuningGuide

4
cincinnatus 1 day ago 15 replies      
I'm sure it has been asked many times before, but I'd love to hear the latest thinking... Why in 2013 is HN still running on bespoke hardware and software? If a startup came to you with this sort of legacy thinking you'd laugh them out of the room.
5
hartator 1 day ago 0 replies      
Not really related but any update on releasing the HN code again?

[the current release is pretty old: https://github.com/wting/hackernews]

6
JayNeely 1 day ago 0 replies      
Being the sysadmin on a site frequented by sysadmins has to be frustrating at times.

Thanks for all you do!

7
conorh 1 day ago 0 replies      
Have you thought about perhaps open sourcing the server setup scripts for HN? I'd love (and I'm sure many others here) to help with the configuration. Perhaps a github repo for some chef recipies that people could work on given the current servers?
8
lsc 1 day ago 0 replies      
Are you bottlenecking on high iowait? or something else?

just one random bit to try... Obviously, I have no insight into your system and I'm not saying I know more than you or anything, but I've been seeing more situations lately where I had massive latency but reasonable throughput and the disks mostly looked okay wrt. smart, and I mostly just wanted to write about it:

[lsc@mcgrigor ~]$ sudo iostat -x /dev/sda /dev/sdb /dev/sdc /dev/sddLinux 2.6.18-371.3.1.el5xen (mcgrigor.prgmr.com) 01/16/2014

avg-cpu: %user %nice %system %iowait %steal %idle

           0.00    0.00    0.05    0.02    0.00   99.93
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util

sda 0.70 75.11 35.66 1.38 4568.62 611.67 139.85 0.36 9.61 0.53 1.95

sdb 0.46 75.10 35.62 1.39 4566.77 611.67 139.89 0.22 5.89 0.45 1.66

sdc 0.80 75.14 35.63 1.35 4569.63 611.63 140.10 0.64 17.18 0.57 2.10

sdd 0.46 75.09 35.62 1.40 4566.60 611.63 139.87 0.13 3.47 0.40 1.49

(this is a new server built out of older disks that appears to have the problem. It's not so bad that I get significant iowait when idle, but if you try to do anything, you are in a world of hurt.)

Check out the await value. re-do the same command with a '1' after /dev/sdd and it will repeat every second. If sdd consistently has a much worse await, it is what is killing your RAID. Drop the drive from the raid. If performance is better, replace the drive. If performance is worse (and with raid z2, it should be worse if you killed the drive) the drive was fine.

(Of course you want to do the usual check with smart and the like before this)

The interesting part of this failure mode that I have seen is that /throughput/ isn't that much worse than healthy. You get reasonable speeds on your dd tests. but latency makes the whole thing unusable.

9
nmc 1 day ago 1 reply      
Thanks for the info!

Out of curiosity, do you have an idea about the source of the corruption problems?

10
ishener 1 day ago 6 replies      
may i ask where are the machines hosted? is that on AWS? if not, why don't you move to a more reliable hosting, like AWS?
11
Goladus 1 day ago 1 reply      
I've been reading this site regularly for almost 7 years. 6-Jan-2014 is the only downtime I remember, and it was really a very minor inconvenience. Sucks about the data loss though, always hard to own that when doing system administration. Thanks for the explanation.
12
erkkie 1 day ago 1 reply      
This reminds me I'm still looking for a (pki?-)encrypted zfs snapshots as a backup service, /wink-wink @anyone

Hoping the box has ECC ram, otherwise zfs, too, can be unreliable (http://research.cs.wisc.edu/adsl/Publications/zfs-corruption...)

13
shawn-butler 1 day ago 0 replies      
Using DTrace to profile zfs:

http://dtrace.org/blogs/brendan/files/2011/02/DTrace_Chapter...

I'm sure other more experienced DTrace users can offer tips but I remember reading this book and learning a lot. And I believe all the referenced scripts were open source and available.

14
richardw 1 day ago 0 replies      
Thanks for the update. No worries, it's just a news message board and no businesses are hurt when it's down. I quite enjoy seeing how these things are solved and I'm sure all will be forgiven if you post a meaty post-mortem.
15
rrpadhy 1 day ago 0 replies      
I am curious to know the server configuration, architecture and the number of hits it is getting.

If someone does offer a new software architecture, and hosting, would people be open to move hackernews there?

16
avifreedman 1 day ago 0 replies      
Assuming the disk footprint is small...

Would recommend a new SSD-based ZFS box (Samsung 840 Pros have been great even for pretty write-intesive load), with raidz3 for protection and zfs send (and/or rsync from hourly/N-minute snapshot for data protection which should eliminate copying FS metadata corruption, as not sure if zfs send will).

Happy to provide and/or host such a box or two if helpful.

17
rdl 1 day ago 0 replies      
I still like hardware RAID because it's conceptually simple and nicely isolated. Sometimes horrible things happen to it, though, too.

I didn't realize HN had enough disk storage needs to need more than one drive. I guess you could have 1+2 redundancy or something.

18
jffry 1 day ago 0 replies      
Thanks for the writeup.
19
rincebrain 1 day ago 1 reply      
ZFS instead of UFS on what, an Illumos derivative, FBSD, or actual Oracle Solaris?
20
scurvy 1 day ago 2 replies      
Why on earth are you not using SSD's? The HN footprint can't be that large. The extra speed and reliability from a pair of SSD's has to far outweigh the costs.
21
lukasm 1 day ago 1 reply      
How about error page show the last static HN page? Most people just need likns
22
carsonreinke 1 day ago 0 replies      
Maybe you could provide details on the current configuration and architecture and some suggestions could be made on how to improve. Just a thought.
23
0xdeadbeefbabe 1 day ago 0 replies      
Don't worry about it. I visited facebook for the first time in years when hn went down. Is hn on linux using zfs or bsd?
24
smalu 1 day ago 2 replies      
The world would be better place if software could exist without hardware.
25
superice 1 day ago 1 reply      
Good you posted this, but it came a little late. After the first series of timeouts you could've posted an update so everybody knew what was going on. But hey, thanks for the update, this clears up a lot.
26
waxzce 1 day ago 1 reply      
Hi, I'm the CEO of http://www.clever-cloud.com/ and I'll be happy to help you on this, ping me on twitter : @waxzce
21
VC Pitches in a Year or Two avc.com
262 points by SethMurphy  2 days ago   96 comments top 32
1
r0h1n 2 days ago 3 replies      
This is already reality in some countries. Like India for instance.

1st Indian entrepreneur: I plan to launch a search engine that understands Indian languages and contexts better than Google.

VC: Well since Google has already paid telcos like Airtel (http://www.airtel.in/free-zone/) so their searches and even some results don't use up any of the data plan, we are passing.

2nd Indian entrepreneur: I have an idea for a social network that is better than Facebook.

VC: Sorry, Well since Facebook has already paid telcos like Airtel (http://www.medianama.com/2014/01/223-airtel-facebook-free-hi...) so their site/app doesn't consume data while being used, we are passing.

2
legutierr 2 days ago 3 replies      
I think the pertinent question now is whether the FCC rewrites its rules to classify ISPs as common carriers. It seems to me, given the local monopoly or duopoly that the vast majority of ISPs enjoy, that this is an obvious move. But I have heard it barely discussed, which is distressing.
3
OoTheNigerian 2 days ago 0 replies      
The issue of Net neutrality is a global phenomenon/risk. Telcos in Nigeria abuse their positions as the primary carrier of data and they are protected by having 'licenses'

MTN and Rocket Internet recently tied a deal. I wrote about the risk here

http://oonwoye.com/2013/12/17/mtn-rocket-internet-deal-worri...

4
antr 2 days ago 1 reply      
I'd like to believe that many European entrepreneurs will reconsider going to the US to start a company. The European Parliament and the Commission have been straight shooters with net neutrality and they will not consent telcos to play with the pipes.
5
zxcvvcxz 2 days ago 5 replies      
Not the point of the article, but is anyone else annoyed by these "entrepreneur-sounding" ideas? I'm so sick of low-tech, solve-first-world-problem ideas and conflating that with entrepreneurship. If I were the VC, I'd tell them to get the fuck out, and it'd have nothing to do with Telcos.
6
TTPrograms 2 days ago 3 replies      
The author is missing the point that the ruling was about the specific language of the FCC regulations. See: http://gigaom.com/2014/01/14/breaking-court-strikes-down-fcc...

"That said, even though the Commission has general authority to regulate in this arena, it may not impose requirements that contravene express statutory mandates. Given that the Commission has chosen to classify broadband providers in a manner that exempts them from treatment as common carriers, the Communications Act expressly prohibits the Commission from nonetheless regulating them as such. Because the Commission has failed to establish that the anti-discrimination and anti-blocking rules do not impose per se common carrier obligations, we vacate those portions of the Open Internet Order."

It seems very likely that the FCC will rewrite their regulations to fix this. Everyone knows that net neutrality is important, and this ruling is just an issue in legalese. It's a little early to resort to torch-and-pitchfork hyperbole.

7
mwsherman 2 days ago 0 replies      
Heres the problem: what AT&T is doing does not violate any technical definition of net neutrality, unless we ad hoc append new parts to it.

This is the core problem of net neutrality arguments, which it is often defined as I know it when I see it. It amounts to principles, but if we are going to have an enforceable law, we need to do better than that.

AT&T is not offering any priority to any bits here. Nothing is being blocked or degraded. Content providers who pay for sponsored data do not get faster bits nor do they slow down anyone elses.

Its free shipping: http://clipperhouse.com/2008/06/03/the-long-game-on-metered-...

Now, I can understand objecting to it on its merits, and Fred is making that argument, which is great. And I can understand why it feels like a violation of net neutrality, but we need to do better than feelings.

Heres how we test whether were defining net neutrality ad hoc: show me a clear, specific, widely accepted definition of net neutrality that describes AT&Ts behavior here, and that existed before this behavior was publicized.

8
n00b101 2 days ago 0 replies      
Entrepreneur: I plan to launch a better [XYZ] hosted on Amazon AWS.

VC: Well since Amazon has paid all the telcos so that services delivered through AWS "telco-optimized elastic IPs" can be free on data plans, all you have to do is include Amazon's surcharges in your business plan.

9
anovikov 2 days ago 0 replies      
Scary thing, this is same thing that created Rockfeller-era monopolies: they had right and did pay railways to create preferences (price-wise and otherwise) for their traffic, literally derailing competition. Internet is the bloodline of the post-industrial economy just like railways were of industrial one. That is much worse than most of you think.
10
avighnay 2 days ago 0 replies      
The blog places its argument on the basis that data plan cost is going to be high. Would this be the case in the future? What if data plan costs are that negligible that it does not matter?

Secondly, compare this to TV networks, consumers watch TV and pay for them too, a part of the cost is subsidized by advertisers who are willing to pay the network to reach the audience. The consumers are in that network only because of the content, remove the content providers or reduce the quality of the content then the consumer vanishes. An empty network is worth nothing.

Would telcos, not harm themselves and their whole data plan business by attempting to charge the content providers (Google et al) and would the content providers 'advertising model' margins justify paying out to the telcos just to get through their infrastructure?

11
napoleond 2 days ago 0 replies      
So basically it will become even harder for startups to disrupt the established players. Until a disruptive telco comes along and everyone switches to them, while also learning an important lesson about net neutrality. Maybe the whole thing will even cause society to re-think the way we allocate access to the internet and cellular networks.

(A man can dream!)

12
rexreed 2 days ago 0 replies      
While I hate the loss of Net Neutrality as much as the next person (as long as that next person isn't one of the incumbent carriers), this is the sort of thing that happens frequently in other markets. You want to see entrenched competition? Try launching a company in the Cleantech markets or in certain hardware or biotech or healthcare fields. Big pocket incumbents regularly flex their muscles here.

The loss of net neutrality is bad from many perspectives, but to be honest, there will ALWAYS be opportunities for startups and entrepreneurs in the space and VCs will not be want for good ones. All this does is shake out many ideas in favor of other ones.

I don't see why the VCs have reason to panic. And while I understand the Entrepreneur ideas were straw men utilized to illustrate a point, the quality of these ideas are pretty low. Maybe we should see the silver lining on this dark cloud in that it will shake out some of these deals from being funded when they probably shouldn't be anyways.

13
ankitml 2 days ago 1 reply      
Facebook already does this in india. I remember seeing an advertisement of a telco that said you wont be charged for data for facebook. Does this means that India already had this non neutral internet? It didn't change the scenario much here.
14
badman_ting 2 days ago 0 replies      
Yes, current giants are going to continue consolidating and buying up businesses and amassing power. In America we used to have ways of dealing with monopolies but that is now pass -- we don't like the government "punishing" successful businesses (in the same way that taxes "punish" the rich). A lot of times I feel like something very bad will have to happen for things to change. Until then, hey.
15
dzink 1 day ago 0 replies      
I am watching the reactions of non-technical people after they hear how this latest Net Non-Neutrality development affects them. Some are saying "Google, Apple, and the other big guys will step in for us". Others reply that when gas prices jumped to $3 everyone cried out and legislators started shuffling, yet we are now paying $4 and being thankful. Unless the outcry gets bigger this is going to pass through the cracks.

Do any of the big tech companies really have an interest in stopping this development? They could afford to buy themselves an "unrestricted by cap" deal with internet distributors and suffocate every other potential competitor?

16
mathattack 2 days ago 0 replies      
I get that this is an issue, but I'm a little curious why AT&T's stock price didn't pop as a result.

http://finance.yahoo.com/echarts?s=T+Interactive#symbol=t;ra...

17
crgt 2 days ago 0 replies      
A friend of mine was part of the team that made the decision for UHC to participate in AT&T's new sponsored data program. When I found this out, I tried to point out the broader implications of a shift towards pay-to-play, but had trouble getting past the narrow vision of "but we just want to get our content out to people that can't pay for mobile bandwidth". Just wanted to say thanks for this article for demonstrating what some of the implications are of a shift to a pay-to-play model. Non-techies really seem to struggle to get their heads around what happens to the entire ecosystem if net neutrality dies, and articles like this are helpful for making the consequences clearer.
18
josefresco 2 days ago 0 replies      
Forget net neutrality, doesn't this reality already exist when you factor in data caps? Most of these startup ideas use a fair amount of data which is now being capped more often than it has in the past.

I may be alone in my worry that as we move to capped data plans, and the pay-per-bit model that many new ideas and concepts (say for example always connected appliances) won't be financially feasible for consumers.

19
exelius 2 days ago 1 reply      
Most VCs would pass on these ideas regardless. They're all in mature, saturated markets. One of the signs of a mature, saturated market is that the incumbents have taken great lengths to create barriers to entry that are very high. This is a discouraging thing for VC investment because it increases the chances that a new entrant will fail.

Besides, you've had to pay for access for years. The big difference is that before, you had to go through a CDN like Level3 or Akamai. Part of what you paid them went to the ISPs to ensure fast connections. All this means is that the YouTubes of the world will begin to buy interconnects with the big ISPs. Small ISPs will likely just band together into a cartel and sell access that way.

Yeah, there will be a fast lane and a slow lane, but the advent of CDNs in the early 2000s already created that anyway. The data caps are disappointing, but not really unexpected if you look at the mobile market. We're reaching the point where in many major cities, there is no media consumption that requires a much faster connection than is already available. So why will consumers pay for more speed when their existing 50mbps cable modem is enough to stream 4k video from Netflix? Those speeds ARE possible today if you buy carriage through a CDN like Akamai (and I regularly get those speeds from Steam downloads) and the fact that Netflix hasn't is really more of an implication of their business model.

The most recent ruling really changes nothing, because net neutrality has been dead for 10 years anyway. While everyone on the internet was complaining about it, the business side moved on and built a few billion dollar companies around it. Capitalism at its finest.

20
delinka 2 days ago 2 replies      
So someone should plan to launch a VPN service that will pay telcos so its traffic doesn't use any of its customers' data plans. Configure the mobile device to route all data via the VPN, encrypted. It'll charge its customers for access to unlimited apps, sites and streams without incurring a data plan hit.
21
SethMurphy 2 days ago 2 replies      
While I agree with his premise, there are already industries that used to be startup focused where there is a gatekeeper. E-commerce has Amazon, advertising has Google (and possibly Facebook), which already destroy new startups before they really get started through this same VC mindset. It seems to me that the balance of power is just moving up the line a bit to industries where VC's have little to no power to influence.
22
andrewescott 2 days ago 0 replies      
There seems to be an assumption in the original argument that not every company will be able to get access to sponsored data, e.g.

> Well since Spotify, Beats, and Apple have paid all

> the telcos so that their services are free on the mobile

> networks, we are concerned that new music services like

> yours will have a hard time getting new users to use them

> because the data plan is so expensive

If a new music service could make a deal with telcos so that their service is free too, wouldn't this problem go away? In other words, if sponsored data was open to all, does this address the concern described?

The real concern seems to be that the cost base of a new service will go up because it will be forced to pay for sponsored data in order to compete, and VCs aren't happy about having to cover increased costs of their portfolio companies.

A similar argument could have been made about CDNs. Because the big services use CDNs to provide a better service, startups have to pay to use CDNs also in order to compete, and hence their costs are higher.

23
thejosh 2 days ago 1 reply      
Australia has been doing this for years for mobile data plans. Even though data plans aren't as expensive as they use to be, Facebook, Twitter, Foursquare, eBay, LinkedIn & MySpace are all "free" on the Optus network.
24
prolifically 2 days ago 0 replies      
I hear him, Internet could become a caste system for businesses. But from what I understood, the ruling was more about the FCC trying to bend rules to accomodate everyone with carrier types (common carrier?). Yesterday's ruling raised awareness but the game is far from over.
25
BrownBuffalo 2 days ago 0 replies      
What's interesting are the comments below the post about Net Noot. The problem really comes less about the fact that traffic shaping may not occur, but there are no safe guards if someone attempts to do so. In smaller markets, its more and more sounding like a small cottage industry will start because of the lack of existing laws to protect the consumer. The larger markets will have power in numbers, but mom-pop towns like Marion, AL with only a small regionarl carrier - not so much. Problematic and there is SOME truth to this in scale of economy.
26
dredmorbius 2 days ago 1 reply      
This is the best possible thing that could happen.

Perhaps we'd see startups aimed at building out solar capacity, grid-scale storage solutions, electricity-to-fuels solutions, solar-powered airships, high-efficiency wind-steam hybrid shipping, high-efficiency retrofits to existing building stock, and management or treatment for TDR-TB, rather than an endless stream of privacy-invading "social" surveilices, games, and new forms of intrusive and annoying advertising.

Though building out an alternet that bypasses the telco's "authorized" channels wouldn't hurt either. Mesh and darknets.

Get cracking, HN!

27
codingdave 2 days ago 1 reply      
I get the point you are trying to make, but it wasn't presented in a way that non-tech folk will appreciate.

My parents' reaction to your scenarios would be something akin to: "Really? For the monthly rate I'm already paying, I now get Netflix and Hulu included for free? NICE!"

28
neovive 2 days ago 0 replies      
I guess this makes open city WiFi networks much more compelling. In most urban areas, how often are people not within range of a WiFi network?
29
tarr11 2 days ago 0 replies      
I guess we need a chrome extension to uuencode your next social network over FB status updates.
30
orenbarzilai 2 days ago 0 replies      
imho in the near future most countries will have unlimited data plans or similar, so this argument will be irrelevant.
31
excitom 2 days ago 0 replies      
Well, I'm no fan of losing net neutrality but if the worst thing that happens is that lame VC pitches are no longer funded, I'm OK with it.
32
hackaflocka 2 days ago 2 replies      
Bandwidth isn't free. The universal all-you-can-eat model is very unfair to the bandwidth supplier.

Either customers will need to be charged by meter.

Or producers will need to pay by meter.

It's only capitalism.

By the way, why can't the VC say, "we love your idea, and we'll front the money you need to pay the telcos."?

22
UTF-8 Everywhere utf8everywhere.org
258 points by angersock  1 day ago   142 comments top 23
1
jbk 1 day ago 2 replies      
This resonate so much for me, in VLC.

VLC has a very large number of users on Windows (80% of our users), yet almost none of the dev use Windows to code. Therefore, we use UTF-8 char* everywhere, notably in the core. We use UTF-16 conversions only in the necessary Windows modules, that use Windows APIs. Being sure we were UTF-8 everywhere took a lot of time, tbh...

But the worse are formats like ASF (WMV) or MMS that use UTF-16/UCS-2 (without correctly specifying) them and that we need to support on all other platforms, like OSX or Linux...

2
huhtenberg 1 day ago 3 replies      
Now, all the advice in the Windows section - don't do this, don't do that, only and always do third - is lovely, but if you happen to care about app's performance, you will have to carry wstrings around.

Take a simple example of an app that generates a bunch of logs that need to be displayed to the user. If you are to follow article's recommendations, you'd have these logs generated and stored in UTF8. Then, only when they are about to be displayed on the screen you'd convert them to UTF16. Now, say, you have a custom control that renders log entries. Furthermore, let's imagine a user who sits there and hits PgUp, PgDown, PgUp, PgDown repeatedly.

On every keypress the app will run a bunch of strings through MultiByteToWideChar() to do the conversion (and whatever else fluff that comes with any boost/stl wrappers), feed the result to DrawText() and then discard wstrings, triggering a bunch of heap operation along the way. And you'd better hope latter doesn't cause heap wobble across a defrag threshold.

Is your code as sublime as it gets? Check. Does it look like it's written by over-enlightened purists? You bet. Just look at this "advice" from the page -

  ::SetWindowTextW(widen("string litteral").c_str())
This marvel passes a constant string to widen() to get another constant string to pass to an API call. Just because the code is more kosher without that goddamn awful L prefix. Extra CPU cycles? Bah. A couple of KB added to the .exe due to inlining? Who cares. But would you just look at how zen the code is.

  --
tl;dr - keeping as much text as possible in UTF8 in a Windows app is a good idea, but just make sure not to take it to the extremes.

3
asgard1024 1 day ago 6 replies      
This may be tangential, but I think that computer languages should have a different type (and literal notation) for human text (strings that may be read by human, may be translated, won't affect program semantics) and for computer string (strings that are strictly defined, not to be translated, and may affect program semantics).

Then we could put all the human language problems into human text type, and leave the simpler computer string type with easier semantics.

In Python, although there are no tools for that, I typically use the following convention: single quotes for computer text and double quotes for human text. I guess you could use byte arrays for computer text as well, but it would be more painful.

4
Pxtl 1 day ago 1 reply      
I was horrified to discover that Microsoft SQL Server's text import/export tools don't even support UTF-8. Like, at all. You can either use their bastardized wrongendian pseudo-UTF-16, or just pick a code-page and go pure 8-bit.
5
vorg 1 day ago 1 reply      
I don't like the way UTF-8 was clipped to only 1 million codepoints in 2003 to match the UTF-16 limit. The original 2.1 billion codepoint capacity of the original 1993 UTF-8 proposal would've been far better. Go Lang uses \Uffffffff as syntax to represent runes, giving the same upper limit as the original UTF-8 proposal, so I wonder if it supports, or one day will support, the extended 5- and 6-byte sequences.

In fact, UTF-16 doesn't really have the 1 million character limit: by using the two private-use planes (F and 10) as 2nd-tier surrogates, we can encode all 4-byte sequences of UCS-32, and all those in the original UTF-8 proposal.

I suspect the reason is more political than technical. unicode.org (http://www.unicode.org/faq/utf_bom.html#utf16-6) says "Both Unicode and ISO 10646 have policies in place that formally limit future code assignment to the integer range that can be expressed with current UTF-16 (0 to 1,114,111). Even if other encoding forms (i.e. other UTFs) can represent larger intergers, these policies mean that all encoding forms will always represent the same set of characters. Over a million possible codes is far more than enough for the goal of Unicode of encoding characters, not glyphs. Unicode is not designed to encode arbitrary data."

6
jasonjei 1 day ago 0 replies      
We constantly have to deal with Win32 as a build platform and we write our apps natively for that platform using wchar. I think the main difficulty is that most developers hate adding another library to their stack, and to make matters worse, displaying this text in Windows GUI would require conversion to wchar. That's why I think they are up for a lot of resistance, at least in the Windows world. If the Windows APIs were friendlier to UTF-8, there might be hope. But as it stands right now, using UTF-8 requires the CA2W/CW2A macros, which is just a lot of dancing to keep your strings in UTF-8 which ultimately must be rendered in wchar/UTF-16.

Maybe there might be a shot in getting developers to switch if Windows GUIs/native API would render Unicode text presented in UTF-8. But right now, it's back to encoding/decoding.

7
randomfool 1 day ago 1 reply      
"This is what made UTF-8 the favorite choice in the Web world, where English HTML/XML tags are intermixed with any-language text."

Except that Javascript is UTF-16, so no luck with 4 byte chars there.

8
belluchan 1 day ago 3 replies      
And software developers, don't forget to implement the 4 byte characters too please. Utter nightmare dealing with MySQL. I believe 4 byte characters still even break github comments.
9
GnarfGnarf 8 hours ago 0 replies      
Interesting. I came to the same conclusion myself a few years ago when converting a Windows app to Unicode. I store all strings as UTF-8, which enabled me to continue using strncpy, char[] etc. I convert to wchar_t only when I need to pass the string to Win32. I can even change from narrow to widechar dynamically. I use a global switch which tells me whether I am running in Unicode or not, and call the 'A' or 'W' version of the Win32 function, after converting to wchar_t if necessary.
10
nabla9 1 day ago 1 reply      
UTF-8 is usually good enough in disk.

I would like to have at least two options in memory: utf-8 and vector of displayed characters (there's many combinations in use in existing modern languages with no single-character representations in UTF-<anything>).

11
elwell 1 day ago 2 replies      
I can only imagine what kind of frustration drove someone to make this site.
12
chj 23 hours ago 0 replies      
Windows is a horrible environment for UTF8 unless MS provides a special locale for it.

At present state, you can choose to use utf8 internally in your app, but when you need to cooperate with other programs (over sockets or files), it's going to be confusing. Some will be sending you ANSI bytes and you take it as UTF8.

13
optimiz3 1 day ago 2 replies      
Most of the post talks about how Windows made a poor design decision in choosing 16bit characters.

No debate there.

However, advocating "just make windows use UTF8" ignores the monumental engineering challenge and legacy back-compat issues.

In Windows most APIs have FunctionA and FunctionW versions, with FunctionA meaning legacy ASCII/ANSI and FunctionW meaning Unicode. You couldn't really fix this without adding a 3rd version that was truly UTF-8 without breaking lots of apps in subtle ways.

Likely it would also only be available to Windows 9 compatible apps if such a feature shipped.

No dev wanting to make money is going to ship software that only targets Windows 9, so the entire ask is tough to sell.

Still no debate on the theoretical merits of UTF-8 though.

14
BadassFractal 1 day ago 2 replies      
Would be lovely if MS Office could export CSV to UTF-8, but nope.
15
wehadfun 1 day ago 2 replies      
I admire and appreciate your concern for something that is missunderstood and ignored. However this webpage took way to long to say what is so great about utf 8.
16
mattfenwick 1 day ago 0 replies      
I think it's unfortunate that it doesn't have more concrete examples. I think having more of those would really help strengthen their case, clarify their points, and make their arguments tangible and understandable to a much wider audience.

One instance where I really wish for examples: they mention characters, code points, code units, grapheme clusters, user-perceived characters, fonts, encoding schemes, multi-byte patterns, BE vs LE, BOM, .... while I kind of get some of these, I certainly don't understand all of them in detail, and so there's no way that I'll grasp the subtleties of their complicated interactions. Examples, even of simple things such as what actually gets saved to disk when I write out a string using UTF-8 encoding vs. UTF-16 -- especially when using higher codepoints, would be hugely beneficial for me.

17
andystanton 1 day ago 1 reply      
"Even though one can argue that source codes of programs, web pages and XML files, OS file names and other computer-to-computer text interfaces should never have existed, as long as they do exist, text is not only for human readers."

I'm a little confused by this statement. Can someone clarify?

18
Dewie 1 day ago 2 replies      
IT is so Anglophile that programs can become slower if you deviate from ASCII...

But of course being so incredibly anglocentric is not an issue, at least that seems to be the consensus of the participants when I read discussions on the Web where all the people who are discussing it write English with such a proficiency that I can't tell who are and aren't native speakers of the language.

19
puppetmaster3 1 day ago 0 replies      
In Javascript, it's UTF16.Also Java.

Can't speak for other of the top.

20
jahewson 1 day ago 1 reply      
> there is a silent agreement that UTF-8 is the most correct encoding for Unicode on the planet Earth

But what about other planets? Is there a Unicode Astral Plane which may encode poorly in the future?

21
angersock 1 day ago 2 replies      
What is currently the best way of dealing with UTF-8 strings in a cross-platform manner? It sounds like widechars and std::string just won't cut it.
22
duaneb 1 day ago 1 reply      
Thank god for emoji.
23
josteink 1 day ago 9 replies      
Looking at the .NET parts of the manifesto, I just have to roll my eyes:

Both C# and Java offer a 16 bit char type, which is less than a Unicode character, congratulations. The .NET indexer str[i] works in units of the internal representation, hence a leaky abstraction once again. Substring methods will happily return an invalid string, cutting a non-BMP character in parts.

While theoretically true, for most practical purposes, this reeks of a USA/American/English bias and lack of real world experience.

You know what? I want to know that the text "" is three characters long. I dont want to know that it's a 6-byte array once encoded to UTF8. Anywhere in my code telling me this is 6 characters is lying, not to mention a violation of numerous business-requirements.

When I work with text I want to work with text and never the byte-stream it will eventually be encoded to. I want to work on top of an abstraction which lets me treat text as text.

Yes, their are cases where the abstraction will leak. But those cases are very far and few in between. And in all cases where it doesn't, it offers me numerous advantages over the PHPesque, amateurish and incorrect approach of treating everything as a dumb byte-array.

It's not. It's text in my program. It's text rendered on your screen. It's just a byte-array when we send it over the wire, so stop trying to pretend text isn't text.

This manifesto is wildly misguided.

23
NSA collects millions of text messages daily in 'untargeted' global sweep theguardian.com
251 points by weu  1 day ago   69 comments top 21
1
moxie 1 day ago 2 replies      
I help develop TextSecure, an Android app which allows users encrypt their text messages:https://play.google.com/store/apps/details?id=org.thoughtcri...

It's fully open source:https://github.com/whispersystems/textsecure

You can also sign up to be notified when it's released for iOS:https://whispersystems.org/blog/iphone-rsn/

2
pvnick 1 day ago 5 replies      
"Communications from US phone numbers, the documents suggest, were removed (or minimized) from the database but those of other countries, including the UK, were retained."

I'm interested in knowing the specifics on this. US data goes into a database and is then proactively removed? Minimization procedures [1] allow the nsa to keep US data up to 5 years to determine where it's coming from. It's also kept if "they contain usable intelligence, information on criminal activity, threat of harm to people or property, are encrypted, or are believed to contain any information relevant to cybersecurity."

So until the documents show otherwise, I assume that most of my text messages from the past 5 years are in an nsa database, and all messages that my friends sent to buy weed are there, perhaps being used to parallel construct criminal cases [2].

And at the very least, all of it would be visible to the systems administrators...

[1] http://www.theguardian.com/world/2013/jun/20/fisa-court-nsa-...

[2] https://www.eff.org/deeplinks/2013/08/dea-and-nsa-team-intel...

3
jrochkind1 1 day ago 1 reply      
It's become like science fiction, or a bad joke.

Whatever you can imagine the most powerful intelligence agency imaginable doing, the NSA is doing. I am literally unable to think of anything that the NSA is _not_ collecting or controlling. Every SMS message sent? Yep. Location data on every cell phone? Yep. Controlling a 100k-computer botnet, including via radio transmitter to contact computers not on the internet? Yep.

I admit I can't keep track of it all; at this point I just assume that anything I can imagine the NSA doing, it's either already been revealed that they're doing it and I just haven't managed to keep track, or they're doing it even though it hasn't been revealed yet.

4
sinak 1 day ago 1 reply      
This is very clearly collection of the contents of messages, and not simply metadata. The fact that US messages may later be "minimized" from the database is vaguely helpful, but not much of a reassurance. That data should never be collected in the first place, and bulk collection of international data is also an unnecessary practice.

If this helps motivate you, we need all the help we can get with https://thedaywefightback.org. If you're a designer, developer (including frontend, backend, devops, mobile app), get in touch by emailing contact@thedaywefightback.org and we'll write back and let you know when we need your skillset the most.

5
VexXtreme 1 day ago 1 reply      
I still don't understand something. If they are intercepting text messages globally, how do they do that if two people are texting each other in within the borders of their own country using their national carrier? Their texts never leave the country. Has the NSA somehow compromised or coerced various carriers around the world to provide them with this data?
6
cryoshon 1 day ago 5 replies      
Great, let's start doing something about it instead of complaining. Here's my cynical take on a back of the napkin recipe for political change:

1. Call, write, tweet, and facebook your representatives, both local and federal, and tell them what you think- be sure to inflate your credentials and threaten to give money/votes/accolades to their political enemies. For bonus points, tell their enemies the same thing. For even more bonus points, run as a candidate yourself.

2. Write/speak/act out in the public sphere, make sure to get as much attention as possible and to be moderately vitriolic and abundantly populist in your rhetoric. Use only words, concepts, and rhetoric that a dim 9th grader would understand.

3. Join a physical protest- if there isn't one near you, it's your job to start one. I'd keep it nonviolent if I were you. I know that this isn't the cup of tea for most HN readers, but there's no way around it: physical presence matters, and the numbers of people who have protested surveillance thus far have been extremely paltry. We don't have the luxury of waiting for someone else to do it for us.

4. Convince your less-enlightened friends and relatives to do items 1-5, or at least be terrified of the government.

5. Start again from 1; repeat until successful.

7
fnordfnordfnord 1 day ago 0 replies      
> "The note warns analysts they must be careful to make sure they use the forms toggle before searching, as otherwise the database will return the content of the UK messages which would, without a warrant, cause the analyst to unlawfully be seeing the content of the SMS."
8
vermontdevil 1 day ago 3 replies      
By now there's pretty much nothing NSA is not doing to collect information from everyone.

Now I wonder if NSA is able to penetrate services like WhatsApp, Snapchat, etc.

9
Tycho 1 day ago 0 replies      
I liked the NSA better when they were just called the Illuminati.
10
angersock 1 day ago 1 reply      
From the slide, a subtitle:

SMS Text Messages: A Goldmine to Exploit

Oy vey--they don't even care anymore, do they? They're not even trying.

Fuck it, have a friendly octopus:

http://static3.businessinsider.com/image/52a3bb30eab8ea8a2d3...

11
vavoida 1 day ago 1 reply      
metacontent (message content) & metadata -> smiley face, slide 2

interesting definition of metacontent

12
WalterBright 1 day ago 2 replies      
Notice in the Arod scandal that all his text messages have been made public? Was there a warrant for that?
13
peterkelly 15 hours ago 0 replies      
"Content derived metadata"

Nice.

14
elwell 1 day ago 3 replies      
Use iMessage instead if you care. I for one welcome our new overlords.
15
easy_rider 1 day ago 0 replies      
NSA sure love their double forward slash and smiley faces
16
exo762 1 day ago 0 replies      
While NSA is busy being menace, you can mitigate damage by using TextSecure.

https://en.wikipedia.org/wiki/The_Guardian_Project_%28softwa...

17
gathly 1 day ago 0 replies      
this is shocking news back in July.
18
dangayle 1 day ago 0 replies      
No mas
19
jonhmchan 1 day ago 0 replies      
Good god.
20
fit2rule 1 day ago 1 reply      
Time to turn the tools on these traitors.
21
zacinbusiness 1 day ago 0 replies      
Who is making these piss poor presentations? They're hideous and terrible. I could look past the gross invasion of privacy if they would at least spend 10 minutes building some decent designs. Get it together, NSA!
24
Obamas Path From Critic to Overseer of Spying nytimes.com
237 points by mjstahl  1 day ago   183 comments top 29
1
grandalf 1 day ago 13 replies      
The headline might as well be a more Onionesque "After gaining power, politician turns out not to actually hold the strongly principled views he expressed while campaigning".

I'd be surprised if Obama holds any of the views he expressed during his campaign. A campaign is a marketing effort intended to install a team of people in power.

Generally speaking, the vast majority of power holders agree that aggressive spying is a good idea. This is closely related to their strong preference for maintaining the status quo across the board. We should not be surprised that Obama did not reverse any of Bush's controversial decisions because they were not actually controversial among those with power or with the potential to gain power.

Generally speaking, when an issue is touted as being highly controversial between the major parties, it consists of 98% solid agreement and 2% hyped up disagreement. The disagreement and the "fray" are part of the choreographed propaganda undertaken by powerful interests to create the illusion of dissent.

2
nathan_long 1 day ago 1 reply      
OK: once he got in office, it turned out that watching everyone seemed like a good idea.

But here's the thing: it's unconstitutional. It's illegal.

"The right of the people to be secure in their persons, houses, papers, and effects, against unreasonable searches and seizures, shall not be violated, and no Warrants shall issue, but upon probable cause, supported by Oath or affirmation, and particularly describing the place to be searched, and the persons or things to be seized".

Meaning: 1) You can't read my email without a warrant and 2) you need specific suspicions of me to get one.

Any interpretation that says the grocery list in my pocket is covered by the fourth amendment, but every electronic communication I make is NOT covered, is insane. "Houses, papers and effects" was the writer's way of saying "everything I can think of belonging to that person." Email and phone metadata and GPS location weren't imagined, but can you seriously say they would have been excluded?

So: balancing security with privacy is a hard thing. It is. But pooping on the constitution isn't a solution.

You want to surveil everything? Say so openly, explain your case, and try to repeal the fourth amendment. We're America: we decide by voting.

Secretly discarding the highest laws of the land is tyrannical, whatever the justification.

3
mbateman 1 day ago 1 reply      
Everything that's wrong with the current intelligence approach in a sentence: "And he trusts himself to use these powers more than he did the Bush administration."

This completely vindicates Snowden's point about the current system being one of policy instead of law, and of enabling turnkey tyranny.

4
pixelmonkey 1 day ago 1 reply      
From the article: "Mr. Obama was acutely aware of the risks of being seen as handcuffing the security agencies. 'Whatever reforms he makes, you can be sure if theres another incident and the odds are there will be in our history therell be someone on CNN within seconds saying if the president hadnt hamstrung the intelligence community, this wouldnt have happened,' Mr. Axelrod said."

And so, the wheel keeps turning...

5
equalarrow 1 day ago 1 reply      
Sigh, I guess this is nothing new. Obama's the ultimate (in my mind) say one thing, do another. At least with Bush we knew he was just a bad guy that didn't give a shit for anything other that war, greedy buddies, and a good walk on the ranch.

With Obama, yah, he came in as the 'outsider' (typical of all candidates I suppose) with all these things he would 'Change' (Shepard Fairey anyone?). But alas, it's been one disappointment after another. Net neutrality, spying, real universal healthcare, not going after politicians of the Bush area that blatantly broke all kinds of laws, etc. All a sham.

It's playing out like a sci-fi story where anyone can be an enemy of the state, just choose your own adventure: leaker, no fly list, dissident, downloader, photographer/videographer; the list goes on and one.

I suppose I shouldn't be surprised as this isn't really a democracy anymore. Every law, decision pretty much has to have some 'but what about business/economy?' question. Excessive lobbying makes sure these decisions/rules will never change short of revolution. The fact that corporations are 'people' and that they have no donation limits anymore, pretty much ends what the founding fathers fought for - we the people.

The message of the 21st century america: get rich. Get above the law and above the fold of the 99%. Go where the rules don't apply to you. Go where you make the rules for everyone else.

6
wmeredith 1 day ago 1 reply      
When you get the package every morning, it puts steel in your spine, said David Plouffe, the presidents longtime adviser. There are people out there every day who are plotting. The notion that we would put down a tool that would protect people here in America is hard to fathom.

That's the whole problem. The NSA spying is being sold as if it stops terrorist attacks. It does not. They have not cited a single incidence.

7
fit2rule 1 day ago 3 replies      
What I'd really like to know is what process does the President-elect endure that turns him into an alien lizard from hell?

I mean, seriously .. its like black and white with Obama. Pre-Presidentiality, Obama was real. After-President'ness, he's become some obscure caricature of all other Presidents who came before him..

So is there some sort of secret Presidential chamber that all the past Presidents get to donate their DNA to, which gets injected into The New Guy, to make him into some sort of transformed hybrid clone, or something? I seriously wonder sometimes, if the enemies of the USA haven't realized that the real backdoor to infiltrating America and bringing it to its knees is in the Presidential Training Program that goes on with newly elected victims. It sure seems like the President of the USA gets a new skin, anyway .. I've only been watching for the last 4 Presidents or so ..

8
wwwtyro 1 day ago 1 reply      
When you get the package every morning, it puts steel in your spine, said David Plouffe, the presidents longtime adviser.

This strikes me as backwards. Seems that a president with a steely spine would be strong enough to maintain the rights of citizens in the face of such challenges.

9
Sambdala 1 day ago 0 replies      
He has more information than he did then. And he trusts himself to use these powers more than he did the Bush administration, said the former Obama aide

Okay, so given the trend of these powers is to increase, and he's not going to be in power after 2016, does he trust the next guy with even more powers, or the guy after that with even more than that?

When civil liberties advocates visited to press him to do more to reverse Mr. Bushs policies, Mr. Obama pushed back. He reminded me that he had a different role to play, that he was commander in chief and that he needed to protect the American people,

The role of President isn't to follow through on the platform you were elected on?

10
rfnslyr 1 day ago 1 reply      
What is honestly the point of even concerning ones self with politics anymore? There has been nobody I've talked to in real life that has their head in the game completely, who knows what they are talking about, myself included.

Nothing is going to change, it's a big boys club, debating it, writing about it, all fruitless.

11
pessimizer 1 day ago 0 replies      
>Mr. Obama was told before his inauguration of a supposed plot by Somali extremists to attack the ceremony[...]. Although the report proved unfounded, it reinforced to Mr. Obama the need to detect threats before they materialized. The whole Somali threat injected their team into the realities of national security in a tangible and complicated way[...]

So a non-existent threat was what made Obama decide that the surveillance state was necessary. Great decisionmaking here.

12
forgottenpaswrd 1 day ago 0 replies      
It is not probably a good idea surround yourself with the people you need to control.

Obama spends most of his time going to eat-dinner with the same rich people that benefit from printing money. The rest of the time it is with the praetorian guard that "protects" him.

Anybody believes he is going to make the same people he surrounds most of the time furious? The same people that put him in charge?

This people are the eyes and ears of the "king of the world". He is living in a bubble.

13
mcone 1 day ago 3 replies      
What's interesting is that the article suggests that Obama himself did not know extent of the NSA's activities. ("At the same time, aides said Mr. Obama was surprised to learn after leaks by Edward J. Snowden... just how far the surveillance had gone.")

If that's true, it seems to indicate that Obama is not an overseer at all.

Have our worst fears been confirmed? Is the NSA an unstoppable organization that reports to nobody except itself?

14
RyanMcGreal 1 day ago 1 reply      
> At the same time, aides said Mr. Obama was surprised to learn after leaks by Edward J. Snowden, the former National Security Agency contractor, just how far the surveillance had gone.

If this is meant to be taken at face value, and it's at least plausible given how the US government seems to operate, how can Obama not follow it up with at least a strong commitment to making the American security apparatus more clear and transparent?

15
gumby 1 day ago 0 replies      
There is an interesting case of regulatory capture nobody seems to be discussing. From the article:

`"But they said his views have been shaped to a striking degree by the reality of waking up every day in the White House responsible for heading off the myriad threats he finds in his daily intelligence briefings.

`When you get the package every morning, it puts steel in your spine, said David Plouffe, the presidents longtime adviser. There are people out there every day who are plotting....'

and then:

`Mr. Obama was surprised to learn after leaks by Edward J. Snowden...[we all know what]'

Every morning the president gets a propaganda dose from the very people he needs to reign in. OF COURSE they are going to tell him the sky is falling in and that they are the only ones holding it back. And since it's exciting and secret there is no cross check or balance.

He should be seizing the example of Snowdon's releases to realize that the books are being cooked. Instead he's been completely taken in by the briefing books. It's really no different from Joe Barton being taken in by BP.

Back in the Reagan era Alan Kay told me about his very short time as a white house advisor. Reagan's briefing book wasn't even a book, it was a three minute video. I'm sure it REALLY played up the Soviet threat, yet the security apparatus was as astonished as anyone else when the USSR collapsed.

16
discardorama 1 day ago 1 reply      
If this thinking, that "we will do anything to prevent another terrorist attack, including give up our liberties", is acceptable, then why isn't giving up guns acceptable to prevent the next school massacre?
17
supersystem 1 day ago 0 replies      
Just seems like a worse version of http://www.newyorker.com/reporting/2013/12/16/131216fa_fact_...

By the way if your getting most of your updates on this subject from HN, you're most likely out of the loop. Since a lot of the insightful content doesn't make it. IIRC there's even a penalty on this subject on HN.

18
yew 1 day ago 0 replies      
Mr. Obama hasn't changed more than would be expected over several years and, while I can't speak with certainty, I would be very surprised to find out that he was lying about his views during the presidential campaign (I don't know about his promises).

He has more information than he did then. And he trusts himself to use these powers more than he did the Bush administration.

Even postulating some revolutionary secret information seems to me to be unnecessary.

Obama has always trusted himself. Most people do. What's changed is that the president is now someone who is trusted by Obama. Everything that's happened up to this point falls nicely out of those circumstances.

19
eof 1 day ago 1 reply      
My heart sunk when Obama won the election the first time; not because I wanted McCain to win, or that I had any actual hope that one of the third-parties could win; but because everyone was so happy.

I was in Burlington, VT; about as liberal a town as you will find in the USA; and there was a strong anti-war movement. That anti-war movement bought Obama's promises hook-line-and-sinker and the same people that were out holding signs and going to rallies were canvasing for Obama. There was a march through the streets when the counting was done; people cheered as if we were finally turning a new leaf.

I hope, so deeply, that people will have learned their lesson; that, if some politician you have never heard of suddenly starts getting a ton of press and magically enters your consciousness; he is being tapped by big players to do so. Obama, more than any other public figure in the last twenty years is proof positive that there does indeed exist a shadowy cartel that are fucking with us for power.

If you believe Obama started out pure-at-heart and was 'corrupted' after becoming president; you are naive beyond all comprehension. Remember early, early when Obama was asked about marijuana? One of the easiest, most obvious blatantly fucked up policies our government carries out.. something that every single non-political marginally liberal person is absolutely crystal clear on should be legal for adults: he laughs derisively like it's a terrible idea.

Why? What is it about Obama being so full of hope and change and feel-goody liberalness that makes him laugh at marijuana? Talk to 50 non politician democrats and you will find 49 think it's obvious to legalize marijuana. But talk to 50 politician democrats, and you will find maybe half of them. The higher up you go, the less likely they are to be pro legalization. Why? Because their interests aren't yours.

If you think that there will be ever be a 'main stream' candidate that will represent your interests over the 'shadowy cartel' of government interest and lobbyists, you are sorely mistaken; and we all pay the price.

20
mildavw 1 day ago 0 replies      
The mere hand-wringing that current leaders are doing over this is so disturbing to me because of where I see it taking the country. It's not hard to imagine, say, Carl Rove or his ilk giving someone at the NSA a wink and a nod that they'll be taken care of if their candidate wins office. The NSA leaks or hands off information that tilts the election their way. Given that collection of all of this data is A-OK, no step in this process is blatantly illegal anymore. If I were Ron Wyden, this is what I'd be saying. "Do you want your representatives picked by the NSA? Because that's the logical path we're going down."

The ability read/listen to all electronic communication without warrants gives the NSA too much power for this not to happen.

21
ck2 1 day ago 0 replies      
Like anyone ever in any kind of law enforcement position:

"it's okay when we do it"

22
coob 1 day ago 4 replies      
What is it, do you think, that causes former critics of spying to become 'overseers'? The Presidency is limited to two terms, so it's not like it's being used to weild power. Are there honest intentions related to safety, or is it all about money/lobbying?
23
dleibovic 9 hours ago 0 replies      
Mr. Obama was angry at the revelations, privately excoriating Mr. Snowden as a self-important narcissist who had not thought through the consequences of his actions.

I could not disagree more with that sentiment.

24
socrates1998 1 day ago 0 replies      
I just don't get why people don't believe politicians.

We are lied to all the time by them.

We don't need the US Congress. We don't even need the President in it's current version.

We can vote on legislation ourselves. We can approve a budget ourselves. We can veto stupid and corrupt laws.

We do everything else online, why can't we govern online?

The people in power don't want this to happen, so they convince us they are actually doing something.

26
rummikub 1 day ago 0 replies      
His administration's descent on this issue has been disappointing. I mean how is the quote any different than something that would have come out of the previous administration?

There are people out there every day who are plotting. The notion that we would put down a tool that would protect people here in America is hard to fathom.

27
lasermike026 1 day ago 0 replies      
The big lie is that surveillance has to do with nation security. It doesn't. You build a surveillance system to suppress political descent not stop terrorist attacks. Business has big plans and they don't want some republic to vote and screw things up for them. They have world to conquer. Follow the money.
28
redknight666 1 day ago 0 replies      
It is also good for the press, if he is going to do anything is another matter.
29
OFailey 1 day ago 0 replies      
Obama is "surprised to find out that people don't trust him".

Barry, that's what happens when you get caught lying almost every time you open your mouth. Clearly your mother didn't teach you much.

25
Nassim Taleb: We should retire the notion of standard deviation edge.org
231 points by pyduan  2 days ago   241 comments top 39
1
bluecalm 2 days ago 26 replies      
So first about the article:

>>The notion of standard deviation has confused hordes of scientists

What an assertion! It also proved to be very useful for hordes of scientists... what about some examples of confused scientists ?

>>There is no scientific reason to use it in statistical investigations in the age of the computer

As someone who uses it daily I am eagerly awaiting his argument.

>>Say someone just asked you to measure the "average daily variations" for the temperature of your town (or for the stock price of a company, or the blood pressure of your uncle) over the past five days. The five changes are: (-23, 7, -3, 20, -1). How do you do it?

Ok... if I am to calculate the average I am calculating the average if I need to know standard deviation I calculate standard deviation...

>> It corresponds to "real life" much better than the firstand to reality.

What the flying fuck. What "real life" ? Standard deviation tells you how volatile measurements are not what mean deviation is. Those are both very real life things just not the same thing.

>>It is all due to a historical accident: in 1893, the great Karl Pearson introduced the term "standard deviation" for what had been known as "root mean square error". The confusion started then: people thought it meant mean deviation.

I don't know how one can read it and not think: "is this guy high or just stupid?".

>>. The confusion started then: people thought it meant mean deviation.

I am yet to see anybody who thinks that standard deviation is mean deviation. It's Taleb though. Baseless assertions insulting groups of people are his craft.

>>What is worse, Goldstein and I found that a high number of data scientists (many with PhDs) also get confused in real life.

One example please ?I can give hundreds when std dev is useful and mean deviation isn't. Anything when you decide what % of yoru bankroll to bet on perceived edge for example.

Ok so he asserted that people should just use mean deviation instead of mean of squares. Guess what though, taking the squares have a purpose: it penalizes big deviations so two situations which have the same mean deviation but one is more stable have different standard deviations. THis information is useful for many things: risk estimation or calculating sample size needed for required confidence (if you need more experiments, how careful should you be with conclusions and predictions etc).He didn't mention how are we going to achieve those with his proposal. Meanwhile he managed to throw insults towards various groups without giving one single example of misuse he describes.

This is not the first time he writes something this way. His whole recent book is like that. It's anti-intellectual bullshit with many words and zero points. He doesn't give any arguments, he throws a lot of insults, he misues words and makes up redundant terms which he then struggles to define.The guy is a vile idiot of the worst kind: ignorant and aggressive. Him gaining so much following by spewing nonsense like this article is for sure fascinating but there is no place for him in any serious debate.

2
dxbydt 2 days ago 1 reply      
The notion of area has confused hordes of scientists; it is time to retire it from common use and replace it with the more effective one of circumference. Area should be left to mathematicians, topologists and developers selling real estate. There is no scientific reason to use it in statistical investigations in the age of the computer, as it does more harm than good.

Say someone just asked you to measure the area of a circle with radius pi. The area is exactly 31. But how do you do it?

scala> math.round(math.Pi * math.Pi * math.Pi).toInt

res1: Int = 31

Do you pack the circle with n people, count them up and verify n == 31 ? Or do you pour a red liquid into the circle and fill it up, then drain it and measure the amount of red ? For there are serious differences between the two methods.

If instead, you were asked to measure the circumference of a circle with radius pi.

scala> math.round(2 * math.Pi * math.Pi).toInt

res2: Int = 20

You just ask an able-bodied man, perhaps an unemployed migrant, to walk around this circle while another man, an upstanding Stanford sophomore, starts walking from Stanford to meet his maker, I mean VC, well its the same thing...

So by the time the migrant finishes walking around the circle, our upstanding Stanford entrepreneur is greeting the VC on the tarmac of the San Francisco International Airport. This leads one to rightfully believe that the circumference of the circle of radius pi is exactly the distance from Stanford to the SF Airport ie. 20 miles. It corresponds to "real life" much better than the firstand to reality. In fact, whenever people make decisions after being supplied with the area, they act as if it were the distance from their university to the airport.

It is all due to a historical accident: in 250BC, the Greek mathematician Archimedes introduced Prop 2, the Prevention of Farm Cruelty Act ( http://en.wikipedia.org/wiki/California_Proposition_2_(2008) ). No I believe this was a different Prop 2. This Prop 2 states that the area of a circle is to the square on its diameter as 11 to 14 (http://en.wikipedia.org/wiki/Measurement_of_a_Circle ) .The confusion started then: people thought it meant areas had to do with being cruel to farm animals. But it is not just journalists who fall for the mistake: I recall seeing official documents from the department of data scientists, which found that a high number of data scientists (many with PhDs) also get confused in real life.

It all comes from bad terminology for something non-intuitive. Despite this confusion, Archimedes persisted in the folly by drawing circles in the sand, an infantile persuasion, surely. When the Romans waged war, Archimedes was still computing the area of the circle. The Roman soldier asked him to step outside, but Archimedes exclaimed "Do not disturb my circles!" (http://en.wikipedia.org/wiki/Noli_turbare_circulos_meos)

He was rightfully executed by the soldier for this grievous offense. It is sad that such a minor mathematician can lead to so much confusion: our scientific tools are way too far ahead of our casual intuitions, which starts to be a problem with a mad Greek. So I close with a statement by famed rapper Sir Joey Bada$$, extolling the virtues of the circumference: "So I keep my circumference of deep fried friends like dumplings, But fuck that nigga we munching, we hungry." (http://rapgenius.com/1931938/Joey-bada-hilary-swank/So-i-kee...)

3
Homunculiheaded 2 days ago 2 replies      
I sometimes think that progress in the 21st century will be summed up as: "The realization that the normal distribution is not the only way to model data".

Taleb's favorite topic is the "black swan event" which is something that the normal distribution, and the idea of standard deviation, don't model that well. In a normal distribution very extreme events should only happen once in the lifetime of several universes. Of course assuming variation inline with a Gaussian process is at the heart of how the Black-Sholes model calculates risk/volatility/etc.

Benoit Mandelbrot argued that financial markets follow a distribution much more similar to the Cauchy distribution (specifically the Levy distribution) rather than a Gaussian. The problem of course is that the Cauchy distribution is pathological in that it doesn't have a mean or variance, you can calculate similar properties for it (location and scale), but it doesn't obey the central limit theorem so in practice it can be very strange to work with.

The normal distribution is fantastic in that it does appear frequently in nature, is very well behaved, and has been extensively studied. However a great amount of future progress is going to come from wrestling with more challenging distributions, and paying more attention to when assumptions of normality need to be questioned. Of course one of the challenges of this is that the normal distribution is baked into a very large number of our existing statistical tools.

4
beloch 2 days ago 1 reply      
I'm a physicist, so I'm one of the people this guy says standard deviation is still good for. However, despite some "oddities" (pointed out by others here) in his article, I'm more than willing to admit a simpler, easier to understand term would be helpful for explaining many things to the general public. Hell, it would be helpful for explaining things to journalists, who we then trust to explain things to the public!

Look at an reputable news site or paper. Odds are they post articles based on polls several times a day. How many report confidence intervals or anything of the sort? These are crucial for interpreting polls, but are left out more often than not. Worse yet, many stories make a big deal about a "huge" shift in support for some political policy, party or figure, when the previous month's figure is actually well within the confidence interval of the current month's poll!

Standard deviation, confidence intervals, etc. are all ways of expressing uncertainty, and it's become abundantly clear that the average journalist, to say nothing of the average person, has no clue about what the concept means. If the goal is to communicate with the public, then we really need to take a step back and appreciate the stupendously colossal wall of ignorance we're about to butt our heads against. When we talk about the general public, we should keep in mind that rather a lot of people know so little about the scientific method that they interpret the impossibility of proving theories as justification for giving religious fables equal footing in schools. This kind of ignorance isn't a nasty undercurrent lurking in the shadows. It's running the show, as evidenced by many state laws in the U.S.! There is absolutely no hope of explaining uncertainty to most of these people.

There is hope of explaining basic statistics to journalists, if only because they are relatively few in number and it's a fundamental part of their job to understand what they are reporting. Yes, I just said that every journalist who has reported a poll result, scientific figure, etc. without the associated uncertainty has failed to adequately perform their job. We need to make journalists understand why they are failing. If simplifying the way we report uncertainties will assist with this, then I'm all for it. Bad journalism is a root cause of a great deal of ignorance, but it's not an insurmountable task to fix it.

If you are a scientist who speaks to journalists about your work, make sure they include uncertainties. If you are an editor, slap your peons silly if they write a sensationalistic poll piece when the uncertainties say it's all a bunch of hot air. If you are a reader, please mercilessly mock bad articles and write numerous scornful letters to the editor until those editors pull out their beat-sticks and get slap-happy. We should not tolerate this kind of crap from people who are paid to get it right.

5
n00b101 2 days ago 2 replies      
Taleb has a good point about people mistakenly interpreting standard deviation (sigma) as Mean Absolute Deviation (MAD). I like that he gives some conversions (sigma ~= 1.25 * MAD, for Normal distribution).

I think it's rather silly to talk about "retiring" standard deviation, but we can't blame Taleb - the publication itself posed the question "2014: What Scientific Idea is Ready for Retirement?" to various scientific personalities.

What Taleb failed to mention is that, once properly understood, standard deviation has distribution interpretations that can be much more useful than MAD. For example, if the data is approximately normally distributed, then there is approximately a 99.99% probability that the next data observation will be <= 4 * sigma.

Not everything is approximately normally distributed, but a lot of phenomena ARE normally distributed. It's a well known fact that the phenomena which Taleb is most interested in (namely, financial return time-series) are not normally distributed. But I would like to know how Taleb proposes to "retire" volatility (sigma) from financial theory and replace it with MAD? Standard deviation is so central in finance that even the prices of some financial instruments (options) are quoted in terms of standard deviation (e.g. "That put option is currently selling at 30% vol"). How do we rewrite Black-Scholes option pricing theory and Markowitz portfolio theory in terms of MAD and remove all the sigmas everywhere? Surely Taleb has already written that paper for us so that we can retire standard deviation?

6
programminggeek 2 days ago 1 reply      
I think because it's called "standard deviation" that it sounds like the thing to use or look for. It sounds more correct because of the word standard.

I feel like it is the same kind of failing due to human perception of language that programmers have with the idea of exceptions and errors, especially the phrase "exceptions should only be used for exceptional behaviors". That's a cool phrase, but people latch on to it because of the word exception sounding like something extremely rare and out of the ordinary whereas we see errors as common, but they are in fact the same thing. Broke is broke, it doesn't matter what you call it, but thousands of programmers think differently because of the name we gave it.

We are human and language absolutely plays a role in our perception of things.

7
cheald 2 days ago 5 replies      
I really tried to get through "The Black Swan" and Taleb's writing struck me as so pretentious and self-involved that it made it impossible for me to finish.

He strikes me as someone who is so desperate to be important and recognized that an assertion like this doesn't really surprise me.

8
Glyptodon 2 days ago 0 replies      
All I know is this reminds me a lot of high school where we had to always compute std dev in problems, homework, and sometimes labs, but nobody really ever explained how to interpret it. It was always like "This is std dev. This is how you compute it. Make sure you put it your tables and report."

Eventually someone (or something) did explain it, but once I understood it, it became clear that it wasn't always a sensible thing to be asked to calculate but was instead just an instinctive requirement.

9
zeidrich 2 days ago 0 replies      
It's not that we should retire the notion of standard deviation. It's more that we should understand the tools that we are using and use the appropriate tool for the job.
10
justin66 2 days ago 0 replies      
Taleb has a textbook draft up which is more technical than his popular writings:

http://www.fooledbyrandomness.com/FatTails.html

There might be something there for the more rabid critics. At least it will keep them off the internet for a few days...

11
JASchilz 2 days ago 1 reply      
The central limit theorem shows us that unimodal data with lots of independent sources of error tends towards a normal distribution. That description is a good first-pass, descriptive model for lots and lots of contexts, and standard deviation speaks well to normally distributed data.

Squaring error isn't just a convenient way to remove sign, it's driven by a lot of data-sets' conformance to the central limit theorem.

12
ClementM 2 days ago 3 replies      
This article is based on paper Taleb published in 2007.If you want to test yourself, submit yourself to experiment in page 3:http://papers.ssrn.com/sol3/papers.cfm?abstract_id=970480
13
spikels 2 days ago 1 reply      
You gotta love the acronyms: STD versus MAD!

Taleb is definitely mad but his use of the MAD acronym (mean absolute deviation) is actually correct. However the STD acronym (all caps) refers to "sexually transmitted disease" and not generally used for "standard deviation". Most people use SD, Stdev, StDev or sigma.

Once again his ability to coin new terminology outstrips his ability to form coherent ideas that are anything more than trivial (eg. we have known about fat tails in stock returns for 50+ years). Like George Soros[1], Taleb's success says more about the state of the world of finance than their contributions to our knowledge.

[1]-See his book "The Alchemy of Finance"

14
lambdasquirrel 2 days ago 1 reply      
I think we'd be better off if we recognized that there are statistical distributions in the world besides the plain old Gaussian. For example, wealth does not follow a Gaussian, so why the heck do we throw around ideas like "above average wealth"?

Is MAD any better? Definitely. But I'd like to see a visual demonstration of how well it models exponential-based distributions. How well does it describe their "shape", the skew of the tail?

15
dschiptsov 2 days ago 1 reply      
Why, it is pretty good in describing probability distributions. What we should retire are idiots, who assume that it predicts an outcome of the next event.
16
ChristianMarks 2 days ago 0 replies      
Climate scientists--among others--have made similar recommendations to use the absolute mean error in place of the standard deviation, depending on the application. Taleb might have cited the extensive methodological literature--for example:

Cort J. Willmott, Kenji Matsuuraa, Scott M. Robeson. Ambiguities inherent in sums-of-squares-based error statistics. Atmospheric Environment 43 (2009) 749752.

URL: http://climate.geog.udel.edu/~climate/publication_html/Pdf/W...

17
puranjay 2 days ago 2 replies      
NNT is my intellectual superhero but the amount of hate he gets is tremendous.

Please understand that NNT's biggest issues are not so much with the way statistical models are applied to economics and finance, but how social scientists sometimes feel compelled to apply them to social fields as well, which is plain unscientific, dumb, and mostly disastrous.

So when you bear down on his arguments, please keep this context in mind.

18
thetwiceler 2 days ago 0 replies      
It is sad that Taleb does not see the value in the standard deviation; standard deviation is far more natural, and more useful, than MAD.

For example, if X has a standard deviation of s, and Y has a standard deviation of t, then the standard deviation of X + Y is sqrt(s^2 + t^2). There is a geometry of statistics, and the standard deviation is the fundamental measure of length.

To retire the standard deviation is to ignore the wonderful geometry inherent in statistics. Covariance is one of the most important concepts in statistics, and it is a shame to hide it from those who use statistics.

Additionally, I will mention that we do not need normal distributions to make special the idea of standard deviations. In fact, it is the geometry of probability - the fact that independent random variables have standard deviations which "point" in orthogonal directions - which causes the normal distribution to be the resulting distribution of the central limit theorem.

19
bayesianhorse 2 days ago 1 reply      
Nassim Taleb somehow likes to beat up on normals...

We Bayesians have similar notions, but we usually try not to overly bully frequentist methods, the poor things. Also, being familiar with Bayesian methods, a lot of what Taleb is saying sounds vaguely familiar...

20
tn13 2 days ago 0 replies      
There is nothing wrong with STD or MAD. The real problem is a lot of people apply them without realizing the nature of their data and what kind of analysis they want to do.

In this case what matters in the end is the kind of impact deviation from mean has on the real world variable you have. I agree that in most Gaussian experiments MAD might be more useful than STD.

STD is more useful when the real world impact of the deviation increases exponentially with the magnitude of deviation and hence it is a good idea of magnify the (x-n) by squaring it. In many cases the impact is linear where MAD clearly works better. For example in cricket where n runs are n times better than 1 run. But in case of shooting. Hitting 9 targets out of 10 might be 100 times better than 1 out of 10 so there MAD will be misleading.

21
cwyers 2 days ago 2 replies      
"In fact, whenever people make decisions after being supplied with the standard deviation number, they act as if it were the expected mean deviation."

Boy, is that statement useless without any kind of context, example or citation.

22
aredington 1 day ago 0 replies      
The way I read it he's proposing two things:

1) Refer to the analysis of Root Mean Square Error always by that name. (RMS is already often used in certain jargon instead of stddev).

2) Stop treating RMS as a default measure of variance. Treat Mean Absolute Deviation as the default measure of variance, because the figure it provides is more consistent with people's psychological interpretation.

It's not really retiring RMS, just retiring the idea that it is a good default statistical analysis.

23
scythe 2 days ago 1 reply      
While the mean deviation as presented is slightly nicer than sigma for intuitive purposes, it isn't as appropriate (iirc) for statistical tests on normal distributions and t-distributions.

More importantly, it doesn't fix the real problem, which is that the mean and standard deviation don't tell you everything you need to know about a data set, but often people like to pretend they do. It's not rare to read a paper in the soft sciences which might have been improved if the authors had reported the skewness, kurtosis, or similar data which could shed light on the phenomenon they're investigating. These latter statistics can reveal, for instance, a bimodal distribution, which could indicate a heterogeneous population of responders and non-responders to a drug, and that's just one example.

I'm not a statistician, so some of this might be a bit off.

24
valtron 2 days ago 0 replies      
He makes a good point about infinite MAD vs. STD.
25
afterburner 2 days ago 0 replies      
I've found MAD a potentially useful measure for monitoring whether something gets out of whack; when using STD I needed to modify it to give less weighting to outliers.
26
TTPrograms 2 days ago 0 replies      
There is some argument that MAD is actually better than RMS for a lot of applications. Apparently it predated RMS, but one of the reasons it was switched to was because RMS minimizing linear regression is much, much simpler to calculate. Also consider comparing the robustness of RMS based regression with MAD based regression. See: http://matlabdatamining.blogspot.com/2007/10/l-1-linear-regr...
27
randomsample2 2 days ago 1 reply      
Standard deviation and mean absolute deviation are both useful, but I think it's silly to suggest that we all adopt exactly one measure of variability to summarize data sets. When in doubt, make a fucking histogram.
28
MaysonL 2 days ago 1 reply      
How often do "six sigma" events occur in financial markets? A hell of a lot more often then the 0.0000001973% that they would in a normally distributed system.
29
al2o3cr 2 days ago 0 replies      
Shorter social scientists: "Gaussian distribution sez wut?"
30
snake_plissken 2 days ago 0 replies      
I've always thought his writings were more allegorical than scientific; you can't rely on the standard deviation to never go against you at the worst possible time. But like anything else, it can and it (probably) will.

Also, yes, his writing style is grating and he takes opportunistic character swipes at pretty much everyone.

31
RivieraKid 2 days ago 3 replies      
I was just wondering about a very related problem. I do 5 measurenments of some random variable (let's say execution time) and average them. How should I report the variability of that average?

State the sample size and standard deviation?

32
Beliavsky 2 days ago 0 replies      
If data is drawn from a Laplace distribution of the form p(x) = exp(-|x|), the mean absolute deviation is more informative than the standard deviation, but if its form is close to the normal, p(x) = exp(-x^2), the standard deviation is more important. So whether to use the mean absolute or standard deviation depends on the distribution of the data. There is a field called robust statistics that looks at this question.
33
notastartup 2 days ago 1 reply      
I've been a long time fan of Dr. Nassim Taleb. First book I've read was the one about his time as a day trader and how on the Black Friday market crash, he made a killing and cleared his desk and never had to work again.

There are those that dislike his ideas because it is threatening to their existing assumptions about probability and statistics. He argues that experts and majority of people do not account for the unpredictable but significant impact a single event can have which often shatters the commonly held belief. For example, swans were white until the discovery of black swans in Oceania, too big to fail multi-national corporations going bankrupt like Lehman's brothers and etc.

He's not anti-academic, but he is against teachings in the common academia that is based on naive assumptions that is specifically tailored to serve those that thrives most off the limited quantitative measures, such as market callers, hedge funds selling complicated quantitative algorithm trades, academics seeking fame and fortune by writing the most logical and quantitative paper without questioning any of the tools they are using, it is this hypocrisy and laziness that is apparent and those that try to deny to the point of making ad hominem remarks against a man, who simply observes these things and decides to write it in an entertaining manner (otherwise nobody would give a shit because the topic would be dry without lay man's linguo).

Keep an open mind, a lot of what he says I do find interesting ideas and it has influenced my thinking process quite a bit, however it's no way in anyway, grounds for cracking jokes or ridicule, in fact when I read some of the comments here, it's a bit shameful. We should be embracing new ideas in order to explore them, regardless of who the explosive nature of the claim, because the black swan event is very real and is not captured or understood completely by our current set of statistical tools and methodology based on questionable assumptions about how the real world operates. For example, 1/2500 chance is not what we really think it means in the real world because black swan events are more common than we think, a percentage probability do not fully reflect it's frequency and the magnitude of it's event.

Note the fall of crime rates in the United States following a decision to legalize abortion, economists and experts would come on television and bring up all sorts of random theories and ideas but little did they realize it was a chain effect from a court ruling passed decades ago until two economists came out with a paper that was ridiculed because it suggested that 'killing babies from poor neighbourhoods = lower crime rate' where most poor neighbourhoods is occupied by African Americans. Because such idea was earthshakingly controversial and still denied even to this day. Because Galileo claimed the earth was round instead of flat, he was executed. This is simply the nature of our world, almost all part of life, there exists a hierarchy that people simply do not ask questions either due to blind trust or the fear of reprisal.

34
vzhang 2 days ago 1 reply      
I'm seriously questioning some people's reading comprehension - he NEVER said STD is not useful! He's only saying the name "Standard Deviation" is badly chosen.
35
etanazir 2 days ago 0 replies      
The minimum uncertainty wave equation is ~ e^(-x^2) ergo the standard measure is in terms of x^2. QED.
36
tehwalrus 2 days ago 0 replies      
at least he's leaving us physicists alone with it...
37
yetanotherphd 1 day ago 0 replies      
I had hoped this would be about the revolution occurring in statistics/econometrics where confidence intervals based on strong parametric assumptions (e.g. the confidence intervals you would obtain using the standard deviation) are being replaced by confidence intervals obtained using the bootstrap (and other non-parametric methods) that don't rely on such strong assumptions.

But no, it is just advocating using Mean absolute distance instead of the standard deviation. Which I guess is to be expected from someone whose work focuses mostly on long-tailed distributions.

Still, I think that non-parametric methods are much more valuable as a solution to dealing with non-normal data than what Taleb is proposing.

38
truthteller 2 days ago 0 replies      
he's really lost the plot. :(
39
roywei 2 days ago 4 replies      
four-day returns of stock x: (-.3, .3, -.3, .3) -> MAD = 0; four-day returns of stock y: (-.5, .5, -.5, .5) -> MAD = 0.
26
PSA: Back Up Your Shit jwz.org
230 points by mfincham  4 days ago   127 comments top 27
1
steven2012 4 days ago 14 replies      
I think that the beauty of Snapchat is that it frees you from this ridiculous notion that a text, IM, Facebook message, etc, has any value.

In my opinion, it doesn't. Also, in my opinion, I believe that feeling the need to save every single conversation you have fuels an over-inflated sense of self-worth, and that everything you say has value and needs to be saved.

I never, ever peruse through my messages, to reminisce over an old conversation. It's too much navel-gazing to suit my sense of pride. What actually matters is the actual relationship you have with a person, which is built on the BODY of IMs, messages, conversations, visits, dinners, parties, etc, that you shared with that person. Sometimes, it's best to leave good conversations in the blurry past, and just remember that a certain person is funny, a great conversationalist, etc.

I'm doing the same sort of thing with Google now. I will disallow anyone I'm in a conversation with to google facts with their phone. When we talk, it's about whatever resides in our own brains, be it good, bad or ugly. The entertaining part of any conversation is the actual conversation, the passion, the humor, etc. If all we wanted to do was pass around facts, then we can forward each other URLs and be done with it. When I'm talking with someone over dinner, we're not hammering out a contract that requires precision, we're having a conversation over ideas, and as funny as it sounds, facts aren't as important as the spirit of the conversation. Unless of course you're in an argument with someone, and then that isn't very much fun so why even bother starting the conversation in the first place.

2
famousactress 4 days ago 4 replies      
Thanks for this. The SMS export from iPhone is something I've been looking for. One of the most important relationships and experiences of my entire life has been documented (trapped) in my phone and it's backups ever since.

I'm looking forward to seeing how well it works, specifically whether it can pull photos/videos as well. If it doesn't yet but it wouldn't be too much trouble to add, I'd be willing to literally pay you to add that.

[Edit: Since a lot of the other comments are questioning the value of saving this stuff I figured I'd share my use cases. It turned out when I thought about it I have at least three:

1. I effectively met my wife on myspace (believe it or not a pretty nasty software bug led to our relationship) and an enormous amount of our initial friendship and courtship ended up documented there. Years ago I painstakingly clicked through for hours and copy-pasted the conversation to a text document.

2. I had a close friend die very suddenly and at a young age. My memory generally kind of stinks and I hated that there were conversations with him that I half-remembered. I went back through social media conversations with him (again, mostly on myspace) a lot in the years that followed. It helped me piece together memories that are very important to me now.

3. This past year my wife and I adopted our daughter. Our relationship with her birthmother has primarily been via SMS and the months that followed were a really exhausting and beautiful blur. It's really important to us that we're able to share that thread with our daughter someday.

In none of these cases did I see it coming that these services would end up having such valuable content in them for me. I didn't know I'd meet my wife. I'll never know when the last time I talk to someone is, and I would have never guessed that one of the most important things I'll have to give my daughter about her birth story is an SMS conversation.

So yeah, having access to this stuff is important to me. Thanks to jwz for pulling these resources together.]

3
borski 4 days ago 17 replies      
"You don't just throw your letters in the trash. You might want them some day."

Maybe it's just me, but I actually /do/ throw my letters in the trash. I /do/ treat Twitter, etc. as ephemeral and passing. I don't care about saving those messages. Am I the only one?

4
randomdrake 4 days ago 1 reply      
Accessing your own data and storing it is great, but there's still the matter of backing it up. jwz wrote a good guide for that as well. It's linked in the article, but not in a way that makes it obvious. Thought it would be good to mention it here:

http://www.jwz.org/doc/backups.html

5
enigmabomb 4 days ago 0 replies      
PSA: This guy's nightclub makes a really mean meatball sandwich.

Make sure that recipe is backed up.

6
dkokelley 4 days ago 1 reply      
Honestly, my Twitter feed, Facebook, and SMS records could all disappear tomorrow, and I would be OK with it. Maybe there's value in my accumulated Facebook connections and history, but most of the value today comes from current content.

Now email, that I value for archival.

7
ThatGeoGuy 3 days ago 1 reply      
I hate to be that guy who plugs his own crap everywhere, but I actually wrote my own blog post recently about backing up my stuff on Linux.

My setup is fairly rudimentary, and I had the help of a friend on IRC, but here's the link if anyone is interested in setting up something simple for a Linux workstation at home or a VPS you can ssh into (really, as long as you can SSH into it with rsync, my method will work). I'd also love any feedback HN can give regarding my mechanism. Hell, if you wanted to fully back up a phone and sdcard on your desktop, you could probably do something similar with "adb pull" or the like.

https://thatgeoguy.ca/blog/posts/howto-encrypted-backups-in-...

That out of the way, I'm often surprised by how often I have to remind either myself or others to make good backups. Phone's aside, there's been enough times where I've nuked my system that backing up all my files should be secondhand at this point. Thankfully, I have a decent system set up now, but I still consider it rough around the edges (especially considering how long archiving backups takes).

8
aestra 3 days ago 0 replies      
I want to have some record of some things but not everything by far. For example, I wouldn't want a record of everything I ever did recorded, but my dad used to walk around once in a while with a video recorder on special times when everyone gathered (birthdays, holidays...) and looking at those artifacts of life from almost 30(!!) years ago is priceless. Many people in those tapes have died since then, and I'm glad they exist. I save a few letters and emails, but not all or even many. I was going back and reading an email I sent a friend about getting together with an ex and seeing the perspective I had on things back then was... weird. When I was in high school (before cell phones or text messages) people passed notes in class, and some of my friends still have a box full of them. They are relics of the past. They take up space, and you probably won't want ALL of them, but I think it is worth keeping a few. I wish I kept one or two. I can't imagine the vacant things mine would have contained. I mean, I still have my yearbooks, I didn't throw those out... same kinda thing really.
9
Ellipsis753 3 days ago 0 replies      
I'm on Gentoo Linux and had my Skype settings set to never delete chat logs. After a couple of months these logs were in the tens of gigabytes. The strange thing is that I rarely even chat on Skype. This should be tens of gigabytes of just text chat. Well it's not any format that I can understand (and Skype lags badly and becomes pretty much unusable if I type /history to look at the logs) so I've had to delete them and for a while now Skype only stores the last month of chat. I can't think why the logs got so big. Perhaps Skype trys to optimize them for quick searches or something?

Anyway, does anyone know of a way to back up Skype text chats? They shouldn't have to use up this much space. (And ideally I should actually be able to load them up and read them too!)

10
anigbrowl 4 days ago 0 replies      
These conversations aren't ephemeral and disposable, they are your life, and you want to save them forever.

Yes they are, and no I don't. I highly doubt JWZ carries a portable recorder to immortalize all his in-person conversations; I certainly don't, even though recording people (for movies) is what I do for a living. Funnily enough, far more of my important memories involve real-life conversations than exchanges on IRC/Facebook/HN.

Yeah,. it's good to have a method of backing this stuff up if you do need it, eg for business communications or any number of other use cases. But most digital chatter is eminently disposable I wish there was a way to have emails expire and self-destruct automatically, so that things like time-sensitive sales offers would quietly vanish once the actionable date had passed unless I made some special effort to retain them.

11
gwu78 3 days ago 3 replies      
"Remember: if it's not on a drive that is in your physical possession, it's not really yours."

So, if we store our data in "the cloud", it's not really ours?

12
codva 4 days ago 3 replies      
I delete all email after 90 days, unless I explicitly moved it to an archive folder.

I've never event thought about saving IMs, texts, Twitter, etc. Civilization has survived a very long time without a written record of every conversation ever. It will continue to do so.

13
flipstewart 4 days ago 1 reply      
I do throw away letters. I'd rather not live in the past or cling to ephemera for emotional reasons, thank you.
14
sturmeh 4 days ago 0 replies      
Chat history serves one purpose for me, the file size quantifies how much I spend talking to a particular person, and I use that to sort people on my contact list.
15
dhughes 4 days ago 1 reply      
Pictures are the worst for backing up, actually no backing up someone else's pictures is worse.

Parents for example, my mom takes a lot of pictures she wants to keep I take lots of pictures I don't care about.

Semi-wheneverly when I manage to get the card from the parent's camera or cellphone to back up it's usually a mess of I backed up 63% of these so which ones are new. Is IMG0003.JPG the same as IMG0003.JPG I saved already wait no one is 2MB and the other is 3.25MB.

Meld helps but it's the same thing what do I have and what is new and what is different with the same name but is different which I just happen to notice due to the file size.

So I end up dumping it all onto something or multiple somethings and swear I'll figure it out next time. Goto step 1.

16
davidgerard 2 days ago 0 replies      
Inspired by this, I went and downloaded all my tweets last night.

Then I looked through them. I can assure you that tweets may not be ephemeral, but they are most certainly disposable.

17
mathrawka 4 days ago 1 reply      
Totally off topic, but the I can read the site fine, but if I switch back to white color site (like HN, or just staring at a wall), my eyes still see the lines of the site for awhile.

It physically affects my vision for a few minutes, albeit just a little bit. Is this normal?

18
X-Istence 4 days ago 0 replies      
The SMS backup tool for iPhone doesn't seem to work for me. I have encrypted backups turned on for my iPhone, will this not work because of that?
19
ballard 4 days ago 0 replies      
Backup personal stuff and code to Tarsnap. Videos would be too expensive, but downsampled home videos might be worth saving too.
20
Aloha 4 days ago 0 replies      
It drives me nuts that pidgin and adium use different logging interfaces - its made switching from Windows to OSX more painful - as I still use finch on Linux, and use file syncing to sync logs and config files across platforms.
21
FollowSteph3 3 days ago 0 replies      
I view this as no different than backng up phone calls. And most people don't care to back up their phone calls. Just because you can doesn't mean it's always worth it...
22
normloman 3 days ago 0 replies      
Always back up online chats so you can blackmail the participants with embarrassing or incriminating statements they made.
23
qwerta 4 days ago 0 replies      
On android you just mount phone partition and query SQLite tables.
24
mmanfrin 4 days ago 3 replies      
PSA: It is not 1999, please don't use neon green text on a black background as your color scheme for text.
25
jheriko 3 days ago 0 replies      
along these lines i'd highly recommend: http://socialsafe.net/
26
LeicaLatte 3 days ago 0 replies      
Social media is just excreta of human activity on the internet. Why back that up? Lame.
27
bestspellcaster 3 days ago 0 replies      
I want to use this opportunity to thank drstanleyspelltemple@hotmail.com for helping me get my lover back after he left me few months ago. I have sent friends and my brothers to beg him for me but he refused and said that it is all over between both of us but when I met this Dr. Stanley, he told me to relaxed that every thing will be fine and after three days and contacted him, I got my man back......Caitlin
27
What Happens When the President Sits Down Next to You at a Caf theatlantic.com
223 points by pliptvo  15 hours ago   145 comments top 36
1
SandB0x 14 hours ago 13 replies      
I swear Facebook is slowly turning into Lotus Notes. It's got the the little messenger panel on the right. It's got the clunky mail client, it's got the calendar and you can invite people to a meeting/party. You have to use it because everyone else uses it and everyone else uses it because it's been around for years.

It's not fun or simple to use any more and you have to be on your guard about what you share and who might see it.

It's not really surprising that teenagers aren't into it.

2
dredmorbius 14 hours ago 4 replies      
The question's been asked "can a company die without an obvious challenger?" Yes, it can.

A company, platform, or technology can "die" in the sense that it loses the initiative, and more importantly, the ability to drive an industry and/or conversation, even though it hasn't yet died.

Apple was "dead" through most of the 1990s. It simply didn't matter, outside of the graphics and design areas, and for a very small cadre of fervent fans. The turnaround shocked me.

IBM very nearly died in the early 1990s, as its place as the center of the business computing world was shaken by anti-trust actions, Microsoft, and the upsurge in Unix vendors. The company's never fully regained its former footing, though it did recover largely.

Microsoft has been in the process of dying for most of the past decade. A highly symbolic moment for me was when The Economist newspaper ran a cover showing the leaders in tech: Apple, Google, Facebook, and Amazon. Microsoft wasn't even mentioned (it reminds me of an earlier revealing moment when the CEO of Visa International named the company's biggest rivals: MasterCard, AmEx, and Microsoft -- I guess it didn't pay to Discover...).

Sun Microsystems was fingered for the walking dead as Linux became ascendant, with its acquisition by Oracle (a panic response of both companies, coming at least five years too late to do either any good) coming long after it was obvious the company had not only staggered but was mortally wounded.

One thing to realize is that a fading icon is often not replaced by a direct competitor, but by one which addresses short

Facebook has dominated Silicon Valley for the past 5 years, stealing initiative from Google (who seems to be somewhat winning it back). Part of the situation is that "traditional" social networking is becoming passe, in part because it's become too Byzantine, and too intrusive. Social networks -- real social networks, not the online instantiations of them -- work best when the groups are relatively small, Dunbar's number is respected, and there's a level of insularity around any given group. TheFacebook at Harvard had those features. Facebook, Inc., 1 billion served, doesn't, and cannot. Another secret is that one of the secret sauces of social is photo sharing (still hard if you don't have your own dedicated server), and that services are sprouting up to offer this (Imgur, Snapchat, etc.), which is essentially disrupting the former Social glue much the way Craigslist gutted classified newspaper advertising in the late 1990s.

I'd like to think that the constant drumbeat of surveillance state revelations we can thank heros and patriots Edward Snowden, Laura Poitras, and Glen Greenwald for are having an effect, though I think that may be giving the general public a bit too much credit. Not totally sure of that though.

3
mberning 12 hours ago 4 replies      
Am I the only person over 30 that uses Facebook, doesn't think it sucks, and doesn't think it is dying? In fact, for my purposes, I think it is a great experience.

Let me explain.

I am quite involved in the automotive community. In the last year or two it seems like almost every manufacturer, tuning shop, engineering shop etc. has set up a GOOD Facebook page and starting posting tons of cool and relevant stuff every day. News, project updates, pictures of upcoming work, event info and pictures...

I feel as if life could not be better right now for the person that wants to get plugged in to the automotive scene.

Log in to Facebook, find the page for your favorite manufacturer or shop and like it, add all your car buddies as friends, ignore/unfollow the shit you don't like, get invited to some private groups by your friends, etc.

I don't know, seems like for me there are quite a lot of people using Facebook that actually like how it works and what it enables.

4
hawkharris 12 hours ago 1 reply      
While I want them to be true, the claims that Facebook is dying don't seem to be supported by data.

Facebook's recent engagement report (also posted on HN) showed that the percentage of users 25-31 has grown by 32%. The percentage of people 35-51 grew by 41%. And the percentage of those over 55 grew by 88%.

What other commenters probably mean to say is that Facebook is dying among teenagers. Even so, the website still has over 13 million teen users. And the users in the older groups are arguably more valuable from a financial standpoint because they have direct access to more disposable income.

Maybe you're skeptical of the report because Facebook helped generate it. Okay, I can understand that. But even if the numbers have been inflated to benefit Facebook's platform, I think you'll be hard-pressed to find any legitimate, large-scale study that doesn't show significant growth in the 25+ demographic.

Uncool? I think so, and a relatively small demographic of young people agrees with me. Dying? Definitely not.

http://istrategylabs.com/2014/01/3-million-teens-leave-faceb...

5
r0h1n 12 hours ago 1 reply      
>> Facebook is so uncool even the president of the United States knows it.

I'm trying to parse the actual meaning of this editorialized headline (the actual headline is "What Happens When the President Sits Down Next to You at a Cafe").

Does this equate the President of the United States to being one of the least clued-in/laggard Internet users? Otherwise what does "even the President of the US" mean?

6
bloat 14 hours ago 1 reply      
Why should it be cool? Why can't it just be useful?

There are many, many uncool people in the world, or people who couldn't care less what is cool, and who just want to keep in touch with some mates, or find out when the next meeting of their local cycling club is, or find out what was played on the most recent podcast they listened to, or, or, or, a hundred other uses.

7
ck2 13 hours ago 1 reply      
Facebook is dying? Best news I've heard this year so far.

Is having your own website cool again yet? Wake me up when we're there.

8
zaidf 14 hours ago 1 reply      
This is actually a great milestone. It is hard to build a sustainable billion dollar business on "being cool" because any minute society might decide you're no longer cool and move on to the next shiny object. Just ask myspace.

Facebook isn't a billion dollar business because it is the new shiny object on the block. it is used because it delivers good old value of helping you stay in touch with your friends.

9
k-mcgrady 13 hours ago 3 replies      
Even Zuckerberg doesn't care if Facebook is 'uncool':

Maybe electricity was cool when it first came out, but pretty quickly people stopped talking about it because its not the new thing, the real question you want to track at that point is are fewer people turning on their lights because its less cool?

10
sdfjkl 12 hours ago 2 replies      
> His hands grasp mine. They feel like the rough surface of your favorite baseball.

What's with this president cult. He's just another human being.

11
the_watcher 10 hours ago 0 replies      
The "Facebook is uncool and dying" story trend is getting so old. It's not dying. The only metric that anyone ever cites is that young teens aren't signing up as often anymore, but they don't mention that user growth on Facebook is still strong, or that most of those young teens will sign up once they hit an age where its utility for them is higher (so an age where they don't interact with 90% of their social circle every day). They are still rolling out really cool new features (Graph search, user-specific trending topics). They are way ahead of the game in mobile (already moving to having multiple specialized apps for different functionalities that deeplink to each other, rather than a single app that tries to pack all of the desktop functionality into it).

Facebook is not dead, dying, or sick. It's in fantastic shape.

12
bqpro1 13 hours ago 1 reply      
I case of such companies like Facebook there is no point to say that they are dead. In fact there are 2 Facebooks: Facebook as a company and FACEBOOK as a presence in modern culture. The second is much more important, then the first one. For many people FACEBOOK is the way they act on the web and communicate with others. Even if Facebook is not growing any more (or even shrinking) it still the only medium that has a digital picture of relations between 1 billion people and that makes is something more then just a company. Of course if I were 16 now, I would not be on Facebook, because in this age you have a lot of things that you wish to keep in secret. But still Facebook is one of first app we install on a new mobile device. I do not see any comparison between Fb and Microsoft or Fb and Apple. Facebook do not produce any goods etc. It just mapping relations and other features of people into digital data.
13
wanda 14 hours ago 2 replies      
The parts the author did not hear:

  We're watching all FB activity and we're stumped  What social networks should the NSA target?  How can the NSA deal with the impermanence of snapchat content?  "They" being the 18-35s, prime hacker demographic, main target of surveillance.

14
vincie 14 hours ago 0 replies      
Facebook has always been uncool. The uncool people are just saying this now to sound cool.
15
billyhoffman 11 hours ago 0 replies      
I can understand the importance of keeping up with people, but I think Facebook lets you do it at such scale that it becomes meaningless. I care what's going on in the life of the best man at my wedding. I care less what's going on in the life of my lab partner in chemistry all those years ago. Facebook makes it trivial to track both which from a technical side is pretty amazing. However I have limited attention and tracking the lives of that many people is just not sustainable.

About a year ago, I deleted every "friend" I had not physically seen and spent time with in the last 18 months. This might not work for everyone, but for me it was incredibly refreshing. Fewer, higher quality relationships was far better. (and yes, I took this to the logical conclusion and just stopped using Facebook. I have not missed it)

16
unethical_ban 7 hours ago 0 replies      
Meta-comment: I read this article with the real title on HN, and it mentioned almost nothing about Facebook. I assume the mods changed the title, which steered the early conversation about the piece. Therefore, due to "title activism" taken too long after the post, I was thoroughly confused for a while.
17
jgreen10 14 hours ago 0 replies      
Things that were once cool either fade or become normal. Facebook has become normal.
18
KVFinn 8 hours ago 0 replies      
>None of this kept me from experiencing immediate, full-on, feverish anxiety.>And thenfor the first time in nearly an hourI could work. I found that I was so accustomed to his voice, how he holds his body, his aura, that ignoring him in person is as easy as ignoring a TV. Easier, in fact. He stops being the president and starts being That Guy Who You See In Tweets, That Guy Who Gives Speeches, That Guy.

Interesting how your mind can put the president into the same bucket as leaving the news on in the background.

19
betawolf33 14 hours ago 3 replies      
"one of our agents will be coming around to swipe you" <- I honestly don't know what this means.
20
canvia 10 hours ago 0 replies      
I think this is in part due to people realizing how much data FB is keeping on them, and then selling. Educated people don't want to be a commodity. Older people are slower to learn/change, but young people are already done with FB.

FB is bloated and appeals to the lowest common denominator. They created a set of easy tools that make the internet more accessible to people that are not technologically knowledgeable. Skype and photobucket and twitter are far too complicated for someone who thinks Outlook is email and Yahoo is the internet. Once people master how to use the tools on FB many will realize that there are better alternatives, some that aren't going to track them all across the internet.

However, there is no single alternative that aggregates these tools in a better fashion.

I was just writing a proposal last night for a new social site to replace FB/g+ that is freemium subscription based with a key selling point that there is no tracking, no ads, and a high level of privacy. It would have a base set of features and that's it. No bloat. No expansion. Just sharing with friends.

It doesn't need to be a billion dollar company. Profit margins don't have to be huge. It just needs to be profitable and deliver a needed service to the niche of people that care about privacy. The rest will follow with network effects. If the tool is effective people will use it forever even without feature creep, i.e. usenet, IRC, HN

21
wambotron 10 hours ago 0 replies      
I always see the "everyone else uses it so I have to" argument when it comes to facebook. I don't use FB at all, and neither do my core group of friends. None of us have run into any issues with this. Want to hang out? Call, email, text. Want to have a get-together? You need to notify me way in advance anyway, and I'll add it to my own calendar.

The whole thing just seems to me that people LOVE facebook-stalking other people, and THAT is why they use it. So many people I work with check facebook for people before interviews or even just meeting a new person. It's creepy.

I don't care what someone puts on their facebook profile. I don't want to or need to know.

22
triangleman 13 hours ago 2 replies      
>According to White House background, provided to me after he left, they met to discuss how to get more 18-34 year-olds to sign up for the coverage under the Affordable Care Act. (The law depends on 18-34 year-olds signing up for healthcare.)

Doesn't the law _compel_ 18-34 year-olds to sign up for health insurance?

23
rwmj 14 hours ago 2 replies      
Is it supposed to be satire? Anyway, I don't think Facebook can be dead without an obvious challenger. There's presumably still a need for people to chat with each other in the way that Facebook does.

(Also can't believe I'm defending Facebook here ...)

24
JacksonGariety 7 hours ago 0 replies      
> "Facebook is so uncool even the president of the United States knows it."

Shouldn't he know better than anyone, since his job depends so heavily on social media?

25
wtvanhest 12 hours ago 0 replies      
Mark my words. There is a place I'm the market for an event manager and digital rollidex to replace Facebook for adults. I don't know how someone will get people to use it, but there is absolutely a place for it. I would leave Facebook immediately if I wasn't going to lose all my contacts.
26
puranjay 11 hours ago 0 replies      
When you talk about Facebook dying, do remember that Facebook owns Instagram where all the cool kids hangout these days.
27
farinasa 11 hours ago 0 replies      
Is it really surprising that the guy with access to all internet traffic has knowledge about internet traffic?
28
jokoon 12 hours ago 1 reply      
I think facebook failed to diversify itself. It stagnated. No local events, no job searching, no tools or ways to search for new people to meet like meetup, no true dating functionality (and god knows there would be a lot of potential).

To make things worse, there were many stories that tainted facebook reputations: the ad system, the privacy settings, the first suspicions of CIA links, and then the Snowden leaks.

It's no surprise. I'm still amazed there are people actually complaining about google+ and making fun of it.

Zuckerberg is not even leading it like Steve Jobs or Bill Gates or Google founders would.

Facebook is just a startup success story, but that ends here. It makes money, people use it like an easy-to-use internet forum and chat applications, but it's as much social as everything else that existed before it. It's just user friendly, and money making.

29
poopsintub 13 hours ago 0 replies      
Facebook's dead based on half a conversation someone might have heard. Keep the journalistic gold coming this way, The Atlantic.
30
Gaurav322 13 hours ago 0 replies      
Robinson Meyer, I think that may be he talked about Facebook?? But, as you said in your blog-post that you missed out so many things due to noise and you clearly listened only two words "Instagram" and "Snapchat" from the president of USA... So, all are you doing is just guessing that If he doesn't talk or you don't listen about Facebook from his mouth, then Facebook sucks... That's not cool... man
31
shmerl 10 hours ago 0 replies      
Facebook was never cool. As well as any other centralized social network.
32
gress 9 hours ago 0 replies      
Another attack piece against Facebook. I wonder who is paying for the current media onslaught?
33
desmondrd 11 hours ago 1 reply      
HN used to be full of things that mattered. This belongs on Reddit.com.
34
yaph 11 hours ago 0 replies      
I suspect Mr. Obama knows more than you like about Faceboook et al.
35
chainlink 10 hours ago 0 replies      
36
julien_c 15 hours ago 0 replies      
Funny piece.
28
The Next Phase of Node.js nodejs.org
223 points by sintaxi  2 days ago   129 comments top 22
1
beaumartinez 2 days ago 4 replies      
> I am starting a company, npm, Inc., to deliver new products and services related to npm.

I don't know whether I should be concerned a core part of Node.js is becoming "businessy". Is this common amongst other big software projects?

Providing "premium" services is a slippery slope. What happens when someone wants to add a feature to npm that the premium services already provide, for example?

Python and Django have non-profits which help sustain their ecosystems, but AFAIK Node.js doesn't. Perhaps it would be a good step.

2
hartator 2 days ago 7 replies      
What about the bnoordhuis story?

It seems not to be active again for his lib: https://github.com/joyent/libuv/graphs/contributors

For people who don't know, bnoordhuis exiled himself 2 months ago after the bashing the node.js community imposed him for not merging a pull about gender. It was a shame because he was in the top 3 contributors. (all details: https://github.com/joyent/libuv/pull/1015)

3
bhouston 2 days ago 1 reply      
So NPM Inc.? Interesting. I do think NPM requires its own team, but I am unsure if NPM itself is a business (although I can see it being a sponsored foundation.) But there are hopefully creative business solutions to be had here.

Q: How does this relate to the money I gave to the Scale NPM project a few months back? https://npm.nodejitsu.com/

4
nilsbunger 2 days ago 1 reply      
What's next?The pip corporation?Worldwide Gems Inc?

"Global ./configure&&make&&make install Industries" ?

5
jnardiello 2 days ago 1 reply      
How about the 200k Nodejitsu crowdfunded not long ago for npm? Also, how the npm revenue model is going to affect packages distribution? To whoever may concern: I expect some clear answers. There's simply too much money going around Node at the moment and while it might eventually be a good thing (as involved companies will push node adoption among devs), it's also scaring and quite weird.

[EDIT: Isaac already partially replied, see other answer]

6
rtfeldman 2 days ago 2 replies      
I assumed from the title this would be an announcement about Node 1.0, but I'm not terribly surprised to see the trend continue of approaching 1.0 asymptotically.

Still, this is interesting stuff. Does anyone know if any similar repos (Ruby Gems, etc.) have their own for-profit companies?

7
majke 2 days ago 0 replies      
Good luck Isaac and thanks for all the fish!

You did a great work as a node lead. The brief moment when our paths crossed, due to a security issue in node, was handled perfectly. Seriously. You will be missed.

8
transfire 2 days ago 1 reply      
NPM "app" store? Should I laugh, cry or try to monetize?
9
ozten 2 days ago 0 replies      
I'd love to see an alternative to npm that:

* Puts security as a high priority

* That is operated as a federated system (think bitcoin block chain)

* Puts uptime as a high priority

Many companies and individuals could run deployments of it, removing the need for a new NPM Inc to pay for service costs.

10
niix 2 days ago 0 replies      
Awesome, so glad Isaac gets to focus on what he loves. Congrats to TJ on his new role!
11
thepumpkin1979 2 days ago 2 replies      
"Open-Source Democracy 101"... 3 Leaders in less than five years: Ryan Dahl(2009-2012) -> Isaac Z. Schlueter(2012-2013) -> Timothy J Fontaine(2014-2016?)
12
matan_a 2 days ago 0 replies      
While there are ways to create private registries [0], having a new packaged way to do it would be very useful to some orgs.

[0] https://npmjs.org/doc/registry.html#Can-I-run-my-own-private...

13
edwinnathaniel 2 days ago 1 reply      
There's nothing wrong to focus on NPM.

Take a look at Maven[http://maven.apache.org] and Sonatype[http://www.sonatype.com/] (dependency management in Java-land).

Works awesomely (and maybe even better!).

14
lnanek2 2 days ago 0 replies      
Kind of disappointing this wasn't more about the technical direction. Really feel the title is misleading and should have been something about leadership/people change so I would have known not to bother reading it.
15
binocarlos 2 days ago 1 reply      
"The peacekeeping budget for the 201314 fiscal year was $7.54 billion" - this is from the Wikipedia UN page

I'm not saying that npm is like, that important, but - take 0.01% (one ten thousandth) and npm would have an annual budget of 3/4 million bucks.

Will we get to a place where core software distribution to devices not humans is deemed as critical infrastructure?

It would be great to know that npm (and github and every other package distribution tool) were somehow too big to fail like banks have shown to be.

Meanwhile - everyone who has worked hard to make npm and node brilliant - thank you!

16
dmourati 2 days ago 1 reply      
The next phase of npm should be to fix this open issue:

https://github.com/npm/npm/issues/4131

17
drderidder 2 days ago 0 replies      
I just wanted to wish Isaac the best with his new initiative. He's been a great contributor and helped make the node community a fun and friendly one.

[edit] - originally thought TJ Fontaine wasn't employed by Joyent but apparently he is.

18
yachtintransit 2 days ago 0 replies      
i think is a great thing! communities flourish when they are support by companies with effective ( preferably transparent) revenue models. exciting times . npm inc , best of luck !
19
mmaunder 2 days ago 0 replies      
I suspect that the ecosystems forming around node will find more success by emulating what Ubuntu has done vs RedHat.
20
jaiball 2 days ago 1 reply      
wonder if this has anything to do with the npm maintainer who is missing. Hope he's ok.
21
drakaal 2 days ago 0 replies      
Node is becoming a business not a community, and the community is taken by surprise. I saw this coming, but I think many people thought Node was about free love and changing the world. It is down hill from here. You can avoid a lot of politics when there isn't any money involved, but now there is, and that changes everything.
22
calroc 2 days ago 1 reply      
Node.js is a scam.
29
I never finish anyth greig.cc
222 points by 3stripe  13 hours ago   78 comments top 38
1
Arjuna 11 hours ago 4 replies      
Just some encouragement here for you good people...

You have to do the work. There is no one else that can do it for you. It is entirely up to you; actually, it has only ever been you, because it is you that rolls out of bed early to bring it. No one is going to do that for you. If you want to do it, then you will figure out a way to make it happen.

Like some of you, I have a family. That makes things a bit challenging at times, and you will likely have to work even harder to find the time, to make the space, so that you can bring it. And, I will add, those that are parents, this is a great example to set for your children. You want to be, "The Daddy that brings it." You want to be, "The Mommy that brings it." Because, you want to instill in them that they can, too. It's a valuable life-lesson. In turn, it is psychologically healthy for you as well, to know that you are a parent that is firing on all cylinders. In addition, it is incredibly gratifying when your child sees your work. "Wow... that's the new game that you're working on! Let me try!" It will fuel you like a Saturn V launching to the moon.

You may look at your project and think, "I'm never going to make it. I'll never finish." Please, I urge you to set these thoughts aside and push through. Think about the analogy of building a wall. A wall is built one brick at a time. Watch a mason build a wall one day. You will observe that he or she lays one brick at a time.

This is how you have to view your project. Sure, it would be amazing to have an entire day, every day to devote to your project. However, the reality is that most of us simply do not have that luxury. So, strive to think of it as a mason: lay one brick at a time, and eventually the wall will be built. Every character that you type into Xcode, Visual Studio, etc. turns into a keyword, a variable name, etc... that subsequently turns into a line. Those lines build up, day by day, and before you know it, you have a program, and you look back and think, "Wow, why did I ever think I could not finish?"

Also, let go of "Internet Time." That is to say, we all read HN and see these impressive "Show HN" posts, and submitted stories about the Next Big Thing... and it seems like things are happening so fast, and we think, "Why even try?" Well, the reality is nothing is happening fast. It is an illusion. Most all of these stories have an incredible amount of time and work behind them, so let go of that illusion, get started, and stay focused.

"There's only this moment and the next moment. Every one of those moments is a test that you get to take one time and only one time." [1]

Strive to drive through each moment. Make it count.

You have to fight. This is paramount. I will say it again... you have to fight! What I mean here is fighting by engaging your Will. Engage your will to get up, to get moving. Engage your will to eat right, to exercise and go to bed on time so that you have the energy to get up and bring it.

What you are going through is what we are all going through; that is to say, we are all grinding, whether it be in a start-up business, or washing the dishes by day as we bootstrap a start-up at night; we are all struggling, fighting to drive our dreams into existence. We are all struggling in some way, whether it be through failure, health issues, personal issues, family issues, etc. No one is immune from the grip of suffering through his or her struggles.

You are not alone.

Embrace the grind. Vince Lombardi said it best:

"And in truth, I've never known a man worth his salt who, in the long run, deep down in his heart, didn't appreciate the grind, the discipline. The difference between a successful person and others is not a lack of strength, not a lack of knowledge, but rather... a lack of will."

You have to use the will to fight those thoughts that say, "I'm getting older. I'm failing. I'm not motivated. I can't finish."

The clock is ticking for all of us... fight! You never know where your work will take you. Do not forget that, you have to dream it first in your mind before you can see it in your life. And to see it in your life, you must work. You may not be able to see things clearly now, but you never know what doors could open for you that you did not even know existed.

Ang Lee, the Taiwanese film director and screenwriter that directed Crouching Tiger, Hidden Dragon and Life of Pi, discusses his struggles to break into film. I urge you to read it:

http://whatshihsaid.com/2013/02/26/ang-lee-a-never-ending-dr...

[1] https://www.youtube.com/watch?v=pBLV6RG4TwY

2
visakanv 12 hours ago 5 replies      
I relate to this, too. Here's some stuff that I read recently that helped me a little:

"When you are young, beginning new projects is easy and finishing them is hard. As you grow older, beginnings get harder, but finishing gets easier. At least, that has been my experience. I think it is true of anyone of at least average intelligence, creativity and emotional resilience. The reason is simple.

When you are young, the possibilities ahead of you, and the time available to explore them, seem nearly infinite. When you try to start something, the energizing creative phase, (which comes with internal brain-chemistry rewards on a fast feedback-loop), gives way to exhausting detail-oriented work, maintenance work, and unsatisfying overhead work. You need to get through these to bank distant external rewards (money and such) that only come with completion. It is then that you are most vulnerable to the allure of exciting new beginnings. So you abandon things halfway. You bank the internal rewards of beginning, but not the external rewards of finishing.

But with age, this changes.

As you grow older, the history of a few completed projects and many abandoned ones in your past starts to loom oppressively in your memory. The early internal rewards of many beginnings are now a distant memory that offer no pleasure in the present. The external rewards of completed projects, which tend to continue to yield dividends (such as completed degrees, financial rewards) loom larger all around you: wealth, strong relationships and perhaps most importantly, an earned ability to see the world differently as the result of having been through many completions.

When a new opportunity opens up at 35, you evaluate it differently than you did at 25. You are able to estimate how long it will take, what the journey will feel like, what the early pleasure and distant pain will feel like, and what getting it done will feel like. You are able to react psychologically to the whole prospect in the form of a narrative that extends beyond the finish line, as a systematic leveling-up of your life. You see the transient pleasures of beginnings diminish to nothing in the far future and the enduring rewards of finishing as a steady source of dividends extending out beyond the horizon."

- http://www.tempobook.com/2014/01/13/when-finishing-is-easier...

3
kadabra9 12 hours ago 2 replies      
You wanna know the saddest part of my day?

When I'm moving around my laptop, and pop into the "projects" folder and see what a graveyard it has become. Dozens of half baked projects that seemed brilliant at the time, that I either lost interest in, decided the concept was too difficult, or (and this is the worst one) let my self doubt convince me that it would never work. The really sad part is, every now and then I'll go back in and check out these projects and a lot of the code and design is actually pretty good. I ask myself, "Why did I think this sucked again?"

The best analogy I can make to this scenario is the self doubt that cripples many writers setting out to finish a book, screenplay, or novel. It's almost as if you tell yourself that the script sucks, to give you a reason not to finish it and move on to something cooler next week. As a writer, what's the best way to overcome this? Just FINISH the god damned first draft. Roll up your sleeves, commit an hour (or two or three) every day to working on this project, and slog through it until you type "FADE OUT" (or "The End") or (if you're coding) make that glorious commit to polish off your project.

No matter how much the project, script or book sucks, there are few feelings of satisfaction that match that.

4
agentultra 12 hours ago 0 replies      
Look at my github and know that I have more repositories on other services and thrice as many on my hard-drive that are unfinished, incomplete duds. I recently reached a milestone: I went from an idea for a book to a self-published, printed title in 3 months. I went to a festival that was mostly focused on comics and video games and sold six copies of my book. It was awesome. Here's the rub: I'm not finished yet.

You're never finished until you stop what you're doing. A writer may "finish" a book but ei has to start another or they've "finished" writing. However for every book they finish how many incomplete, half-baked ideas do you think they've run through? Is every idea they have golden and worth pouring months and years of effort into? No.

Some ideas deserve to die.

But once you've found that one worth pursuing there's nothing to do but roll up your sleeves and put in the time. You will vacillate between euphoria and despair. You may come to regret ever starting and hate yourself. But if it means anything you will force yourself to press on through those darkest moments. And before you know it you'll be done... and ready for the next project. Creativity isn't the rush you feel when you have a good idea and dream about conquering the world: it's process and discipline. It's writing 1500, 2000 words a day no matter what.

I find it helps to have someone to nudge you onward. An editor, a mentor... someone you can discuss the project with who has an objective opinion. They will help you in that moment when you're thinking of giving up.

5
equalarrow 6 hours ago 0 replies      
Wow, great post. Who isn't guilty or fallen prey to this?

The few projects I've finished, they've always been for-pay. Either contracting or as an employee. For my own, almost none.

However. A little over two years ago, I started coding an edu-based app that a friend and I designed (he's the biz side). I was gonna become a father and I thought, what better space to be in since I'm going to be dealing with it in the coming years.

Typical story, coded nights, mornings, weekends. After the baby came, coded less but still the same schedule. We launched the site last fall. And then.. we started having having users saying our site was too complicated, the change log and bug list kept growing. So, this thing I had worked on for so long and pushed into existence by sheer will, just burned me out.

I walked away for a few months and even though we were making almost $1k/mo, I felt it not worth my time anymore. But now, in the past week, I'm changing my tune.

At my day job, we're going thru the final phases of closing on m&a suitors. At first I thought this was awesome, but then looking at the suitors jobs list and reading them, I realized none of them are interesting. Do I want to code day an night? No. Do I want to spend all my day at an office? No. Do I want to help push someone else's dream closer to IPO? No.

I realized after this that I already have the dream (work-wise). We have edu partners lined up, some good potential biz deals, and it's all hinging on just spending a few weeks and fixing things. But having a family, working for something for a long time and not really seeing the reward (yet) - it's hard to keep going thru it all. But, I visualized and thought about "what would it be like to sit at my desk (anywhere I want) and keep making the thing I built better?".

This is the only thing I have ever 'finished' (will it ever be) and I think looking back, I did it all for the right reasons and kept pushing forward. Regardless what anyone else said about it (almost all the responses about the site were positive).

For me, in the end it is about doing what you love, channeling your passion to reach the goal(s). Goal 1 - launching it - reached, done. Goal 2 - helping people enjoy using it - restarted.

Lesson learned: don't give up. Every hour makes a difference.

6
ChuckMcM 6 hours ago 0 replies      
I like to pursue ideas enough to validate if they really are "easy" or "hard." That helps me think about if they are worth pursuing. If they are "easy" I try to figure out what the other people who had this idea got hung up on, if they are "hard" I try to figure out if the hardness is intrinsic to the problem or the approach. I think of this as sort of the 'minimum work' to do on any new idea. Just having an idea and writing it into my notebook doesn't count :-)
7
eddieroger 13 hours ago 1 reply      
I immediately commiserated with this article when I saw the headline, because I have the same problem. I wish the resolution of the blog post was something other than "break the big thing in to little bits and do those" - something that I've known for a long time. For me, the most successful thing I've come across, and coincidentally the hardest part to change, was not being a perfectionist and just getting to MVP. Sometimes you have to say no, and just finish.
8
delinka 11 hours ago 0 replies      
I have a similar process to the author, but I get further. Must have a name! Is it available everywhere? Once I've settled some of that detail, it's on to the code with the basic features.

I'll need a database (and thus a schema), and a REST API, and security considerations ... that's all fine. Start actually writing and testing and OH BOY another feature idea! Write it down for later, continue back where I was. But that new feature will require this change to the current design. And to do that now I have to change this other part and ... repeat until I give up.

tl;dr: feature creep kills me, even with my own ideas. I cannot just Let It Be and produce a 1.0 with minimal features.

9
JelteF 12 hours ago 0 replies      
This is very recognizable. I have had this a lot, but accidentily I had an idea 2 days ago that I have put more time in already than any other of my fun plans or cool projects.

It's called PyLaTeX [1], it's a Python interface for LaTeX that supports creating documents and snippets. One of the coolest features I think is the conversion of NumPy matrices to LaTeX ones.

The HN new page [2] is just a bit rough on someone posting that doesn't know a lot of people that can upvote it, reddit [3] was a lot more forgiving.

[1] https://github.com/JelteF/PyLaTeX

[2] https://news.ycombinator.com/item?id=7075212

[3] http://www.reddit.com/r/programming/comments/1vcqxw/pylatex_...

10
chipsy 5 hours ago 0 replies      
I only have starting problems. Right now my starting problem is the laundry. I abandon a lot of stuff, but only because I've built enough of it to see the flaws. I don't see that as a problem. It looks like a problem if you predicate the goal on external societal factors like "get attention/money," because a thing has to be finished (to some degree) to be part of society. But we already know that the external stuff is a poor motivator.

So stop beating yourself up about finishing. Play in society and worry about things when you feel it's necessary, but if it's your private, creative work, that is the time to be bold and selfish. Don't try to fit in for the sake of it, do things because you want them. You shouldn't care about "finished", because you should be engrossed in the act of creation.

11
whizzkid 12 hours ago 2 replies      
It was the same for me for several ideas/projects that i started with.

Then i realised what was the reason makes me not touching the project after a while...

The first steps are always known and can be done by almost everyone.

- Buy the domain

- Construct the idea

- Even, start coding the project

And here comes the reasons why you slowly starting not to touch the project anymore;

- You realise that you are not sure how to deploy this on production.

- You realise that you will need a business model and you are just a developer/designer and have no idea about those.

- Then comes the tax issues, and realisation that you need a company.

- You don't know how to licence your idea/project

I can list some more but these are just enough to make you feel that you need to find some people to help you which is not free but expensive unless you have friends that are expert in those areas.

Yep, you give up the project...

12
nathan_f77 4 hours ago 0 replies      
I finally managed to finish one! http://www.youshouldbuythese.com

I think the secret to a successful side-project is deciding on an MVP that can be built in a single weekend. I've found that I'm also much more likely to keep iterating on a project if it's already in a 'finished' state.

I seem to run out of steam or lost interest in my more ambitious side-projects, where working for a whole weekend will only get you 10% closer to a finished product.

13
tunesmith 8 hours ago 1 reply      
This might be a bit cogsci, but one thing that has really helped me over the years is to switch my mental language away from statements like "I never finish anything", to statements like "I have had trouble finishing things in the past."

The first is a static judgment I am applying to myself, and it's a definitional straitjacket. The second is simply an observation, and it leaves room and opportunity for positive change.

When we tell ourselves we "are" certain things or "always/never do" certain things, we are defining ourselves in a way that makes it harder to change, due to the reinforcement.

At this point, I pretty much automatically recognize negative "judgments" and do the translation... I think it helps a lot.

14
tlarkworthy 13 hours ago 1 reply      
I like the building things. I finish building them too. Unfortunately, that isn't the job done. You then have to promote it and get users. That is harder than the building stage.

Christ you are in for a shock if you think the immediate step after reserving social media handles is the hard bit...

15
mathattack 5 hours ago 0 replies      
I absolutely love these, "Get off of HN and get some work done" posts that get voted to the top of HN. Are we heroin addicts that know what's bad for us? :-)
16
krrishd 3 hours ago 0 replies      
I think one of the biggest things that causes is losing faith in your idea. No matter how good it is, the more you think about it without doing anything to go along with the thinking, your brain will naturally find minute flaws in the idea, making you move on.
17
Edmond 11 hours ago 0 replies      
It could have something to do with the reason you wanted to start a project in the first place.

If you start a project just to play with some new shinny framework/tech then it is likely that once the novelty wears off nothing of interest would remain.

If however you start a project because you are excited about some product vision that doesn't already exist or not in the form you've envision then the drive to bring that idea/vision to fruition can be a powerful motivator.

I wrote a blog post on this matter some time back:http://colabopad.blogspot.com/2009/12/on-joys-of-creativity-...

18
jjoe 8 hours ago 0 replies      
Sadly this is why employers and early investors give preference to Ivy League graduates. That's likely because the programs ensure graduates are most likely to push the bar higher and achieve. But those who make the exception list (achievers non-league) turn out to be even stronger achievers because their determination comes from deep within rather than from training.

Disclosure: I'm not a leaguer.

19
Kiro 13 hours ago 0 replies      
> check the availability of urls and social media accounts

Are people really checking the availability on social media? I know that if I get the .com I want nothing else matters and the social media handles can be anything.

I know the feeling anyway! The initial excitement and the downfall.

20
mbrock 12 hours ago 1 reply      
Nobody ever finishes anything. Have you ever heard about a finished project? Linus Torvalds hasn't "finished" Linux yet. But it's certainly out there it's alive!

I don't think "man, I'm so far from being finished." That's just a demotivating way to see your project. The big milestone is having something that's useful enough for people to be interested in, no?

21
easy_rider 10 hours ago 0 replies      
I'm going to finish my beer right now.
22
hawkharris 12 hours ago 0 replies      
Great ideas are like juggling clubs. You can keep two or three of them in flight if your coordination is good enough. Add more than that to your routine and you'll probably get smacked in the face.
23
smoyer 11 hours ago 0 replies      
I agree with advice that you should document your idea in your notebook, but don't put any immediate effort into implementing it. The ideas you keep coming back to are the ones that you're truly interested in ... and yet you still have to be careful that those are viable ideas.

In any case, rushing after each new idea is a great way to spend time, but you need that time to be executing on the few ideas you actually choose to pursue.

24
billnguyen 7 hours ago 0 replies      
I struggle with this too, my github/bitbucket is a barren wasteland of half finished products. I believe that its fully a mindset thing and realize that success is not an accident nor some ephemeral spark of genius. Success is a a choice, every day.

I find this YT video on Steph Curry to be an amazing story of how success is built by they choices we make every day.

http://www.youtube.com/watch?v=riy59ubGJiU

'Are the habits you have for today on par with the dreams you have for tomorrow?' Not yet... but its getting there.

25
standup75 12 hours ago 0 replies      
I experienced this. I have a lot of started projects on my mac. Some i spent a few days on, other i spent a few months. In 2011 I started a game and that was the first project I finished (spiderdash.net). Although not really finished, but close enough. It took me over a year, and I really like it. What I realized, is that I do love the execution actually, but I am too unsure of the potential to focus on anything else, that is sales and marketing. So in the past 6 months I started building the ideas of my friends, I get them engaged because it's their ideas, and I get to do just the execution with the right amount of freedom. If you're like me, do things, but do it with someone else. The other good part about this is that you're going to create a unique mix of competencies. My friends are not developers, but they also are subject matter experts. So we are mixing 2 very different kind of expertise, and that's rich.
26
rajbala 11 hours ago 0 replies      
I no longer care about what people think. I care that I may not get enough people to think anything at all.
27
jlwarren1 8 hours ago 1 reply      
I started to read this article, but I gave up about half way through.
28
iterable 6 hours ago 0 replies      
It's why investors put so much weight on the team. Execution is 90%. We've all heard this endless times. But it really hits you in the face when you actually do a startup. A team that can't execute will probably f*ck up a great idea, whereas a team that can execute can do wonders with a mediocre idea
29
bartl 7 hours ago 0 replies      
It's the "doing things" that I have least trouble with. It's the other things, the things that he does first, that I have trouble with.
30
ipetepete 10 hours ago 1 reply      
I read somewhere recently that if you fantasize/talk about your ideas/goals it triggers your brain to let you feel a similar satisfaction for actually accomplishing the said idea/goal. Of course now I can't find the article so take it with a grain of salt.

I did find this article which is related

http://blog.bufferapp.com/how-our-brains-stop-us-achieving-o...

32
marsay 10 hours ago 0 replies      
This article gave me idea for a new project. Yes, it's all clear in my head and I have to start working on it right now and abandon all other projects.

A site where you take responsibility for finishing your project. If you don't, you will pay heavy price. Lets say we will spam your inbox with a thousands of letters that remind you of your promises.

33
the_cat_kittles 10 hours ago 0 replies      
I think "finishing" something means accepting its flaws (which will always be there) ...that makes it psychologically difficult.
34
duochrome 12 hours ago 0 replies      
I've finished something before. It pays back somehow.

But now I barely working on anything. I admit I don't like any work at all. Working for Google or SpaceX? No.

I think we want to finish something because we are not satisfied with our current life. If you feel your current status is okay, it's not easy to get motived to put yourself into some extra work.

35
brennanm 12 hours ago 1 reply      
Same way. I'm a 30%'er. I'll do the name, branding, front end mockups... but then I'll lose it. I wont want to dive into the back and write any backend or server code.

That's why you need a team. Everyone has optimistic days and pessimistic days. On a good team you all wont have them at exactly the same time. You're team will push you through and help you finish.

36
Gaurav322 12 hours ago 0 replies      
It is really motivating article and today, I am going to generate a best marketing strategy for tumblr and try to execute that. (only one as you say)...
37
melling 11 hours ago 0 replies      
"Real artists ship."
38
hiccup 8 hours ago 0 replies      
Is it awful that I didn't finish reading his post?
30
Value is created by doing samaltman.com
217 points by lpolovets  1 day ago   103 comments top 35
1
natural219 1 day ago 14 replies      
All advice is a two-way street. I think at this point, the proverbial "advice market" is saturated with this idea that "doing" is far superior to "thinking". This is because, traditionally, hackers tend to lie squarely on the "thinking too much" end of the spectrum.

I'm interested to read at least one post on the opposite advice -- what happens when you're too heavy on the "doing" end of the spectrum? Are there examples of startups failing because of too much emphasis of execution? At the very least, the "ideas are nothing, execution is everything" meme is clearly logically flawed. You can execute as fast as possible while creating zero customer value.

2
minimax 1 day ago 2 replies      
Writing software no one wants does not create valuethats called a class project.

My corollary to that is that if you write software that does create value, it doesn't matter what editor or language you use. So if you're super duper productive in Perl (or whatever) don't worry too much about chasing after whatever language HN is fetishizing at the moment. It will change in a few months anyway.

3
zoba 1 day ago 1 reply      
I agree, although this makes me think of the great number of people who have the potential to create value but cannot, because their ability to "do" is blocked. Most people I know who cannot create value have an issue of time. Most everyone is locked into their job with rent payments, student loans, and other debts. Its frustrating to see that those with wealth can also purchase time for themselves to use to get more wealth, while the majority of people are strapped in for the ride.

Of course non-wealthy people can build things in the evenings and weekends, but, then they begin dealing with issues of stress and burnout. I'm glad YC helps to give folks a chance. I do think one of the less talked about benefits of a strong middle class and even wealth distribution is people have time/ability to innovate on their own.

4
theoh 1 day ago 5 replies      
Seriously, this _is_ the labor theory of value. Don't know why the other comment to point this out is dead.http://en.wikipedia.org/wiki/Labor_theory_of_value

It's completely inadequate. See, for examplehttp://www.ted.com/talks/rory_sutherland_life_lessons_from_a...

5
jmathai 1 day ago 4 replies      
I realize this is pointed out in the original post but it's important to state for those of us who only read comments and headlines. The longer I do this the more I believe that value isn't created by doing. It's created by selling.

As hackers and engineers the idea of "creating value by doing" resonates with us. We're happy to hole up in a dark room and create. It feels great. Wish I could do that for the rest of my life.

The reality is if you're selling something then that's the only validation you should be looking at to know if you're creating value.

I say this from the perspective of starting up where the point is more easily defined as creating direct economical value.

6
lquist 1 day ago 2 replies      
Value gets created when a company does things like build widgets and sell them to customers. As a rough guideline, its good to stay in roles where youre close to the doing.

Its easier to sit around and talk about building a startup than it is to actually start a startup. And its fun to talk about. But over time, the difference between fun and fulfilling becomes clear. Doing things is really hardits why, for example, you can generally tell people what youre working on without NDAs, and most patents never matter. The value, and the difficulty, comes from execution.

A corollary of this seems to be that investors (such as YC) provide little value because they don't execute. This seems incorrect on its face; most YC founders find the experience to be very beneficial. Maybe I'm missing your point, Sam?

7
abiekatz 1 day ago 1 reply      
What work is for an investor and an entrepreneur are quite different.

As PG says entrepreneurs should "Live in the future, then build whats missing."

Investors allocate capital to what will be more valuable in the future: whether it is something that is missing now or whether it is something that will simply continue to grow in value.

Venture Capital is an interesting middle ground because not only do you allocate capital but you also assist the companies in creating value and you have to compete to be able to invest in the most promising companies. By blogging well you are building your brand and this will help you meet great entrepreneurs and increase your ability to have access to invest in their companies. Of course your reputation will mainly be based on the track record of your investments and how much you help the founders you work with but writing well and having it consistently on top of hacker news helps.

Plus, your writing helps clarify your thoughts and provides you with useful feedback which can help you refine your investment thesis.

Warren Buffett has said he spends 80% of his days reading and 20% talking on the phone. He only has needed one good idea per year to be the best investor of all time. His schedule doesn't sound like work to most people but it clearly has worked well.

8
j_baker 1 day ago 0 replies      
You know, coffee and chat and conferences and whatnot are to the startup world as meetings and hierarchy and process are to big companies. They're both excuses not to get anything done.
9
b1daly 17 hours ago 0 replies      
Why does this truism get endlessly repeated? Is this news to anybody that if you don't make something that people will pay for, your business won't make money? In any case, it's so simple minded that I think it's wrong. It seems obvious to me that a successful enterprise needs the whole system to work: ideas (for a product), all the different kind of makers, sales, marketing, legal, various managers. Or at least these tasks need to be done.

Seeing big companies with money to blow on a product that no one wants take it all the way to market is sort of incredible to me.

Guess I'm just grumpy right now, but I sometimes feel like these posts are jokes put out there by successful people to see if they can make non-successful people feel stupid.

10
tomasien 1 day ago 0 replies      
I was hoping there would be 0 comments on this - that would have been people paying attention!
11
keithba 1 day ago 0 replies      
>> you should try to work on what you really care about

So Good They Can't Ignore You[1] calls this the Passion Hypothesis[2], and argues (very well) that this is the wrong way to think about finding a career.

Instead, create a craftsman-like mentality and work ethic, and then use deliberate practice to get very, very good skills. With great skills, you will enjoy your work much more.

I believe this advice aligns with the rest of the blog entry very well. Creating value (and doing it well) requires an advanced skill set.

[1] http://www.amazon.com/Good-They-Cant-Ignore-You/dp/145550912...

[2] This is an arguable point, since caring about something could be different than being passionate about it.

12
CapitalistCartr 1 day ago 1 reply      
In business, money is how we keep score. Like in a game, the score doesn't lie. i do not mean to imply that money is the ultimate why: money is rarely why we do what we do. But in this game of business, it is the score.

If you're not making money by doing it, its a hobby. There is nothing wrong with hobbies, but they aren't business. Want to know if you are heading in the right direction? Are you headed towards money positive? That's your answer.

13
Patrick_Devine 1 day ago 0 replies      
I think there is a little bit of "black and white" fallacy happening here. There isn't an "either/or" when it comes to an idea and executing. An idea without execution is nothing, but executing without an idea is also nothing. Successful people have an idea, and they execute.
14
flyosity 1 day ago 0 replies      
This reminds me of when I see friends in the industry go mostly radio silent for weeks on Twitter or Facebook, only posting once or twice in a timespan they used to post 20 times or more. It usually means they're busy doing and that I need to get busy doing as well, not surfing and reading.
15
commanda 1 day ago 0 replies      
This definition of value ignores ancillary departments such as marketing, office management, HR, and legal. These are sometimes known as "cost centers" in a business organization, but in that they support the engineers/artists/makers, they most definitely are creating value. Sometimes the chain can be long, but it's usually present.
16
erikpukinskis 1 day ago 0 replies      
The fact that the notion that "we need to go to space" is a foregone conclusion at the dinner parties this person goes to is exactly the reason I avoid San Francisco.
17
mmsimanga 1 day ago 0 replies      
Trouble is not much publicity is given to the boring "doing" stuff that successful people go through. We are shown the successful people having Q&A sessions on Slashdot or being the keynote speaker at a conference. So you have to constantly remind yourself that, it is the result of the work they put before they "made it". So you need to put in your work. Thanks for this timely reminder
18
mkramlich 1 day ago 0 replies      
> Another example of not-quite-work is every night in San Francisco, there are dinner parties where people get together and talk about the future. Its always fun and usually not very contentiousmost people agree we need to go to space, for example. But at the end of it, everyone goes home and works on something else.

except for Elon Musk and team at SpaceX

19
ForHackernews 1 day ago 1 reply      
> As a rough guideline, its good to stay in roles where youre close to the doing.

Why? It seems to me like the further you are from the doing, the more likely your role is to be prestigious and highly paid.

20
beat 1 day ago 0 replies      
A minor quibble here... while value is created by doing, value doesn't necessarily reflect in something that can be sold to other people for money.

"Writing software no one wants" can create value if the person writing it wants it. Just not monetary value.

21
arithma 20 hours ago 0 replies      
It was quite surreal reading this article right after reading Betrand Russel's "In Praise of Idleness."But I agree with the author, the worst idlers are the ones who think are working.

http://www.zpub.com/notes/idle.html

22
jqm 21 hours ago 1 reply      
I've never agreed with the idea that labor is the "sole" or even "primary", source of value.

For example, I find nothing more valuable than clean air and water.

Off topic, but I believe this should be the basis for funding society rather than appropriating a portion of people's productive labor.

Land is valuable. But you don't really "own" it because no one made it. Likewise the Aluminum in a can or the steel in your car. You just borrow them for a time. When we are all dead and gone these things will still be here and people will likely use them.

Society should charge for the use of what really belongs to everyone instead of this part time modern indentured servant hood called payroll taxes. It seems much more just.

As a side benefit... consumption is discouraged and production rewarded. What better for society than that?

23
Q_E_D 1 day ago 0 replies      
There seems to be an optimal mix between "work" eg:designing and "pseudo-work" like having meetings to discuss ideas. Which of those activities will lead to a better design is unclear, since much of the "deep thinking" which is the basis of coming up with something original happens outside of consciously focused thought eg: many writers only write 4 hours a day and do something else the rest of the day, or the common belief that many problems are solved in your sleep and "realized" in the shower.

So, lets not go in either direction for any one person; and let everyone optimize towards their own mix of work and pseudo-work to come to their own original ideas about X.

Some people will not have any original ideas and will therefore advocate work as a method to "focus" on derivative outcomes that make money.

Other people will "find inspiration" while taking a bath and run out of the bathroom to write down the genuinely valuable result and advise everyone to do the same.

These two groups are very different and most of us are somewhere in between.

24
toddh 1 day ago 0 replies      
Value is never created, it is a human construct assigned by humans to something for a variety of reasons. It isn't tied to work or any type of currency. So "commenting on HN, tweeting, reading about other companies" can indeed have value if humans deem it having a value.
25
cynusx 1 day ago 0 replies      
I have not have noticed that Sam Altman was bad at writing, most of the posts that made it to HN are pretty solid. :)
26
codingtheone 21 hours ago 0 replies      
I feel there are two kinds of people, those who have the ability to sit down and talk about things without feeling the guilt of not doing it and those who feel I posters for just talking all the time. There are people I know who are talking about the same ideas for more than a decade without doing anything about it, yet they can come back the next day and talk with all the excitement and ambition in the world as if they're oblivious to the fact that they won't do squat about it
27
beersigns 1 day ago 0 replies      
This was a pretty thought provoking read for me; it put into words some of the beliefs I have operated by. However it's also important to remember that to be 'doing' you need to have a clear picture of why you're doing what you're doing. A lot of people I've met(and myself at times) do not know what they want to do; they've only learned about things they don't want to do in their experiences. I do agree it's best to gravitate towards projects that truly interest you though; if you don't believe in what you're doing you risk drifting into complacency.
28
rhizome 1 day ago 0 replies      
OT: can someone explain why some bloggers do not include (or as here, minimize) the date of writing the post? It strikes me as a conceit that their writing will remain timeless.
29
izietto 16 hours ago 0 replies      
At first I read "Valve is created by doing" :-$
30
iterable 1 day ago 0 replies      
Couldn't agree with you more. And I think the effervescence of the Silicon Valley tech scene creates tons of events, roles and initiatives that are a complete waste of time (For example: some meetups, happy hours and incubators of incubators of incubators). The value is created by making and selling things!
31
sarojt 15 hours ago 0 replies      
I dont really agree with this, there is always value in mentoring the creators rather than being a creator yourself.
32
mindcrime 11 hours ago 0 replies      
If you cant figure out to raise hundreds of millions of dollars, go work for SpaceX (joining a great company is a much better plan than starting a mediocre one).

I love Altman's stuff, but I'm not so sure I agree with "joining a great company is a much better plan than starting a mediocre one". Why? Because no matter how great a company is, if you're an employee, you're still just an employee. You have a "boss" (OK, maybe, just maybe Valve aside), somebody who has you "under their thumb" and who can boss you around and redirect your energy and time, and/or fire you at a whim. And it doesn't matter how great your boss is, or how much you like him/her, you still have "a boss". That sucks. It sucks major donkey balls.

Well, it does for certain kinds of people anyway. It's a mindset thing. I cannot stand having a "boss" in the traditional sense. I'd much rather be running my own show, no matter how mediocre it is (assuming it gets at least to the point of qualifying as a "lifestyle business" and I can pay myself enough to pay the rent).

And yeah, yeah, I know that "you always have a boss" in a sense. Pedants don't bother replying to this. I'm not talking in metaphorical senses or generalities here. A Board of Directors, or "the market" or "your customers" are your "boss" in a metaphorical sense, but that's not the same thing as having one discrete person who can come into the room and go "Sooooo, Peter, you DID get the memo about how we're putting the NEW cover sheets on the TPS reports now, riiiiight? Yeeaaaaaaaaah" and yank your chain.

33
film42 1 day ago 1 reply      
Value is created by caring.
34
jason_wang 1 day ago 0 replies      
set noprocrast = yes;
35
ffrryuu 1 day ago 0 replies      
But the Fed can just prints money and out compete you.
       cached 18 January 2014 03:11:02 GMT