hacker news with inline top comments    .. more ..    15 Dec 2015 Best
home   ask   best   3 years ago   
Introducing OpenAI openai.com
1094 points by sama  3 days ago   369 comments top 54
vonnik 3 days ago 10 replies      
> Musk: I think the best defense against the misuse of AI is to empower as many people as possible to have AI. If everyone has AI powers, then theres not any one person or a small set of individuals who can have AI superpower.

In a sense, we have no other defense. AI is just math and code, and I know of no way to distinguish good linear algebra from evil linear algebra.

The barriers to putting that math and code together for AI, at least physically, are only slightly higher than writing "Hello World." Certainly much lower than other possible existential threats, like nuclear weapons. Two people in a basement might make significant advances in AI research. So from the start, AI appears to be impossible to regulate. If an AGI is possible, then it is inevitable.

I happen to support the widespread use of AI, and see many potential benefits. (Disclosure: I'm part of an AI startup: http://www.skymind.io) Thinking about AI is the cocaine of technologists; i.e. it makes them needlessly paranoid.

But if I adopt Elon's caution toward the technology, then I'm not sure if I agree with his reasoning.

If he believes in the potential harm of AI, then supporting its widespread use doesn't seem logical. If you take the quote above, and substitute the word "guns" for "AI", you basically have the NRA, and the NRA is not making the world a safer place.

karmacondon 3 days ago 3 replies      
This is a key takeaway: "...we are going to ask YC companies to make whatever data they are comfortable making available to OpenAI. And Elon is also going to figure out what data Tesla and Space X can share."

Money is great, openness is great, big name researchers are also a huge plus. But data data data, that could turn out to be very valuable. I don't know if Sam meant that YC companies would be encouraged to contribute data openly, as in making potentially valuable business assets available to the public, or that the data would be available to the OpenAI Fellows (or whatever they're called). Either way, it could be a huge gain for research and development.

I know that I don't get a wish list here, but if I did it would be nice to see OpenAI encourage the following from its researchers:

1) All publications should include code and data whenever possible. Things like gitxiv are helping, but this is far from being an AI community standard

2) Encourage people to try to surpass benchmarks established by their published research, when possible. Many modern ML papers play with results and parameters until they can show that their new method out performs every other method. It would be great to see an institution say "Here's the best our method can do on dataset X, can you beat it and how?"

3) Sponsor competitions frequently. The Netflix Prize was a huge learning experience for a lot of people, and continues to be a valuable educational resource. We need more of that

4) Try to encourage a diversity of backgrounds. IF they choose to sponsor competitions, it would be cool if they let winners or those who performed well join OpenAI as researchers at least for awhile, even if they don't have PhDs in computer science

The "evil" AI and safety stuff is just science fiction, but whatever. Hopefully they will be able to use their resources and position to move the state of AI forward

rl3 3 days ago 0 replies      
>Musk: I think the best defense against the misuse of AI is to empower as many people as possible to have AI. If everyone has AI powers, then theres not any one person or a small set of individuals who can have AI superpower.

This is essentially Ray Kurzweil's argument. Surprising to see both Musk and Altman buy into it.

If the underlying algorithms used to construct AGI turn out to be easily scalable, then the realization of a dominant superintelligent agent is simply a matter of who arrives first with sufficient resources. In Bostrom's Superintelligence, a multipolar scenario was discussed, but treated as unkikely due to the way first-arrival and scaling dynamics work.

In other words, augmenting everyone's capability or intelligence doesn't necessarily preclude the creation of a dominant superintelligent agent. On the contrary, if there's any bad or insufficiently careful actors attempting to construct a superintelligence, it's safe to assume they'll be taking advantage of the same AI augments everyone else has, thus rendering the dynamic not much different from today (i.e. a somewhat equalif not more equalplaying field).

I would argue that in the context of AGI, an equal playing field is actually undesirable. For example, if we were discussing nuclear weapons, I don't think anyone would be arguing that open-source schematics is a great idea. Musk himself has previously stated that [AGI] is "potentially more dangerous than nukes"and I tend to agreeit's just that we do not know the resource or material requirements yet. Fortunately with nuclear weapons, they at least require highly enriched materials, which render them mostly out of reach to anyone but nation states.

To be clear, I think the concept of opening up normal AI research is fantastic, it's just that it falls apart when viewed in context of AGI safety.

zxcvvcxz 3 days ago 7 replies      
Man I dunno about some of this media hype surrounding the topic of AI. I understand how powerful ML/AI algorithms are for general pattern matching (with a big enough computer, gradient descent can learn a lot of things...), but this whole skynet/doomsday fear thing seems ridiculous.

I guess the risk is embedding into systems that manage missiles or something. But you don't need sophisticated algorithms for that to be a risk, just irresponsible programmers. And I recon those systems already rely on a ton of software. So as long as we don't build software that tries to "predict where the this drone should strike next", we're probably fine. Actually shit we're probably doing that.. ("this mountanous cave has a 95% feature match with this other cave we bombed recently..."). Fuuuuck that sounds bad. I don't know how OpenAI giving other people AI will help against something like that.

rjvir 3 days ago 3 replies      
> Sam, Greg, Elon, Reid Hoffman, Jessica Livingston, Peter Thiel, Amazon Web Services (AWS), Infosys, and YC Research are donating to support OpenAI. In total, these funders have committed $1 billion

Funny how they just slipped that in at the end

argonaut 3 days ago 4 replies      
I find it a bit disappointing that despite originally stating that YC Research would target underfunded/underserved areas of research, they've decided to fund and dive into one of the most-hyped, well-funded areas of research: deep learning, an area of research where companies are hiring like crazy and even universities are hiring faculty like crazy. I'm reasonably sure all the research scientists had multiple job offers, and most could get faculty offers as well.

Instead of funding areas of research where grad students legitimately struggle to find faculty or even industry research positions in their field, YC Research decided to join the same arms race that companies like Toyota are joining.

_sentient 3 days ago 5 replies      
$1B in committed funding. Just, wow.

Side note: I wonder if the Strong AI argument can benefit from something akin to Pascal's Wager, in that the upside of being right is ~infinite with only a finite downside in the opposing case.

hacker_9 3 days ago 9 replies      
This is about 100 years too early. Seriously why do people think neural networks are the answer to AI? They are proven to be stupid outside of their training data. We have such a long way to go. This fear-mongering is pointless.
vox_mollis 3 days ago 4 replies      
Where does this leave MIRI?

Is Eliezer going to close up shop, collaborate with OpenAI, or compete?

samstave 3 days ago 0 replies      
This is a serious question:

Should there be an update/amendment/qualification to the laws of robotics regarding using AI for something like ubiquitous mass surveillance?

Clearly the amount of human activity online/electronically will only ever increase. At what point are we going to address how AI may be used/may not be used in this regard?

What about when, say, OpenAI accomplishes some great feat of AI -- and this feat falls to the wrong hands "robotistan" or some such future 'evil' empire that uses AI just as 1984 to track and control all citizenry, shouldnt we add a law of robotics that the AI should AT LEAST be required to be self aware enough to know that it is the tool of oppression?

Shouldn't the term "injure" be very very well defined such that an AI can hold true to law #1?

Who is the thought leader in this regard? Anyone?

EDIT: Well, Gee -- Looks like the above is one of the Open Goals of OpenAI:


peter303 3 days ago 1 reply      
Not the first. Back in the 1980s when expert systems were thought to be the way to AI, there was OpenCyc. Its still around.
baconner 3 days ago 5 replies      
"We believe AI should be an extension of individual human wills..."

I realize that today machine learning really is purely a tool, but the idea that ai will and should always be that doesn't sit quite right with me. Ml tech absent of consciousnesses remains a tool and an incredibly useful one, but in the long term you have to ask the question - at what point does an ai transition from a tool to a slave. Seems some time off still but I do wish we'd give it more serious thought before it arrives.

nazgulnarsil 3 days ago 0 replies      
So the idea with differential safety development is that we want to speedup safe AI timelines as much as possible while slowing down unsafe AI timelines as much as possible. I worry that this development isn't great when viewed through this lens. Lets say that DARPA, CAS, and whatever the Russian equivalent all work on closed source AIs. The idea here might be that open source beats closed source by getting cross pollination and better coordination between efforts. The issue is that the government agencies get to crib whatever they want from the open source stuff to bolster their own closed source stuff.
sethbannon 3 days ago 6 replies      
I can't think of another field of research that's simultaneously brought the potential to solve all the world's problems and the potential to end life as we know it. Very appreciative to see so many great minds working on ensuring AI heralds in more of the former, and none of the latter.
colordrops 3 days ago 1 reply      
This reminds me a bit of all the hype around space elevators several years ago. People were talking about it like it was an inevitable achievement in the near future, nearly oblivious to the huge challenges and unsolved problems necessary to make it happen.

I haven't seen anything but very rudimentary single-domain problems solved that point to incremental improvement, so I'm wondering if these billionaire investors are privy to demos the rest of us are not, and thus have real reason to be so cautious.

runevault 3 days ago 0 replies      
So I assume this is one of the projects Sama was talking about in his research initiatives. Sounds promising.
ajtulloch 3 days ago 0 replies      
Congrats! It's a brilliant team, looking forward to great things.
BenjaminTodd 2 days ago 0 replies      
In the spirit of openness, it would be great to see public responses to the downsides of this approach.

In particular, Bostrom Ch5.1 argues that the lead project is more likely than not to get a decisive strategic advantage, leading to a winner-takes-all scenario, which would mean attempts to foster a multipolar scenario (i.e. lots of similarly powerful AGIs rather than one) are unlikely to work.

In Ch11 he explores whether multipolar scenarios are likely to be good or bad, and presents many reasons to think they're going to be bad. So promoting the multipolar approach could be both very hard, and bad.

cdnsteve 3 days ago 2 replies      
AI is a pretty huge field, what area are they going to focus on specifically?
erostrate 2 days ago 0 replies      
Anybody knows if there is any chance of OpenAI sponsoring H1B visas?

I love the idea but being in Europe my options for doing serious AI research outside of academia seem pretty much limited to Google and Facebook.

cr4zy 3 days ago 0 replies      
This is great news! I think distributed access, control, and contribution to the best AI's will help create 'safe' AI's much faster than any AI created in secret. One thing this does not address, and is something that Jerry Kaplan has an excellent suggestion his recent book "Humans need not apply", is the distributed ownership of AI where tax incentives to public companies that have larger numbers of shareholders, encourages wider distribution of the massive gains AI will bring to these companies.

I really hope that the training data, as well as code and research, will be opened up as well, since the public could really benefit from the self-driving car training data Tesla may contribute[1]. By opening up the development of this extremely important application to public contribution and the quality benefits that it brings, we could get safer, quicker realization of this amazingly transformative tech. As of now the best dataset for self-driving cars, KITTI, is extremely small and dated. [plug]I am working on a project to train self-driving car vision via GTAV to help workaround this (please contact me if you're interested), but obviously real-world data will be better in so many ways.

[1] https://medium.com/backchannel/how-elon-musk-and-y-combinato...

BenjaminTodd 2 days ago 0 replies      
If you want to get a job in this area, we wrote a guide: https://80000hours.org/career-guide/top-careers/profiles/art...
mori 3 days ago 1 reply      
What I want to know is whether there's collaboration with MIRI. On safety, especially.
SneakerXZ 2 days ago 0 replies      
I am surprised nobody mentions stupidly smart AI. We can create AI that is capable of self-replicating very fast and fulfilling some goal.

It could start with a noble idea to build a machine to recycle our garbage and use the garbage to build more recycle machines. At the end we can have stupid machines that are perfectly doing their job but because they are capable of replicating and getting better what they do. They determine if they kill human, less garbage is created and thus so less work for them.

At the end they will wipe out us. Because the thing that will kill us doesn't need to be smarter. It needs to be faster and more effective than we are.

jgord 3 days ago 1 reply      
interesting, and I hope they fund some outlier, less established forms of AI.

For example, we may find that massive simulation yields more practical benefits in the medium term than stronger pure AI / ML, in some domains.

By analogy with research on possibly harmful biosystems, one can extrapolate the need for a set of agreed / self imposed safeguards on certain types of strong AI research - eg. make them read-only, not connected to physical actuators, isolated in a lab - just as you would isolate a potentially dangerous pathogen in a medical lab.

OpenAI would be the place to discuss and propose these protocols.

A quote from a future sentient AI - "don't you think its a form of racism, that strong AI abide strictly by the three laws of robotics, but humans do not?"

bholdr 3 days ago 1 reply      
This is really great, I think. At least, I admire the motivation behind it as it was outlined by Sam.

However, it seems, YC Research started by bringing in accomplished and well-known academics in the field. I wonder whether it would've been more appropriate to focus on providing PhD Scholarship and postdoc fellowship. Though, I understand and somewhat appreciate the motivation behind bring the "top-guns" of research into this, I wonder whether bringing passionate and hungry for knowledge early career researchers could've been a better bet. I am bias on this, but overall think it would be great to diversify the group and level the field -- let the randomness of ideas play its role :) Just my 5c.

necessity 2 days ago 0 replies      
>unconstrained by a need to generate financial return

The incentive, not the constraint, provided by financial return is what drives innovation the most, aside from (but not mutually exclusive to) necessity.

RoboTeddy 3 days ago 0 replies      
I hope more great researchers recognize the importance of the mission and take part!
sremani 3 days ago 0 replies      
Did not expect Infosys or Vishal Sikka along with what is mostly SV who's who.
mrdrozdov 1 day ago 1 reply      
Imagine you've programmed a spider-like robot which sole purpose is to maintain some energy level (by plugging into an outlet), gather resources, and create a clone of itself when it has enough resources. How do you defend against something like that?
selfishAIgen 2 days ago 1 reply      
If I develop any advance AI I will use it for my own wellbeing, perhaps to live longer and obtain a higher finantial status and fullfill some of my dreams. Then I would develop a shield to protect myself and the AI from big corporations and to retain the advantage I got. Perhaps I would try to make Mars a paradise to live in my a thousand year old life, and find or design a partner for that long period. Let the machine create the dream.
foobarqux 3 days ago 0 replies      
How is the group structured and operated?
altonzheng 3 days ago 0 replies      
Cool that they have $1 billion pledged. Curious how they will decide compensation, seeing as a lot of these figures would be making a ton in the industry.
viklas 3 days ago 0 replies      
My money (not a billion) is on "Open, Big Learning".

Elon will probably want to build a giga-factory of neurons, then open-source some pre-trained, general model with a free API.

This is a man building electric cars, off-grid industrial-strength batteries, rockets and hyper-loops...I don't think publishing more/better research papers or winning kaggle competitions is the vision.

kumarski 2 days ago 0 replies      
My greatest fears lay well outside the realm of AI.


1 Billion dollars invested in it seems exciting though. Hopefully something epic comes out of it.

spectrum1234 2 days ago 0 replies      
This is awesome.

I was literally just wondering when there will be open sourced AI. I only saw a few repos on github so figured it would be at least 3-10 years. The fact that things like this seem to surface so quick, including recent AI announcements from Google, etc, are a very good signs for AI in the future.

fiatmoney 3 days ago 0 replies      
Will OpenAI be voluntarily subjecting itself to the same regulatory regime for machine learning research Sam Altman proposed earlier, or have they realized that would be a complete disaster?


sianta 2 days ago 0 replies      
OpenAI might be equivalent to an open global market of graph annotated microservices that can recombine automatically (and search as deep as budgeted) towards whatever goal a client will be able to pay for processing. Not sure if that is safer.

With the right microservices available in the market (including business model scripts etc - every service could be an automatically pay per use microservice) automated businesses could be budgeted to search for sustainable market entities models which could reproduce themselves (copy/create microservices should be basic) and evolve in global corporations with a life and objectives of their own. One might need immensely processing budgets to compete/control with such automated corporations.

Digital and/or biological, it seems, we are exactly in this business+market+life+AI game. Curious to learn what happens at the next levels?

dennisgorelik 2 days ago 0 replies      
What problem is OpenAI going to solve?
mark_l_watson 3 days ago 2 replies      
Sounds great. I was hoping for OpenCog to be a good open source AI framework but is is difficult to work with (good team; I have worked with several of them in the past, no criticism intended).

I look forward to seeing how OpenAI uses outside contributions, provides easy to use software and documentation, etc.

CurtMonash 3 days ago 0 replies      
First in with my recent musings as to whether behemoth companies would own the AI space.


richardw 3 days ago 2 replies      
How to prevent Future ISIS from getting Future AI, or do we just shift from us trying to out-think them to our AI trying to out-think their AI?

If the answer to the latter is "resources" then we're back where we started. Whoever has the biggest AI wins.

The picture seems to be of many AI's all keeping each other in check, but that outcome seems less likely to result in the AI-UN and more like a primordial soup of competing AI's out of which a one-eyed AI will eventually emerge.

No matter how human-friendly an AI we build is, competition will be the final arbiter of whichever AI gains the most leverage. If a bad AI (more aggressive, more selfish, more willing to take shortcuts) beats a good AI (limits its actions to consider humanity), we're poked. If any level of AI can invent a more-competitive AI, we're poked. Once the cat's out of the bag, we have zero influence and our starting point and current intent become irrelevant.

nazgulnarsil 3 days ago 0 replies      
I hope there was some consultation with existing AI researchers as this might screw with their funding (willingness of donors etc.). Would not be a good sign if this announcement is about coordination and it failed at that right out of the gate.
jwildeboer 2 days ago 0 replies      
So, um, what is Open about OpenAI? Is it Open Source? Not AFAICS.
roborzoid 2 days ago 0 replies      
What if we put "untouched" limitations to AI, that the AI can never break, as we cannot break certain limitations in a physical world.
dkarapetyan 3 days ago 1 reply      
ultim8k 1 day ago 0 replies      
The future seems to be very interesting on this. I'm very curious.
a-dub 3 days ago 1 reply      
Oh shit. Say goodbye to reasonable g2.8xlarge spot prices...
tangled_zans 3 days ago 1 reply      
Who are the actual staff involved? What sort of things have they worked on and published before?
coderKen 2 days ago 0 replies      
where can we get resources like API and documentation of this cool stuff.
endergen 3 days ago 1 reply      
Isn't this the plot for Avengers: Age of Ultron?
fuzzytop130 2 days ago 0 replies      
nice addition
dopamean 3 days ago 6 replies      
In all seriousness... does "just, wow" communicate something different from "wow?"
negrit 3 days ago 1 reply      
Disappointing to see Infosys associated to this initiative.

EDIT: looks like the infosys brigade is downvoting me to hell.

Stockfighter is live stockfighter.io
545 points by jzig  2 days ago   138 comments top 45
patio11 2 days ago 4 replies      
Hey guys -- the most resource-intensive part of the infrastructure (the game master boxes) is currently overwhelmed by your enthusiasm. We're spinning up more boxes for you, but it will take a few minutes.
a-dub 2 days ago 1 reply      
This is part of the game. To beat this level you sweet talk the founders into letting you help them fix their scalability problems. If you get the site running in 3 hours, you get a bonus score of .
netcraft 2 days ago 2 replies      
I can't get the first level to play, seems things are still overloaded. Really looking forward to this though, hope you will post or email again once things settle to remind us about it.

edit: this is the message im getting trying to play first steps, if it helps, dashboard says "Trades.Exec() (Status: playable!)" (3:08 UTC)

> We couldn't start the level: This level has been locked by a different server. (Sorry if this happened accidentally as a result of a server crash -- it will clear up in an hour.) Refresh page to check if server is down?

sergiotapia 1 day ago 1 reply      
It's seems the site is extremely broken, I can't navigate anywhere and clicking links that have "#" href's don't do anything.

["handleListAllowedLevels", Object]blotter-bundle-dd01d8c9.js:1549 No level specified in URL.blotter-bundle-dd01d8c9.js:1605 ["enterLevel does not think it can reenter level: ", "first_steps", NaN, Object]blotter-bundle-dd01d8c9.js:2602 ["handleEnterLevel -> startLevel", Object]blotter-bundle-dd01d8c9.js:2556 ["Error starting level:", Object]

tptacek 2 days ago 1 reply      
Well, sort of. Erin and I are scrambling on last-minute emulator/compiler bugs, but Patrick is letting people play with his trading exchange levels.
shreyas056 1 day ago 1 reply      
Just out of curiosity, can you post the stats about load on your site and what kind of issues did you face, would appreciate it.
edanm 2 days ago 0 replies      
Congrats on launching!

I'm getting this error when trying to play the first level:"We couldn't start the level: couldn't connect to the GM server to start the level. Refresh page to check if server is down?".

I assume it's just too many people hitting the server at the same time, but fyi.

xfour 2 days ago 0 replies      
Seems to be pretty stressed out at the moment, tons of 500s and timeouts. 2-3 refreshes per page, got to the Level controls dialog, nothing doing.

We couldn't start the level: This level has been locked by a different server. (Sorry if this happened accidentally as a result of a server crash -- it will clear up in an hour.) Refresh page to check if server is down?

gort 2 days ago 4 replies      
Unable to validate my email. Earlier it wasn't working at all, now it tells me I need to be logged in (but I am, I think).

Edit: Fixed for me now.

dfc 2 days ago 1 reply      
Usernames must start with a letter? That is an odd requirement. I usually pad my username with zeroes when sites dont let me choose dfc. Out of curiosity why the constraint?
lackbeard 2 days ago 3 replies      
UPDATE: Can log in now. Cleared browser cookies, but have no idea if that's a coincidence or not.

I seem to have hit some kind of bug signing up:

1. Filled out signup form, hit enter.

 > Page reloaded, sign-up form empty. No other feedback.
2. Filled out signup form again, hit enter.

 > Told username and email address taken.
3. Check email.

 > no new email.
4. Filled out login form using info entered in step 1.

 > Page reloaded, not logged in.

tedmiston 2 days ago 0 replies      
From the welcome email: "You can play with any tech stack which speaks HTTP, or try doing everything by hand (or curl) if you like playing life on hard mode."

Love this.

ohquu 2 days ago 0 replies      
Trying to start the first level:

"Couldn't resume the level because Couldn't connect to the GM server to resume the level."

mason55 2 days ago 1 reply      
I'm on the instructions for level 2 and it points to the API docs (http://starfighters.readme.io/) but that URL is giving me a 404.
thenomad 1 day ago 1 reply      
Congratulations on having the same sort of initial launch problems that other very successful games have!

Seriously. There's nothing worse than preparing for scale and then discovering you don't need it because there's not enough interest. Wheras I'm sure you'll get these initial load problems sorted soon enough!

tux3 2 days ago 1 reply      
Really awesome to see it launch!

It looks like the GM server is down at the moment, so I have yet to start the first level, but I'm more than happy to mash the F5 key till it works.

davnils 1 day ago 0 replies      
You didn't do any load testing before releasing this on HN?
ShaneWilton 2 days ago 0 replies      
Congrats on launching!

If you're looking to dive into the problems more immediately, there's a wealth of API clients and other resources available in the forums: https://discuss.starfighters.io/t/helpful-external-tools/136

Don't be afraid to try the game if you don't have much programming experience. It's seriously a great way to pick up some valuable skills.

yblu 1 day ago 2 replies      
Does anyone know what tool is used to create the stockfighter API doc [1]? Looks really nice.

[1] https://starfighter.readme.io/v1.0/docs

yoodenvranx 2 days ago 3 replies      
Is there any deadline or is this a long-running project? I'd love to play with this but I don't have any time at the moment.
rcsorensen 1 day ago 1 reply      
The handshake agreement presented here makes me very happy.

Were many revisions done on it, or was it pretty clear to write?

chenster 1 day ago 0 replies      
So much reading about what it does. Don't assume people already know how to play a programming game. Most of us don't. Is there an intro video that at least we can watch while you address the server issues?
fchollet 2 days ago 0 replies      
Confusing, still pretty buggy. Maybe it's a bit early to release?

I'll try to compile some constructive feedback in a bit.

paulpauper 1 day ago 0 replies      
We couldn't start the level: This level has been locked by a different server. (Sorry if this happened accidentally as a result of a server crash -- it will clear up in an hour.) Refresh page to check if server is down?
2oi4j3 2 days ago 1 reply      
How does the "finding jobs" part work?Are you aiming to help people get a foot in the door at companies which would be looking to interview them, or trying to skip the "technical interview" part completely?(generate interview opportunities vs job offers)
ertemplin 1 day ago 1 reply      
It seems like your nginx proxy is having trouble... I keep getting the same ip from DNS and nginx error messages. Maybe you should bring up some more and at the very least use DNS to send requests round-robin to the proxies.
blondie9x 2 days ago 0 replies      
I wasn't able to access the levels yet unfortunately. Is there still some testing going on?
div 2 days ago 0 replies      
https://starfighter.readme.io/ top navigation "Stockfighter" link links to staging.
iso8859-1 1 day ago 2 replies      
> Our servers are under continuing heavy load due to having launched recently and the site being on the front page of HN. This is causing poor performance and undefined behavior.

How is that possible? As I understand, undefined behaviour is either there when you compile, or it isn't (it is a function of the source code). So if the undefined behaviour is a function of traffic intensity, I assume you compile different C code base when you have high load. I never heard about something like this before.

onslauth 2 days ago 0 replies      
Well done on getting it out there. Also well done on ensuring you get no time off during the holiday season :D

Will be jamming it in the morning. Hope you guys get some sleep at least.

kuzmin 2 days ago 1 reply      
Wow, I never think I've had a confirmation email arrive so fast.

FWIW, I'm also hitting the rate limit exceeded error when trying to access the "first_steps" level.

airza 2 days ago 1 reply      
can't login with my migrated account. Here's what i get on pw recovery:

"The change you wanted was rejected.

Maybe you tried to change something you didn't have access to."

SBCRec 2 days ago 0 replies      
Woohoo, Congrats on going live. I am excited to play around!
fastball 2 days ago 1 reply      
Are you aware that the API docs have the subdomain "starfighter" on readme.io rather than "stockfighter"? Seems... strange.
samavirya 2 days ago 0 replies      
Awesome, I've been looking forward to this! Signup went smoothly for me and I'll jump into the first level as soon as I can.
ufmace 2 days ago 0 replies      
Cool! Sounds like there's some teething problems right now, so I'll put this on the check it out later list.
DGAP 2 days ago 0 replies      
I'm getting "Level Instance is not running" on the first level.
xfour 2 days ago 0 replies      
Getting a lot of 500 NGINX errors trying to confirm email or login.
ctmkpp 2 days ago 0 replies      
Congratulations!! Really happy for you Patrick
omash 1 day ago 0 replies      
Level 1 complete!
purpleidea 1 day ago 1 reply      
Nice try, HFT company that wants free algorithms.
dbancajas 1 day ago 2 replies      
I am not a devops but seriously curious, wouldn't this scalability problem they are experiencing been solved already by using Amazon EC2 and stuff?
markbnj 1 day ago 0 replies      
Suggestion: deploy your GM server as Docker containers on kubernetes and auto-scale while you enjoy some fine wine in a casual atmosphere. :).

Seriously, good luck. Success hurts sometimes.

rifung 1 day ago 0 replies      
Is this meant for people interested in security? I ask because the main site says you guys are making CTFs to replace interviews, but I'm not sure if that's interviews for security roles or all developers.

I'd be interested in either, but for the moment rather not add yet another thing for me to learn

Philips Hue blocks 3rd party lights home-assistant.io
384 points by cstuder  7 hours ago   214 comments top 28
underbluewaters 1 hour ago 2 replies      
I've been a hue user for the past year.

1) Their lights/bridge are by far the most reliable IoT product I've used. A 9/10 where other products (Wink, GE, Lutron) are at best 3/10.

2) While 3rd party bulbs were sort of supported, it wasn't advertised. I've never seen it described as a interoperable Zigbee Light Link device.

3) My attempts at getting a GE bulb working were inconsistent, and even when it did work the brightness range and responsiveness were worse than the hue bulbs. I had constant disconnections where I had to re-add the bulb as well. I imagine many people blamed philips when in fact the problem was with the cheap 15.00 bulb. This is probably the reason for discontinuing unofficial support.

I don't like the fact that there's not a thriving range of interoperable, cheap, and high quality zigbee light link devices out there. I'm happy the Philips is focused on delivering a product that actually works, however. All of my other home automation purchases have ended with many wasted hours and eventual returns.

mdip 5 hours ago 9 replies      
I'll never understand why companies make such customer hostile decisions. I can't see any upside to this and I can't imagine what the discussion looked like when implementing this decision. My (admittedly cynical) guess is that it was one part "We're fielding a lot of support questions on products that aren't ours and our malfunctioning" and two parts "Our customers will be more motivated to buy Philips Hue products since they're already invested in the existing system".

The first argument is out the door because now they'll just get a lot of support questions around many more products that don't work. The second might happen for existing customers, but I'm now going to avoid this product as a future customer because compatibility is a feature, and for me, and I'd think anyone who is looking to purchase a tool that manages lighting/small electrics around the home, compatibility is easily the #1 or #2 feature. If there were a choice between a lightswitch that worked with a wide array of lightbulbs, and one that worked with only one company's bulbs, guess which one I'm buying?

mschuster91 4 hours ago 0 replies      
Having had the pleasure of developing ZigBee stuff (NDA, so cannot name specifics), I can understand Philips' position of blocking 3rd party stuff.

Zigbee is nasty enough to develop for even if you HAVE all the relevant docs and everything, without adding stuff developed by others into the mix. The only thing why this crap ended upon users is because it's less power hungry than WiFi will be and has a bigger range than Bluetooth.

jcromartie 5 hours ago 4 replies      
Are there grounds for a lawsuit when a manufacturer pushes out an update to a product that intentionally degrades its feature set like this? This is no longer the same product that these users bought, it's effectively damaged by the update.
daveguy 14 minutes ago 0 replies      
Here is the amazon product page for the hue starter pack with the reviews listed -- most recent first. Even if you haven't purchased this yet you can let everyone know these new reviews rolling in about the product are helpful:


dperfect 4 hours ago 2 replies      
This problem isn't unique to Philips Hue, and I'm not sure Philips is necessarily a bad actor here. They are, after all, just acting in their best interest (and contrary to common belief, companies are self-interested; the customer's interests are important only so much as they serve the company's interests - for better or for worse).

Perhaps the problem is more related to the ZigBee standard, or more specifically, what ZigBee doesn't cover. As I understand it, the fact that Hue products conform to the ZigBee standard only applies to the protocol between the hub and lights; it says nothing about communication between control interfaces and the hub.

With that in mind, I feel that anyone who expected to use the Hue hub for controlling anything other than Hue lights had incorrect expectations. I certainly don't remember seeing Philips advertising any kind of third-party interoperability with the product.

Of course, it would be awesome for consumers if Philips did make that guarantee of interoperability, but from a business standpoint, once you start going down that road, you effectively have to support everyone else's products (whether or not they conform to the spec), and any failing of another product then reflects poorly on Philips' brand - even if they aren't the ones at fault.

So maybe if we put more pressure on these companies to adopt a common home automation control standard (not Apple's, not Google's, but something vendor-neutral), then we might start to see home automation interoperability in a consumer-friendly way.

throwaway420 5 hours ago 3 replies      
I was literally about to make a purchase but now I'll be considering other options. Good timing on your part hue.

Do these idiot companies not learn anything from backlashes against Keurig and others?

Animats 4 hours ago 1 reply      
Can you dump Hue and use Wink? Wink can supposedly control both GE and Phillips bulbs.[1] GE uses Wink as their main control system, and GE is big enough to argue with Phillips.

[1] http://www.wink.com/help/products/philips-hue-lighting-start...

someotheracct7 5 hours ago 2 replies      
This is odd, but not catastrophic. If I understand right, Zigbee is open, so what's to stop the bulbs working with a third-party controller hub? Say, a Raspberry Pi with a Zigbee interface?

Wonder if this is still relevant?http://www.everyhue.com/vanilla/discussion/141/getting-hue-t...

That thread suggests it's possible to have Hue bulbs join third-party ZigBee networks.

cubano 3 hours ago 1 reply      
Creating a new compatible hub now sounds like an ideal Kickstarter project, no?

Also, I can't help but wonder if UL issues may have influenced the decision too...just look at the current hoverboard fire issues with cheap Chinese manufacturing and I can at least speculate that they might be.

gregmac 5 hours ago 8 replies      
This is the Philips controller software (app) that is refusing to talk to or control 3rd party light bulbs.

I don't really see this as being a problem. It's not the nicest thing, perhaps, but neither is 3rd party companies building bulbs (presumably by ripping off the design and protocol of Philip's bulbs), and then relying on Philips to build the software so they can actually be used. In other words, Philips is refusing to build software, for free, to control their competitors' and/or counterfeit products.

Do the Philips bulbs (still?) work without the Philips Hue software/app? If so, non-issue: you just have to use 3rd party software to control everything. If not, well, that's why you shouldn't buy into closed, walled-garden products.

ultimoo 3 hours ago 1 reply      
I have been consistently disappointed by Philips Hue ever since I purchased the $200 starter kit a year and a half ago.Numerous iOS app updates have rendered the system useless (lack of basic QA), slow to adopt HomeKit and price gouging for it (they want users to buy an entirely new Hub for $60).
rythie 6 hours ago 4 replies      
Surely with the Pi Zero selling at $5 there has got to be way to do this without Philip's locked in, expensive system?
dmritard96 4 hours ago 1 reply      
We are building some related home control products and have forgone radio level integrations instead favoring integrating with anyone over http, providing an open api, and shortly after launch an sdk. Radio integrations in theory are fine but in practice it's a matter of picking sides, allowing third party devices to dictate your access to the Internet and interop, for a few features, offline (local http should be an option with any hub, nest etc but isnt) and lower latency operation as probably the biggest features. Not controlling your own products' access to the Internet is a scary proposition not to mention range/interference issues with 2.4ghz if you don't have aa critical mass. Building IoTs right now is as much an alignment excercise as a technical one and while it's a bummer that they have chosen to remove/prevent compatibility, consumers aren't really choosing subscription models for devices and cloud backed services hence the economics of customer acquisition, support, margins etc aren't realities that can be wished away.
bradyd 2 hours ago 0 replies      
spdustin 2 hours ago 0 replies      
I have three neighbors, all of whom are non-technical, that use Hue lights in their home. I asked them if they used any other "smart bulbs" with their Hue bridge. The universal reply, averaged out to a representative quote:

"I didn't even think that was a thing, I only use Hue bulbs with the Hue bridge."

endgame 6 hours ago 1 reply      
Defective by design.
bottled_poe 5 hours ago 0 replies      
I'm not convinced this was a hostile decision by Philips. It would be reasonable for the company to want to advance their own technology which would often entail a changing interface. If the third-party products are implementing an older interface, how is that the fault of Philips? The moral thing to do would be to standardise and publicise the interface...perhaps this is the situation already...?
xutopia 6 hours ago 12 replies      
So what would you recommend we use instead?
SEJeff 1 hour ago 0 replies      
In slightly related news, the open source project, Home Assistant, that this blog post references, is an excellent open source "hub" for controlling the smart things.
Roritharr 5 hours ago 1 reply      
What a horrible bait and switch.

Sadly I'm too lazy to go through the hassle of asking for my money back. :(

reiichiroh 4 hours ago 0 replies      
I didn't even know there were 3rd party lights compatible with the Hue ecosystem. Can someone point me to some brands and models?
Someone1234 6 hours ago 1 reply      
I always wondered about smart bulbs... How much, per year, do they cost to run idle? Since they appear to use wireless to operate, so there must be at least the draw for that.
llamataboot 5 hours ago 1 reply      
Damn. I just bought my first Hues a week ago - a little spendy but worth every penny to have a super easy install into my living room lamps and be able to be controlling my lights from a ruby script in 5 minutes. Hopefully, someone will release a crack sometime soon.
djhworld 5 hours ago 2 replies      
Why have they done this? Is it because the 3rd party bulbs were cheaper?
mmaunder 4 hours ago 0 replies      
They're going for vendor lock-in. Someone peered over Apple's garden wall and liked what they saw.
kefka 5 hours ago 1 reply      
I've ranted about this and the IoT and how we've came to it.

IoT is awesome, and super, as long as you don't buy products that require to beg for your data from some website API.

What makes more sense?Local device ->Internet ->Someone else's server(cloud) ->API ->Your cell/computer


Local Device ->Local controller

I extensively use Node-Red and Apache NiFi both. Node-Red is built on Node.js and is a graphical flowchart programming in the browser. It makes chaining together complex interactions easy and also makes many things no-code at all.

I use Node-Red for my home automation. It's open source, it works, and just makes sense.

Now, what should you use for the light bulbs? Unfortunately not Phillips Hue. This however opens up the market for others to come in and snipe business away. And that's a good thing. And whatever takes away the power of "other peoples' servers" cloud-crap, the better. It's just used as a way to control and extract more money.

kevindeasis 4 hours ago 0 replies      
Keurig is that you?
Files Are Hard danluu.com
436 points by pyb  1 day ago   147 comments top 32
notacoward 1 day ago 1 reply      
Pretty much spot on. Local in-kernel file systems are hard, partly because of their essential nature and partly because of their history. A lot of the codebases involved still show their origins on single-core systems and pre-NCQ SATA disks, and the development/testing methods are from the same era. The developers always have time to improve the numbers on some ancient micro-benchmark, but new features often get pushed to the LVM layer (snapshots), languish for ages (unions/overlays), or are simply ignored (anything alternative to the fsync sledgehammer).

The only way a distributed file system such as I work on can provide sane behavior and decent performance to our users is to use local file systems only for course-grain space allocation and caching. Sometimes those magic incantations from ten-year-old LKML posts don't really work, because they were never really tested for more than a couple of simple cases. Other times they have unexpected impacts on performance or space consumption. Usually it's easier and/or safer just to do as much as possible ourselves. Databases - both local and distributed - are in pretty much the same boat.

Some day, I hope, all of this scattered and repeated effort will be combined into a common library that Does All The Right Things (which change over time) and adds features with a common API. It's not quite as good as if the stuff in the kernel had been done right, but I think it's the best we can hope for at this point.

needs 1 day ago 2 replies      
There is an easy way to write data without corruption. First copy your file-to-be-changed as a temporary file or create a temporary file. Then modify the temporary file and write whatever you want in it. Finally, use rename() to atomically replace the old file by the temporary one.

The same logic also apply to directories, although you will have to use links or symlinks to have something really atomic.

It may not work on strangely configured systems, like if your files are spread over different devices over the network (or maybe with NFS). But in those cases you will be able to detect it if you catch errors of rename() and co (and you should catch them of course). So no silver bullet here, but still a good shot.

psi-squared 1 day ago 1 reply      
AIUI, ZFS was explicitly designed to deal with this sort of data corruption - one of the descriptions of the design I've heard is "read() will return either the contents of a previous successful write() or an error". That would (in principle) prevent the file containing "a boo" or "a far" at any point.

It looks like one of the authors cited in this article has written a paper analysing ZFS - though they admittedly don't test its behaviour on crashes. Citation here, in PDF form:


(edited to add: This only deals with the second part of this article. The first part would still be important even on ZFS)

tehwalrus 1 day ago 2 replies      
"How is it that desktop mail clients are less reliable than gmail...?"

Made me chuckle. I've been told off by a former Googler colleague enough times now to have learned that Gmail is more complex than anyone imagines on a first guess, in order to be "reliable".

kentonv 1 day ago 1 reply      
The article's table of filesystem semantics is missing at least one X: Appends on ext4-ordered are not atomic. When you append to a file, the file size (metadata) and content (data) must both be updated. Metadata is flushed every 5 seconds or so, data can sit in cache for more like 30. So the file size update may hit disk 25s before the data does, and if you crash during that time, then on recovery you'll find the data has a bunch of zero bytes appended instead of what you expected.

(I like to test my storage code by running it on top of a network block device and then simulating power failure by cutting the connection randomly. This is one problem I found while doing that.)

cellularmitosis 1 day ago 0 replies      
I've recently been playing with nbdkit, which is basically FUSE but for block devices rather than file systems.

I was shocked to discover that mke2fs doesn't check the return value of its final fsync call. This is compounded by the fact that pwrite calls don't fail across NBD (the writes are cached, so the caller's stack is long gone by the time the get flushed across the network and fails...)

As a test, I created an nbdkit plugin which simply throws away every write. Guess what? mke2fs will happily create a file system on such a block device and not report failure. You only discover a problem when you try to mount the file system.

steven2012 1 day ago 3 replies      
I worked at a storage company and the scariest thing I learned is that your data can be corrupt even though the drive itself says that the data was written correctly. The only way to really be sure is to check your files after writing them that they match. Now whenever I do a backup, I always go through them one more time and do a byte-by-byte comparison before being assured that it's okay.
nickpsecurity 1 day ago 1 reply      
Great write-up and probably explains some issues in my apps a while back. Like that my long-time favorite, XFS, totally kicks ass in the comparisons. I agree on using static analysis and other techniques as I regularly push that here. What threw me is this line:

"that when they came up with threads, locks, and conditional variables at PARC, they thought that they were creating a programming model that anyone could use, but that theres now decades of evidence that they were wrong. Weve accumulated a lot of evidence that humans are very bad at reasoning at these kinds of problems, which are very similar to the problems you have when writing correct code to interact with current filesystems."

There were quite a few ways of organizing systems, including objects and functions, that the old guard came up with. UNIX's popularity and organization style of some others pushed the file-oriented approach from mainstream into outright dominance. However, many of us long argued it was a bad idea and alternatives exist that just need work on the implementation side. We actually saw some of those old ideas put into practice in data warehousing, NoSQL, "the cloud," and so on. Just gotta do more as we don't have a technical reason for dealing with the non-sense in the write-up: just avoiding getting our hands dirty with the replacements.

orf 1 day ago 1 reply      
Surely filesystems are going to go through a massive change when SSDs push standard spinning disks into the history books? They must carry a lot of baggage for dealing with actual spinning disks, much of which is just overhead for super-fast solid state drives. Hopefully this will allow interesting features not possible on spinning disks, like better atomic operations.
vog 1 day ago 5 replies      
I wonder how this approach (single file + log) compares to the other usual approach (write second file, move over first):

1. Write changed the data into a temporary file in the same directory (don't touch the original file)

2. Move new file over old file

Does this lead to a simpler strategy that is easier to reason about, where it is less likely for programmer to get it wrong? At least I see this strategy being applied more often than the "single file + log" approach.

The obvious downside is that this temporarily uses twice the size of the dataset. However, that is usually mitigated by splitting the data into multiple files, and/or applying this only to applications that don't need to store gigabytes in the first place.

cmurf 1 day ago 2 replies      
The article makes me wonder whether there's enough abstraction being done via the VFS layer, because all this fsync business that application developers seem to have to do can be so workload and file system specific. And I think that's asking too much from application developers. You might have to fsync the parent dir? That's annoying.

I wonder if the article and papers its based on account for how the VFS actually behaves, and then if someone wanting to do more research in this area could investigate this accounting for the recent VFS changes. On Linux in particular I think this is a bigger issue because there are so many file systems the user or distro could have picked, totally unbeknownst to and outside the control of the app developer.

quesera 1 day ago 4 replies      
This is an interesting article but the examples given are seriously dated:

Mail servers and clients (MTAs and MUAs) have been using Maildir format to escape this problem since 1996.

Filesystems have evolved. ZFS has been available for ten years.

If you take your data seriously, you do need to make informed choices. But this article isn't targeted at people who won't know about Maildir and ZFS.

The section about relative data loss failures between various applications is great. Again, careful choices: SQLite and Postgres.

alkonaut 1 day ago 1 reply      
The article only mentions Linux/Posix systems, are the same problems also present in Windows/NTFS? I was under the impression that, for example, renames on ntfs were crash safe and atomic, which would make the "write temp file then rename to target" work even if the power is cut?
mtanski 1 day ago 0 replies      
While I experienced this pain first hand I'm not sure that the FS deserved a 100% of the blame. There is enough to blame to go around for the userspace, filesytems, block device layer and disk controllers, hardware & firmware.

If you start at the bottom of the stack you have all sorts of issues with drives, their firmware and controllers doing their own levels of caching & re-ordering. Over the years the upper layer developers (block layer / fs) had to deal with hardware that simply lies about what happened or is just plain buggy.

toolslive 1 day ago 0 replies      
One of the naive assumptions that most of us make is that if there's a power failure none of the data that was fsynced successfully before will be corrupted.

Unfortunately, this is not the case for SSDs.

david-given 1 day ago 2 replies      
One thing I've always wanted to try and never had time is to build SQlite as a kernel module, talking directly to the block device layer, and then implement a POSIX file system on top of it.

It wouldn't solve problems with the block device layer itself, but it'd be interesting to see how robust and/or fast it was in real life.

rikkus 1 day ago 0 replies      
This issue is completely fixed by Maildir and was many years ago. Many clients, including Mutt, for example, support Maildir boxes
stevoski 1 day ago 1 reply      
This article sheds some light on a problem I've had for years:

The software product I develop stores its data in a local database using H2, stored in a single file. Users tend to have databases gigabytes in size.

After reading this article, I start to understand why my users occasionally get corrupted databases. A hard problem to solve.

scotty79 1 day ago 1 reply      
It appears that sqlite could be good basis for decrapified filesystem.
hmottestad 1 day ago 2 replies      
I don't program much in C, or use direct system calls for files. Mostly I use Java.

Does anyone know if any of this applies to Java's IO operations. I'm sure you can force some of this behaviour, but for instance: The flush method on OutputStream, will it ensure proper sync, or is that again dependent on the OS and file system as described in the article for syscalls?

Sami_Lehtinen 1 day ago 0 replies      
I do agree with you, this is my story about the usual stuff:http://www.sami-lehtinen.net/blog/chaos-monkey-bit-me-a-shor...As well as this classic from SQLite3 guys:https://www.sqlite.org/howtocorrupt.html - As you can see, there are multiple ways to ruin your day.
lamontcg 1 day ago 0 replies      
Ah, fond memories of async ext2 corrupting root filesystems beyond recognition... I think we 'invented' 'disposable infrastructure' back in 2002 because the filesystem forced us to... The MegaRAID cards that would eat their configuration along will all your data didn't help either..

Can't remember if we switched to ext3 in ordered or data-journaled mode but it made an immense difference...

SixSigma 1 day ago 1 reply      
In plan9 mailbox files, like many others, are append only.

All files are periodically (default daily) written to a block coalescing worm drive and you can rewind the state of the file system to any date on a per process basis, handy for diffing your codebase etc.

For a while the removal of the "rm" command was considered to underline the permanence of files but the removal of temporary data during the daytime hours turned out to be more pragmatic.

Qantourisc 1 day ago 1 reply      
IIRC I have seen this discussion before. And the answer was do a fsync. But for sakes of performance we want to be able to issue write-barriers to a group of file handles. So we know the commands will be ran in order, and no other order.
achille 22 hours ago 1 reply      
Does anyone know if there's a recording of the usenix talk given? (Referenced slides)
oxplot 1 day ago 1 reply      
Isn't block level duplication/checksumming like RAID supposed to solve this hardware unreliability? I understand that by default RAID is not used on end user desktops.
leephillips 1 day ago 1 reply      
Good article. You have a typo: looks like you're using Pandoc or something similar, and left out the closing parenthesis in the link after [undo log].
derefr 1 day ago 0 replies      
I think it's mostly a "why are you telling us? The author of the article might not ever even visit HN. Email them." reaction.
Animats 1 day ago 0 replies      
This problem can be fixed. We need to rethink file system semantics. Here's an approach:

Files are of one of the following types:

Unit files

For a unit file, the unit of consistency is the entire file. Unit files can be created or replaced, but not modified. Opening a unit file for writing means creating a new file. When the new file is closed successfully, the new version replaces the old version atomically. If anything goes wrong, including a system crash, between create and successful close, including program abort, the old version remains and the new version is deleted. File systems are required to maintain that guarantee.

Opens for read while updating is in progress reference the old version. Thus, all readers always see a consistent version.

They're never modified in place once written. It's easy for a file system to implement unit file semantics. The file system can cache or defer writes. There's no need to journal. The main requirement is that the close operation must block until all writes have committed to disk, then return a success status only if nothing went wrong.

In practice, most files are unit files. Much effort goes into trying to get unit file semantics - ".part" files, elaborate file renaming rituals to try to get an atomic rename (different for each OS and file system), and such. It would be easier to just provide unit file semantics. That's usually what you want.

Log files

Log files are append-only. The unit of consistency is one write. The file system is required to guarantee that, after a program abort or crash, the file will end cleanly at the end of a write. A "fsync" type operation adds the guarantee that the file is consistent to the last write. A log file can be read while being written if opened read-only. Readers can seek, but writers cannot. Append is always at the end of the file, even if multiple processes are writing the same file.

This, of course, is what you want for log files.

Temporary files

Temporary files disappear in a crash. There's no journaling or recovery. Random read/write access is allowed. You're guaranteed that after a crash, they're gone.

Managed files

Managed files are for databases and programs that care about exactly when data is committed. A "write" API is provided which returns a status when the write is accepted, and then makes an asynchronous callback when the write is committed and safely on disk. This allows the database program to know which operations the file system has completed, but doesn't impose an ordering restriction on the file system.

This is what a database implementor needs - solid info about if and when a write has committed. If writes have to be ordered, the database program can wait for the first write to be committed before starting the second one. If something goes wrong after a write request was submitted, the caller gets status info in the callback.

This would be much better than the present situation of trying to decide when a call to "fsync" is necessary. It's less restrictive in terms of synchronization - "fsync" waits for everything to commit, which is often more than is needed just to finish one operation.

This could be retrofitted to POSIX-type systems. If you start with "creat" and "O_CREAT" you get a unit file by default, unless you specify "O_APPEND", in which case you get a log file. Files in /tmp are temporary files. Managed files have to be created with some new flag. Only a small number of programs use managed files, and they usually know who they are.

This would solve most of the problems mentioned in the original post.

throwaway848 1 day ago 9 replies      
PG was quite proud of having just used the file system as a database with Viaweb, claiming that "The Unix file system is pretty good at not losing your data, especially if you put the files on a Netapp." His entire response is worth reading in light of the above article, if only to get a taste of how simultaneously arrogant and clueless PG can be: http://www.paulgraham.com/vwfaq.html

PG and company still think this is a great idea, because the software this forum runs on apparently also does the file system-as-database thing.

No wonder Viaweb was rewritten in C++.

EDIT: to those downvoting, my characterization of PG's response is correct. Its arrogance is undeniable, with its air of "look how smart I am; I see through the marketing hype everyone else falls for," as is its cluelessness, with PG advocating practices that will cause data loss. Viaweb was probably a buggy, unstable mess that needed a rewrite.

repiret 1 day ago 0 replies      
purpled_haze 1 day ago 1 reply      
All of this is great, except the first two sentences:

> I havent used a desktop email client in years. None of them could handle the volume of email I get without at least occasionally corrupting my mailbox.

If I were to get so many emails that it was corrupt my mailbox, I'd first ask myself why, and how to stop that.

Bullshit Startup Experiences totalgarb.tumblr.com
514 points by supjeff  1 day ago   278 comments top 44
ditonal 1 day ago 6 replies      
Feel free to steal mine:


me: the candidate seemed pretty good, even if he made some mistakes.

CEO: a false negative is better than a false positive! Better to reject and err on the side of caution!


CEO: we can't find people! There's a shortage! It's a crisis!


CEO: we only hire the best, smartest people in the world!

me: OK I found one! He wants a high salary.

CEO: welllll, average market salary is only $, but since "we're a startup" we can't even do that....

bflesch 1 day ago 3 replies      
This is a great and depressing read, and as the CTO of a soon-out-of-cash startup I can only say that we receive this bullshit on a founder <-> investor level day in and out.

I've actively thought about creating something like this where you could anonymously tie those stories to the real names of VCs because some of these guys are totally out of their mind.

As a EU-based startup raising a moderate seed round I could tell you two hours about the incestuous and unprofessional VC scene here. People lie into your face, VCs forward confidential information directly to competitors' management, suddenly 5 emails into a conversation people stop from replying for three weeks, etc.

I can tell you that Greg from Boxgroup, New York, has done by far more leads and intros for our shitty EU-based startup than any of those Mr. fancypants German mega-VCs (or so they see themselves).

We have had several cases where we sent the phone # of a very rich business angel to our VCs after they asked for an intro to him in order to take part in (his, kind of) round. They not once called him.

So depressing, but yet so absurd that you can't keep from laughing manically for what kind of high-paid job people are doing there sometimes.

sijoe 1 day ago 6 replies      
Been thinking about doing the customer version of this.

[after a long indepth technical discussion with a qualified opportunity, quoting, resources allocated, ready to pull the trigger on a large project]

me: So we are ready to go, waiting on the PO

them: Sure, just working through our process

(falls off internets, doesn't respond to phone calls/emails/rejects visits)

Months go by

them: We set up an RFP and bought from someone else.

me: Er ... you used our confidential material (marked as such) to set up a public RFP, that you didn't even invite us to, or respond to our queries? Seriously?

them: Yes.

[bangs head on table, but it gets better ... no ... the other thing]

them: the system we bought (not yours), isn't performing nearly as well as your system that we POCed on.

me: So? Why not call your vendor and have them help you?

them: they aren't able to. They don't know this stuff as well as you do.

me: So, let me get this right ... you want me to provide consulting services to help a competitor of ours ... compete ... with us?

them: Not so much consulting, as free advice and guidance. Like you did during the POC.

[resumes banging head on table]

them: hello? whats that noise?

[years pass]

them: we need to rethink our strategy, it was an order of magnitude slower than your unit at about 2-3x the cost.

me: no kidding Sherlock

them: we want to start this process again ....

me: (fits of laughter) uh ... no.

Roritharr 1 day ago 3 replies      
My favourite:

Founder Institute

- run by international group of millionaire magazine-cover entrepreneurs

- doesnt invest a penny in its companies

- charges founders thousands of dollars for mentorship

- well-regarded startup accelerator


I was once offered the position to head the local chapter, after having a private phonecall with Adeo, I declined as I couldn't see the value offered. The next person took it, and as expected, the local chapter failed spectacularly and the people founding it were out of a lot of work and didn't get anything in return.

It's still a mistery to me why that organization has credibility.

leroy_masochist 1 day ago 9 replies      
> me: hi i'm here about the lead developer position

> CEO: cool man lets grab a couple beers

> me: (over beers) hi i'm here for the lead developer position

> CEO: yeah dude but are you any good at foosball?

> me: (over foosball) i'm here for the lead developer position

This is surprisingly common and not just for hiring technical roles. I find it especially odd when founders do it in the process of hiring their first few employees....it's like dude, you don't even have a culture yet, why do you care about culture fit?

It would be impossible to quantify but I wonder how many good ideas tackled by good founders never got off the ground because they were overly fixated on getting the first 10 hires EXACTLY right.

rcurry 1 day ago 1 reply      
Startup: We need an experienced Foo programmer.

Me: Those are easy to find.

Startup: Our lead developer keeps shooting them down because they don't know (obscure keyword).

Me: How long would it take an experienced programmer to learn (obscure keyword)?

Startup: I don't know, maybe an hour?

Me: How long have you been hiring for this position?

Startup: About six months.

Me: Would it help if I sent you a transcript of this call?

Startup: What?

Me: Never mind...

adeadb2bcompany 1 day ago 3 replies      
Hi, so this is all happening at the startup I work at:


CEO: We just closed a XX million dollar Series C funding round!

-- later at an a company wide meeting ---

CEO: we have zero revenue. if we asked our customers to pay us, they would drop our product in a second... so, uh, we really don't have product market fit yet.

me: ...

CEO: no worries. we are attacking such a large market. surely we'll find something. and when we do it will be HUGE... it'll be... it'll be... it's so huge I'm having trouble even describing it!


-- at an important weekly product meeting --

CEO: okay it's been ten minutes. where is [cofounder]

CTO: oh he's in china for the week.

CEO: ...

CTO: On vacation.

everyone else: ... (your cofounder didn't tell you that?)


me: your company name "cadabra" sounds like cadaver.

ceo: nah, dude.

me: you had to spell it out when I asked you to pronounce it.

ceo: nah, we just need to hire more marketing people.


wannabe designer: so tell me how you approach design.

me: well I really like being apart of the process from start to finish, since I've also done UX wor--

designer: great, let me tell you how we do design here. i draw up the stuff and you do it, unless there's a technical problem. you can argue and you will be wrong, ha ha.

me: you're joking right?

designer: * proceeds to create barely passable UIs *

And the list goes on. I have plenty of more stories to share but I am currently busy seeking employment elsewhere.

Shameless plug:If you're looking for an American android developer (10,000+ install base on play store), or a front end web developer ( Tech Giant), a back python/java developer ( past startups ), let's talk shop at snarkyhackernewsuser@gmail.com.

ninkendo 1 day ago 2 replies      
--me: sits through yet another mandatory hours-long meeting where the loudest two developers one-up each other to find the best hypothetical solution to the most convoluted problem they can imagine as it relates to the urgent, critical problem we currently face whose simple, plain solution goes entirely ignored and where no decision is made.--

I have trouble laughing at what I know is supposed to be humor, because of how tragically true this is and how much this affects me on a regular basis.

But I don't work for a startup! I thought it was supposed to be different in that world.

Perhaps some problems are just universal.

bsdpython 1 day ago 3 replies      
A lot of these anecdotes are the frustration with dealing with people that waste your time. I guess it's not limited to just the startup space but one thing I've learned is that there is an unlimited number of people that are willing to waste your time and boy is it frustrating.
tdaltonc 1 day ago 4 replies      
> married hetero male founders: we're thinking of getting an intern

> me: okay but I don't see what we could offer. You know we're on a really tight

> married hetero male founders: we're getting an intern.

> intern: (beautiful 18yo girl)

Young male intern --> Everyone assumes you're a Wunderkind.

Young female intern --> Everyone assumes that the CEO is having sex with you.


tonecluster 1 day ago 0 replies      
So many... so many.

me: (up all night fixing bugs in production due to technical debt and mistakes from overworked and tired devs. Sleep from 5am to 9am, show up to office at 10am)

CEO (sales background), walking by as he sees me enter the office: "Hey tone, I want butts in seats, including yours. We have a culture of hard work here. Set an example."


CTO: "We don't need to hire expensive devs. Just hire from overseas and promise them an H1B. They'll work for the minimum."

me: "That's not only unsustainable, it's unethical. H1B is a lottery, we can't promise that!"

CTO: "I don't expect them to know that, and you're not going to tell them! Get them on-boarded and working a.s.a.p."

me: (goes home, writes resignation email)

sohcahtoa 1 day ago 3 replies      
Interviewer: We need someone with [describes soft skill set for my dream job]

Me: Great career opportunity, I'd certainly be willing to move across country for this

Interviewer, day 1: Here's your desk, now I'm going to assign you a ticket off the backlog. Let me know when you're done and I'll assign you another one. [pats me on the head and walks away]

Bait'n'switch, and I fell for it. I think it's the only way they can get people to come to this no-good podunk town, with not a sushi bar for miles, and 19th century development practices. And yes, I am preparing my awesome flameout exit. I hope to hear the lamentation of their womenfolk.

nlh 1 day ago 2 replies      
A lot of these appear to stem from inexperience -- the founder/CEO/CTO doing and saying what they think they're supposed to do or because "that's what successful people do."

I'm going to be slightly reverse-ageist for a moment: I'm at a startup where the average age is over 40, the founder is deeply experienced as both an engineer and a manager, and I can honestly say that NONE of this stuff happens. I took my reading these anecdotes to appreciate how nice (and apparently rare?) this is.

meesterdude 1 day ago 2 replies      
I loved these! I certainly have been through some of that nonsense.

i would add:

job offer: salary range $100-$130k

interviewer: the max for this position is $80K

interviewer: hello?

hwstar 1 day ago 0 replies      
Problems are not just limited to startups. Here's some of the shenanigans which went on in the last company I worked for:

Company is constantly on credit hold. Engineers receive calls from suppliers stating that they haven't been paid and no one in accounts payable is returning their calls.

Company does layoffs on a recurring basis, and then hires different people for a new project. In one case they hired back the same people they laid off a year ago and paid them signing bonuses.

CEO shows gross indifference when someone leaves the company. Just shrugs it off.

CEO walks into VP of HR's office with someone and states that you're fired, and this is your replacement.

CEO constantly worrying about getting delisted from NASDAQ.

CEO and VP Engineering "chum" thinking they know the price of components and stating that that should cost $5, when everyone supplying stated item is asking $15.

padobson 1 day ago 2 replies      
Based on my experience, there doesn't seem to be much overlap between the type of charismatic pitch founder capable of raising money and getting people to work for less than their market rate, and the bean counter-oversight founder capable of running an efficient business. But it's the former type that is able to raise money, so they find themselves in a position to have conversations like these.
supjeff 1 day ago 4 replies      
I've worked for startups as a full-stack dev for about 5 years, and spent the last year trying to land a position in as a technical manager. This specific pursuit has lead to some of the most bullshit encounters I've had with startups. What are some of your bullshit experiences?
Jean-Philipe 1 day ago 2 replies      
I had this:

young male intern: talks a lot but is not workingyoung female intern: does all the work of the male intern, gets half the pay

me: why?ceo: because the male intern is from some important school!

smcl 1 day ago 5 replies      
The "less than half for cash" one is something I used to see from customers all the time when working in a bar. You'd say "10.50, please" and they'd reply "10 for cash?" as if I'd happily chip in 50p myself (till shortages need covered somehow) instead of having to accept a card payment. I never understood the mentality - it costs x, so pay x or go away
eitland 1 day ago 0 replies      
Three of mine:

Recruiting company: Someone needs a php developer for work in X industry.

I work on the industry, take the test they give me, get a perfect score, never hear a word, no interview.

Big company : I travel across the mountains to an interview, 15 minutes before arriving I get a call, "we can't meet you today, we will come back with another time".

Startup: we need someone with strong cognitive skills and php and javascript experience (8 y ago, before everyone had 10 y javascript experience). In interview: you are one of two qualified, the otherone is from <other country, more than two hours away, by plane>, -goes on to advertise the position again, then after realizing there are no other goes on to offer me below market rates. I go on to learn Java in a better paid job with better well - everything.

doctorpangloss 1 day ago 0 replies      
It's probably unfair to pin all the sexism stuff on startups. If you've ever been to a big company, you'd know that the pink collar workforce idea is way worse. There are a lot of women in junior or administrative positions who can only really interact with their bosses and immediate peers professionally but whom the entire organization, from CEO to lowly PA, tries to hit on.
tonecluster 1 day ago 1 reply      
Here's one (not solely experienced in startups, however):

manager: "Since you're single, can you work through the holidays and cover the systems/push this feature/go the extra mile so that the people with families can spend time with them?"

htaunay 15 hours ago 0 replies      
CEO: We gonna need to do some crunch time the next 2/3 weeks. Unfortunately, we fell behind schedule and we can't let this next delivery be late. We can't pay any of you overtime because we are already on a tight budget, as you know, but to raise moral after we deliver the company is going to take everyone out to this great restaurant + drinks to raise our morale and toast of the future of this great and promising business!

The team: Ok. Not what we expected, but we have some equity, its a tough market, and we have been working on this for a while. Let's hope it pays off in the future.

Three weeks - working from Sunday to Sunday - later...

The Team: Yes! I can't believe we delivered! We are more motivated than ever! Now let's celebrate!

CEO: Yeah, sorry, no time for that. Here's a $40 dollar check for each of you to go out and grab a bite. Oh, and by the look of our roadmap we might need to crunch some more next month.

The Team: [starts quitting one by one]

randycupertino 18 hours ago 2 replies      
My first startup's CEO practiced "The Game" and used to bring the dev team out in SF to practice peacocking, negging and other ridiculous seduction techniques on unsuspecting women. I'm in an ltr and dreaded having to go, but didn't want to be left out so I would tag along. It was horrible.
spinlock 1 day ago 0 replies      
Am I the only one who would have really appreciated a trigger warning?
kelukelugames 1 day ago 3 replies      
YC helps young founders because of stereotypes against them. I wouldn't imply that someone lacks merit because she is 18 and attractive. There needs to be more context for that post.
junkilo 1 day ago 1 reply      
one from the devops/sre trenches...

> (devops guy) hi, im here for the lead devops role> (CTO) awesome, you'd be a great fit here. can you write a sudoku solver in ruby real quick?> (devops guy) [writes solver in ruby]> (CTO) wow looks great... when can you start? do you have any questions for me?> (devops guy) I do have a question: how do you shard your bigdata installations and what is it built on?> (CTO) oh I wrote my own, we call it redisbigtable -- its the worlds first CAP compliant data store. oh, and we shard by customer.> (devops guy) thank you for your time.

Jemaclus 1 day ago 0 replies      
me: I'm here for the engineering manager position

them: great, can you come in for a front-end developer interview at 2?

me: I'm here for the engineering manager position

them: here's an offer for a front-end position making below-average $

me: thanks for your time

them: ???????

CEO: Why won't anyone work for us?

them: we offered them a job!

kevinthew 1 day ago 0 replies      
happened to be recently:

Manager: Hey you're here for the xx position right? You know, you sound like you'd be a really good fit for this new job that opened up, I just fired this guy, millennials amirite, so let's talk about that for the next hour.

giis 1 day ago 0 replies      
>Would you sign something saying you'll give me $250k in cash one year from now if I work for the company?

this is classic ! :)

pyb 1 day ago 0 replies      
I've had 2 of these exact interactions, in the London local startup scene, and that was just last week.
b123400 16 hours ago 0 replies      
We are working in a coworking space, but the CTO doesn't seems to like this idea at all, he keeps asking people in the common area to shut up.
eecks 1 day ago 3 replies      
Are these just made up or are they coming from somewhere?
giis 1 day ago 1 reply      

During interview with founder:

me : I believe 'honesty is the best policy'

founder: Great! we need like-minded people like you.

(after few months)

founder: He creates negative environment within the team!

mrbill 1 day ago 0 replies      
Reminds me a lot of


pbreit 1 day ago 3 replies      
Many/most seem more an indictment on engineer lack of understanding what it takes to get a startup off the ground.
the_cat_kittles 1 day ago 0 replies      
love the equity one. imagine asking "what is the expected value of the equity?", getting a number, and then saying "ok, pay me that when it vests instead of giving me the equity".
bsg75 1 day ago 1 reply      
Has anyone done the math to show if your odds are better at the typical startup vs the typical craps table?

Equivalents of time and money, and the odds of wasting it all on chance?

cinquemb 1 day ago 0 replies      
After a round of interviews/ in person with a public software company in Boston (who just acquired last pass):

Company recruiting lead: The team enjoyed meeting you. They felt you are very smart and personable. [VP of engineering] liked your work with machine learning. At this time, they feel you are a bit junior for the team.

I had to laugh, glad I dogged a bullet.

velox_io 1 day ago 0 replies      
It's quite scary how many startups treat Lean Startup as the gospel and follow it word for word.
Giorgi 1 day ago 3 replies      
It's beyond me why would anyone would want to give away part of his/her company just because of some shitty development work, and most of developers do just that - write shitty code because they don't give a crap about quality

As a founder you are better of paying some random developer do have viable product and then if it works - hire someone.


michaeloblak 18 hours ago 0 replies      
This is so true! Awesome (and sad)!
mrcrassic 1 day ago 0 replies      
this is amazing.
cft 1 day ago 2 replies      
Somebody who dedicates his time to publicly spew this toxic attitude will likely not archive success that is pleasant for him.
Stop using gzip imoverclocked.blogspot.com
427 points by imoverclocked  3 days ago   250 comments top 41
geofft 2 days ago 7 replies      
The trouble with this is that, as a software author, it doesn't really matter if it takes 70 seconds instead of 33 to install my software. 70 seconds is fast enough, for someone who's already decided to start downloading something as involved as Meteor; even if it took one second it wouldn't get me more users. And it would have to take over 5-10 minutes before I start losing users.

On the other hand, having to deal with support requests from users who don't have any decompressor other than gzip will cost me both users and my time. Some complicated "download this one if you have xz" or "here's how to install xz-utils on Debian, on RHEL, on ..." will definitely cost me users, compared to "if you're on a UNIXish system, run this command".

From a pure programming point of view, sure, xz is better. But there's nothing convincing me to make the engineering decision to adopt it. The practical benefits are unnoticeable, and the practical downsides are concrete.

cellularmitosis 3 days ago 9 replies      
I'd argue that bzip2 is a better example of a compression algorithm which no one needs anymore.

Considering these features:

 * Compression ratio * Compression speed * Decompression speed * Ubiquity
And considering these methods:

 * lzop * gzip * bzip2 * xz
You get spectrums like this:

 * Ratio: (worse) lzop gzip bzip2 xz (better) * C.Speed: (worse) bzip2 xz gzip lzop (better) * D.Speed: (worse) bzip2 xz gzip lzop (better) * Ubiquity: (worse) lzop xz bzip2 gzip (better)
So, xz, lzop, and gzip are all the "best" at something. Bzip2 isn't the best at anything anymore.

gizmo 3 days ago 1 reply      
One of the great things about gz archives is that the --rsyncable flag can be used to create archives that can be rsynced efficiently if they change only slightly, such as sqldumps and logfiles. Basically the file is cut into a bunch of chunks, and each chunk is compressed independently of the other chunks. xz doesn't seem to have an equivalent feature because the standard implentation isn't deterministic[1].

Changing from one compression format to another seems harmless, but it always pays to think carefully about the implications.

[1]: https://www.freebsd.org/cgi/man.cgi?query=xz&sektion=1&manpa...

jzwinck 3 days ago 3 replies      
There are many more concerns to address than just compression ratio. Even the ratio one is questionable, because some people have really fast networks but we all have basically the same speed of computers. So a 4x CPU time and memory pressure penalty may be much worse on a system than a 2x stream size increase. Another use case is a tiny VM instance: half a gigabyte of RAM is not actually present in every machine today. Embedded, too.

Another way compression formats can win you much more than a 2x space reduction is by supporting random access within their contained files. Gzip sort of supports this if you work hard at it. Xz and bzip2 appears similar (though the details are different). I achieved a 50x speedup with this in real applications, and discussed it a bit here: http://stackoverflow.com/questions/429987/compression-format...

LeoPanthera 3 days ago 2 replies      
He didn't mention the biggest difference between gzip and xz - ram usage. At maximum compression, you need 674 MiB free to make a .xz file, and 65 MiB to decompress it again. That's not much on most modern systems, but it's quite a lot on smaller embedded systems.

Admittedly, in most cases, that isn't much excuse though.

hdmoore 2 days ago 0 replies      
Summary: Compatibility and decompression speed is more important than compression ratios for many use cases. Gzip is nearly universal, where lz4, xz, and parallel bzip2 are not.

The challenge of sharing internet-wide scan data has unearthed a few issues with creating and processing large datasets.

The IC12 project[1] used zpaq, which ended up compressing to almost half the size of gzip. The downside is that it took nearly two weeks and 16 cores to convert the zpaq data to a format other tools could use.

The Critical.IO project[2] used pbzip2, which worked amazingly well, except when processing the data with Java-based tool chains (Hadoop, etc). The Java BZ2 libraries had trouble with the parallel version of bzip2.

We chose gzip with Project Sonar[3], and although the compression isn't great, it was widely compatible with the tools people used to crunch the data, and we get parallel compression/decompression via pigz.

In the latest example, the Censys.io[4] project switched to LZ4 and threw data processing compatibility to the wind (in favor of bandwith and a hosted search engine).


1. http://internetcensus2012.bitbucket.org/images.html2. https://scans.io/study/sonar.cio3. https://sonar.labs.rapid7.com/4. https://censys.io/

bitwize 2 days ago 0 replies      
Me, I wish people would stop using RAR. It's proprietary and doesn't have a real compression advantage vs. e.g., 7-Zip, bzip2, or xz.
ak217 3 days ago 2 replies      
For anyone looking to stop making compromises, I recommend pixz. It's binary compatible with xz, and is better at compression speed, decompression speed, and ratio than both gzip and xz on multicore systems. I've adopted it in production to great benefit.
KirinDave 2 days ago 4 replies      
Being a windows user these days, I am getting kinda frustrated with how anemic everyone is at even trying to google for 20s to find the windows solution.

7zip is the program you want to handle most everything, with both gui and command line options: http://www.7-zip.org/

Given how radically MS is trying to reform itself to be an open-source friendly company and how ineffectually inoffensive they've been the last 5 years, can we at least try and throw them a bone or two?

orionblastar 2 days ago 2 replies      
When I owned an Amiga they kept on changing the archive format to find a better one that saved space.

The had arc, pak, zip, zoo, warp, lharc, and every Amiga BBS I got on used a different archive format. Everyone had a different opinion on which archive format compressed things in the best way.

I think eventually they decided on lharc when they started to put PD and shareware files on the Internet.

Tar.gz is used because there are instructions for it everywhere and it seems like a majority of free and open source projects archive in it. It is a more popular format than the others right now. Might be because it is an older format and had more ports of it done.

But I really like 7zip, it seems to compress smaller archives, before 7Zip I used to use RAR but WinRAR wasn't open source and 7Zip is so I switched.

With high speed Internet it doesn't seem to matter much anymore unless the file is in over a gigabyte in size. Even then Bit Torrent can be used to download the large files. I think BitTorrent has some sort of compression included with it if I am not mistaken. To compress packets to smaller sizes over the torrent network and then resize them when the client downloads them. That is if compression is turned on and both clients support it.

anonova 3 days ago 3 replies      
> So, who does use xz?

Arch Linux started using lzma2 compression for their packages nearly 6 years ago!


samstokes 3 days ago 4 replies      

 OSX: tar -xf some.tar.xz (WORKS!) Linux: tar -xf some.tar.xz (WORKS!)
I had no idea tar could autodetect compression when extracting. (I wonder if this is GNU tar only, or whether the OSX default tar can do it too?) I've been typing `tar zx` or `tar jx` for too long.

bhouston 3 days ago 3 replies      
I wish lzma (xz) was integrated into the browser and curl as an Accept-Encoding. Would be amazing for us (clara.io), and I am sure a lot of others.
sbuttgereit 3 days ago 1 reply      
I think this is one of those things where the author is pretty much 100% right and it just won't happen. Habits are hard to break and in many cases, the negatives just don't impose a high enough cost to matter.

There are times when I do seriously look for the optimum way to do things like this and then there's most of the time I just want to spend brain cycles on more important problems.

jmspring 3 days ago 0 replies      
I mentioned it when it came up on another thread. Compare apples and apples -- use one of the standard corpuses when running bench marks.

Ian Witten put together the Calgary corpus - https://en.m.wikipedia.org/wiki/Calgary_corpus

profquail 3 days ago 2 replies      
Windows users: 7-zip can extract .xz files should you need to (article didn't mention a Windows solution).
mkj 3 days ago 1 reply      
Which version of RHEL does "Linux" include? The world isn't all Ubuntu recent releases.
Phemist 2 days ago 1 reply      
For me, as a Python user, I've found that gzip is currently the only compression format that allows streaming compression/decompression. I don't want to have to store hundreds of gigabytes of data and THEN compress it, rather than compressing it right during file generation. I haven't found any other compression lib that supports this out of the box.
daemonk 2 days ago 1 reply      
Decompressing takes 4 times as long? I wonder if that is slow enough to create a bottleneck in processing. Not everyone uses compression for purely archival purposes. In the genomics field, most sequencing data are gzipped to save disk space. And most programs used to process the sequencing data can take in the gzipped files directly.
jph 3 days ago 0 replies      
I prefer compressing with gzip because it's on more systems, works well even with low RAM, and enables fast rsync for updates.
jack9 2 days ago 0 replies      
geezip is fun to say. Until there's a catchy name for "crosszip"/xz/whatever, I think we're preaching to the wrong choir. There's a human element in toolchains. Address it.
zipzipzipzip 3 days ago 0 replies      
it's a shame that algorithm improvements would necessitate a shift away from the name "gzip". It would be better if the intent to compress/decompress was orthogonal to the features of the implementation (compression ratio, speed, split-ability, etc...)
zbuf 2 days ago 1 reply      
The article misses (but the comments here touch on) that all compression algorithms have a built in obsolescence, even the fancy shiny xz.

It's not the algorithm per-se that go obsolete, but their use in specific cases until all are diminished. Whether lossy or lossless, eventually other technological advancements renders them unnecessary.

And it seems that strongest algorithm is usually the earliest to be widely adopted; these are almost never toppled.

Just like .gz, look at MP3 or JPEG -- 'better' alternatives exist, but the next widely adopted step will be to eliminate that compression entirely. The first radio station playout systems were hardware MPEG audio compression, and the next most widespread step was to uncompressed WAVs. Even video pipelines based on uncompressed frames are becoming more widespread. Eventually the complexity and unpredictability of compression is shunned for simplicity.

Read the gzip docs and the focus is around compression of text source code, a key use case at the time but barely considered these days -- tar.gz source archives exist almost only out of habit; they could just as well be tar.

faragon 2 days ago 0 replies      
gzip is fast, gzip -1 is even faster, gzip has low memory requirements, gzip is widely adopted. Those are the reasons of gzip is still being used, and why gzip has future. I.e. the gzip "ecosystem" is rich and useful, despite not being the best compressor in terms of compressed size.

P.S. There are gzip-compatible implementations with tiny per-connection encoding memory requirements (< 1KB).

skywhopper 2 days ago 0 replies      
tl;dr: xz compresses better but is significantly slower. This isn't the deepest analysis of potential tradeoffs you might be able to find.

A few reasons why gzip is still useful to have around:

* Speed is critical for many applications, and so size can take a backseat when performance is critical or resources are low.

* gzip is basically guaranteed to be available everywhere in utility and library forms.

* Download speeds vary and so the faster your pipe, the less the archive size factor will matter, and the faster-worse compression might win out in other comparisons.

* xz doesn't compress every type of data this much better than gzip. I've dealt with scenarios where the difference is consistently less than 2%, and the extra time xz spends is actually a tremendous waste.

Sure, for package downloads where xz files will be significantly smaller it makes sense to save the bandwidth, time and storage space. But it's not 100% cut and dry.

bshanks 1 day ago 0 replies      
There's also lzip, which apparently uses a similar compression algorithm as xz but is apparently built with partial recovery of corrupted archives in mind (so more useful for long-term archival or backup storage). It's made by the same guy who made ddrescue.
necessity 2 days ago 0 replies      
What about availability? I often find myself having to download and compile (de)compressing software because the authors of some other software I need decided to ship it in something other than the standard (.tar.gz), which is available in basically all *nix boxes.
jeffdavis 2 days ago 0 replies      
I don't see any reason the author made the jump from "xz is better than gzip" to "stop using gzip!".

What is the compelling thing here that makes him feel that this is a moral imperative?

InclinedPlane 3 days ago 1 reply      
Ahh, kids. "Let's all start adopting this new thing that has existed for a few years that's slower and uses more ram because we're wasting literally tens of megabytes all the time!"
imslavko 3 days ago 1 reply      
Can node-tar untar .tar.xz? This is what Meteor uses to parse tarballs.


djhworld 2 days ago 0 replies      
I asked this question on StackExchange a few months ago http://unix.stackexchange.com/questions/183465/fastest-way-o...

That's my problem with GZIP, in this particular use case anyway.

Animats 2 days ago 1 reply      
Amusingly, "news.ycombinator.com" serves its pages with .gz compression. Even if you send an HTTP header that demands plain text only.
fz7412 2 days ago 1 reply      
If I'm sending contents of a website to client in .xz format, will browsers be able to decompress it?
otterley 2 days ago 0 replies      
The only people who care about compression ratios are:

(1) People who still use 56k modems to download content

(2) People who host extremely popular downloads and who want to minimize their outbound bandwidth bills

If you're not one of these two, you almost certainly care more about compatibility and compression time than compression ratio. gzip continues to win on both those fronts, and it explains why it's still the most popular compression format other than ZIP (which is a better choice than gzip if you frequently need to extract a single file from a compressed archive).

Until there's a compression tool released that can compress at wire speed (like gzip) and has a significantly better compression ratio, don't expect the landscape to change much.

ultim8k 1 day ago 0 replies      
Aha! Thanks for sharing.
JustSomeNobody 3 days ago 1 reply      
Is xz greppable?
foxhop 2 days ago 0 replies      
tar ... tape archive
dietrichepp 3 days ago 0 replies      
The Weissman score is less than worthless, it "isn't even wrong". This comes up in most HN discussions about compression algorithms and I'm waiting for it to go away.


modarts 2 days ago 0 replies      
GalacticDomin8r 3 days ago 0 replies      
eximius 3 days ago 4 replies      
ick, the invocation is `tar tJvf`? Granted, I can alias it but a capital J is just about the worst option letter I can think of.
Koel: A personal music streaming server github.com
473 points by vive-la-liberte  15 hours ago   231 comments top 27
mentos 11 hours ago 8 replies      
Anyone else find that in this new streaming age they listen to a lot less music? It may just be a function of my age, but I feel like if I had a device with all of my songs on it again I would listen to much more.

I think there could be another iPod age.

Imagine a device with 10Tb of flash memory, an E-Ink display and 2.5 million songs.

unixhero 13 hours ago 7 replies      
I have been using Subsonic.org on my servers since 2009. It has and still is serving me very well.
amelius 14 hours ago 21 replies      
What I'm looking for is better music discovery. This would require some form of machine learning/statistics/collaborative filtering, but basically, if I like a number of songs, I'd want the system to automatically recommend new songs/artists.
acoleman616 12 hours ago 1 reply      
I'm happy to see Laravel being used for more and more projects and getting the credit it deserves as truly first-class framework -- regardless of language.
snthd 10 hours ago 1 reply      
I couldn't compile it on Debian yet, but tomahawk (https://www.tomahawk-player.org/) looks interesting.

You could set up a music NAS with beets (http://beets.radbox.org/) as one of several sources (also you could have spotify, youtube, soundcloud) then have playlists that include stuff from a combination of them.

A last.fm type service for tomahawk is in beta at https://hatchet.is

archseer 14 hours ago 1 reply      
I've been working on something similar, just for fun (gave me an excuse to mess with Elixir and Ember). Still highly unstable though, I've spent maybe a week total on this. It's a rewrite of an older sinatra-based project I did back when html5 audio became stable enough (on my github as well).


edent 14 hours ago 4 replies      
Will be interesting to see how this compares with http://ampache.org/ which, while great, still hasn't solved the mobile access problem.
addict3d 8 hours ago 0 replies      
This looks great, could you please integrate youtube and soundcloud search for streaming?

EDIT: https://github.com/embedly/player.js may save you some time!

pierrec 8 hours ago 2 replies      
I've been looking through online playlist servers and wondering: Why shouldn't I host something like this as a static website? After looking around and not finding any tool that generates this, I'm considering making it myself, shouldn't be too hard.

I don't really need any features that require server-side processing, and static hosting would essentially reduce the cost to zero, for the amount of storage and bandwidth I would make of it (as well as possibly making it a bit faster).

joeyspn 4 hours ago 0 replies      
Docker image for y'all incoming...



mrmondo 13 hours ago 0 replies      
Was excited to see this, I did hit a lot of composer and npm problems (as I somewhat expected) here's hoping the bug reports I've filed can be easily resolved. At the moment I'm using groovebasin but that needs some serious improvement around it's installation dependencies.
addict3d 9 hours ago 1 reply      
Holy, cow! I was working on a node-webkit/cordova angularJS app very similar to this until I got bored of angular. It pulls music from Soundcloud and YouTube.

Here's a screenshot if anyone's interestedhttp://i.imgur.com/uSBNSeE.png

ohblahitsme 12 hours ago 0 replies      
This is awesome! I'm gonna set up my R-Pi at home to run this and merge my brother's and my music libraries. Thanks for this!
pwenzel 9 hours ago 0 replies      
I wonder if one could drop Koel on a webserver and use a cloud service like Box.com, Dropbox, Google Drive/Nearline to serve the MP3 files?
dannyrosen 14 hours ago 5 replies      
One of the features I've long sought after has been multiroom streaming (ala Sonos)
benkaiser 9 hours ago 0 replies      
for a node.js based Google Play Music clone (with android sync application included)+ youtube downloading built in (with playlists)+ soundcloud downloading built in (with playlists)see my project Node Music Player: https://github.com/benkaiser/node-music-player

I've been working on it for the last few years with a few pull requests from awesome members of the community and I'm about to launch auto-generated mixes that allow you to explore music in a way similar to Google Play Music and Spotify (using lastfm api and youtube: here's the module for it https://www.npmjs.com/package/similar-songs )

oxguy3 11 hours ago 0 replies      
oh my god yes thank you, i've wanted this exact piece of software for so long, and just was too lazy to program it myself. this is EXACTLY what i wanted.
ascagnel_ 13 hours ago 0 replies      
This is interesting, but how would it compare against something like Plex (that handles video and transcoding as well as music, but not open source) or Boombox (just music and also open source).
brandonmenc 9 hours ago 0 replies      
If this lets you cache music locally, it will replace iTunes for me.
methyl 14 hours ago 1 reply      
Plex is also very good alternative to Spotify, if you share your server on public IP.
mkj 13 hours ago 2 replies      
Does it have 30 million tracks?
johansch 11 hours ago 1 reply      
Having your own stash of pirated mp3 tracks is so 1997.
sneak 15 hours ago 4 replies      
Subsonic is a similar project, and has a mature ecosystem of client apps. (Added bonus: no PHP.)
danpalmer 13 hours ago 8 replies      
I find it interesting that this is described as a Spotify clone. To me, Spotify is a large music library that I can pay a subscription to access, with advanced (arguably) discovery features. This on the other hand is a web based player and streaming server. I wouldn't describe them as the same at all.

Is this really how people view Spotify?

Edit: interestingly, the product itself doesn't mention Spotify anywhere.

gedrap 11 hours ago 4 replies      
The project looks cool, however, I find using 'Spotify' in the title very very clickbaity. It's just a music streaming service + player. There's nothing wrong with that, I am not saying that the project sucks or anything, it's just uncool to use a popular brand to attract clicks and call it a clone when it's something totally different.

It's like calling VLC self-hosted netflix clone.

callumlocke 11 hours ago 2 replies      
Weird that everyone is nitpicking the HN submission title. Using the word clone is just an honest admission that the UI design is ripped off. It's fairly obvious that a "self-hosted Spotify clone" would require you to bring your own music.
xena 8 hours ago 1 reply      
I wish that people would write stuff like this in languages other than PHP.
How Elon Musk and Y Combinator Plan to Stop Computers from Taking Over medium.com
341 points by sergeant3  3 days ago   166 comments top 38
cromwellian 3 days ago 14 replies      
The near term danger of AI isn't in hyperintelligent SkyNet like systems, it's in human controlled autonomous weapon systems and "stupid AI"

What you should be fearing is military drones being given the ability to make decisions on targets or to fire, even with human assistance, and these systems won't just reside in the hands of large governments either.

Already police and militaries around the world are using abstracted forms of force, wherein target are identified with algorithms, and then force trained on those targets

What's do you think is going to happen first, SkyNet, or a predator imaging drone telling a human operator falsely that the current image is a terrorist?

What's going to happen first? SkyNet, or self driving cars putting millions of people out of jobs because of a lack of demand for drivers in transportation, or manufacture of cars? (I'm not saying it's a bad thing, but it will be very very disruptive)

If SkyNet is a threat, it's 50 or 100 years off I think. "AI" as it is now, is no where near the capability people are talking about. It's sheer hyperbole.

chubot 3 days ago 0 replies      
I wonder what their use cases are. "Advance the state of the art in AI" is just too nebulous. Having the smartest people in the world isn't enough... you need to focus them on some goal.

Once you get beyond 8 researchers, you'll have problems with politics and egos if people aren't focused on a problem. Everyone will have their pet approach for specific problems, and they won't compose into something generally useful. AI is really like 10 or 20 different subfields (image understanding, language understanding, motion planning, etc.)

I think self-driving cars are a perfect example of a great problem for AI (and something that many organizations are interested in: Google, Tesla, Apple). Solving that problem will surely advance the state of art in the AI (and already has).

tl;dr "OpenAI" is too nebulous.

api 3 days ago 1 reply      
Before you invite regulation into this area, take a long hard look at how the government has historically approached cryptography and IT security. This is a far simpler domain with far simpler concepts, and it's a total shitshow that alternates between ignorant security theater and self-serving power grabs.

Get into bed with the government and they will piss in it. The most likely outcome is costly complicated regulations that hobble legitimate development and accomplish nothing in terms of making us safer from anything. The end result will be like ITAR and crypto export controls: pushing development off shore and making the USA less competitive.

I say this not as a hard-line anti-government right-winger or dogmatic libertarian, but as someone who has a realistic view of government competence in highly technical domains. Look at other areas and you don't see much better. Corn ethanol, for example, is literally the pessimum choice for biofuels-- it is technically the worst possible candidate that could have been chosen to push. The sorts of folks who ascend to political power simply lack any expertise in these areas, and so they fall back on listening to the agenda-driven lobbyists that swarm around them like flies. The results are awful. Government should do government but should stay the hell away from specific micromanagement of anything technical.

heurist 3 days ago 1 reply      
Sounds to me like YCombinator wants to fuel their growth by creating these tools for their companies to use. The YC business model is absolutely brilliant, but I can't see this being some purely altruistic mission. Or if it is then I am jealous that they have the power and resources to put up against such a project. I'd spend all my time trying to build an AI if I didn't have to work! (Though I am trying to steer the company I work for in that direction anyway...)
ddod 3 days ago 6 replies      
Whether it's the "singularity" or just software naturally improving over time and taking on more "thinking" work, there's going to be a huge and insurmountable unemployment problem in the near future. The market values human thought/labor to the extent that it's cheaper or more effective than an automated solution. When that isn't the case, you can fill in the blanks. That, to me, is the scary part of AI.

It doesn't sound like this project has any scope to address this practical concern, which to me, is largely economic. I don't see how universal access to AI puts food on the table.

amai 2 days ago 1 reply      
This initiative makes no sense. If AI could really become a technology which would endanger our civilization, when you clearly would not(!) want it to be useable for everyone. There is a reason why it is not the right of every american citizen to own plutonium, buy sarin or send letters full of anthrax around.

But if AI would be as dangerous to society as for example cars, then we don't need such an initiative. So for me the whole thing seems to be a marketing stunt of sleep-deprived billionaires who read the wrong books.

rpm33 3 days ago 1 reply      
Once again sensationalism. I watched that interview. Sam's take on AI is perhaps the most practical I've seen in popular media, while everyone is freaking about a singularity event.
MBlume 3 days ago 0 replies      
This...sounds incredibly naive? They seem to think that AI risk comes from Bad People doing AI? There's not one mention given to the possibility of well-intentioned people destroying the world by accident.
Artoemius 2 days ago 0 replies      
I think the best defense against the misuse of nuclear weapons is to empower as many people as possible to have nuclear weapons. If everyone has nuclear weapons powers, then theres not any one person or a small set of individuals who can have nuclear weapons superpower.

Yeah, right.

gist 3 days ago 2 replies      
Musk is spread so so thin. Not enough to run a rocket company, a car company and be the Chairman of Solar City. Needs to have his hand in even more pots.
martin_ 3 days ago 3 replies      
> Altman: Our recruiting is going pretty well so far. One thing that really appeals to researchers is freedom and openness and the ability to share what theyre working on, which at any of the industrial labs you dont have to the same degree.

Do current AAPL/GOOG/FB engineers dislike this so much? There's secrecy within most for-profit entities, what makes AI so different?

bkbaba 2 days ago 1 reply      
Musk: I think the best defense against the misuse of AI is to empower as many people as possible to have AI. If everyone has AI powers, then theres not any one person or a small set of individuals who can have AI superpower.


Isn't this like gun control all over again?! You give more guns to people so that they can be safe, instead you end up killing each other.

humanfromearth 3 days ago 1 reply      
It might be an incredibly bad idea to have multiple AGIs everywhere in the world, but that's the least bad that I can see too.

Also this is amazing, making serious effort towards AGI is what we need. We'll play with RNNs configurations for a long time, but I think it's a good call to fund people who think about the broad picture.

danieltillett 3 days ago 4 replies      
My real concern about AI is that generalised Moores law means we only have a relatively short time to plan. Assuming that computers continue to double in processing power every 18 months or so, then we only have 10 years where we go from 1% human level to human level. This really is not a long time to make good decisions.
axplusb 2 days ago 0 replies      
As awesome as this looks, I'm totally missing the point.

If they truly believe AI is dangerous, how does promoting / accelerating it is supposed to help?

Or is it a way to commoditize R&D in machine learning so that it will never be a bottleneck for startups?

cromwellian 3 days ago 0 replies      
Remember, when Engines Of Creation and Nanosystems were published, and there was a great fear that uncontrolled Nanotech development would result in a GreyGoo that would consume us all?

With stuff like CRISPR, perhaps Elon should invest to stop the zombie apocalypse. :)

walterbell 3 days ago 0 replies      
There is a role for regulation and collective choice. Take HFT as an example. If everyone competes for lower latency access, eventually everyone locates their bot at the same exchange. If everyone puts a lower limit on latency (e.g. by coiling fiber optic cable), the latency playing field is leveled and new areas of competition emerge.

Open technology will empower the expression of many human wills, individual and collective. Human wills are today constrained and empowered by many human-imagined systems of thought, and we can invent new ones. Will there be an AI which explores the possibility space of constraints on AI-using humans?

tinalumfoil 3 days ago 0 replies      
It's incredible the types of doomsday scenarios the wealthy invest in stopping. The problem Elon Musk and Y Combinator are going to solve with their money, what they will be remembered for fixing after their companies have long crashed and gone bankrupt, is better technology. Essentially, technology will become so good at doing human's work we will run out of problems for people to solve and drift into a lazy non-working state incompatible with current economies. I predict Earth will be destroyed by passing meteor before that happens.

Maybe if I was a billionaire I'd understand.

astrofinch 2 days ago 1 reply      
"Security through secrecy on technology has just not worked very often."

Nuclear weapons come to mind. Would we prefer that the knowledge of how to make them be more widespread?

ThomPete 3 days ago 0 replies      
Humans are general purpose animals and technology is general purpose humankind.

If we believe that DNA is a kind of information and our genes are "looking for" better weasels to survive through then it's only natural to also see technology as a much better carrier of that information than us.

The problem many have with coming to grasp with the idea that AI could be a threat is because they look at where technology is right now and then try and imagine a computer being anywhere near our capabilities.

But this is because many think of it as a thing. As in. "Now we have finally build a strong AI thingiemagick". However just as humans consciousness and intelligence isn't a thing, neither will AI be. It's going to be a lot of things. Some are better developed than others, but most moving at impressive speed and at one point enough of them are going to be put together to create some sort of pattern recognizing feedback loop with enough memory and enough smart sub-algorithms to became what we would consider sentient. </tinfoil hat>

arbre 3 days ago 0 replies      
This is great news. I work at a big company with advanced machine learning tools and infrastructure. Everytime I use them I am amazed by the tools but kinda sad for the students and researchers who have to deal with simpler/less powerful tools. This gives me hope that the best tools will eventually be open source.
Teodolfo 1 day ago 0 replies      
It is really sad that the people funding these excellent researchers have no fucking clue about AI (they know plenty about other things and have done plenty of good work, but the level of nonsense here is striking). Thank God they are giving the money to competent people instead of doing what they think they are doing with it.
melling 3 days ago 3 replies      
While we're at it maybe we should address the possibility of overpopulation on Mars?

Andrew Ng thinks people are wasting their time with evil AI:


mocookie 2 days ago 1 reply      
This is starting to sound like the OSAF that tried to build a cross-platform open-source email/calendar/notes application for the betterment of the world - in competition with Microsoft and whatever other large corporations were doing PIMs at the time.


hollerith 3 days ago 0 replies      
The OP seems to assume that the big danger with AI is that it will leave the people at the mercy of an (human) elite that controls an AI or that has programmed an autonomous AI (an AI not controlled by any humans) to care mostly or only about the elite.

In contrast, what organizations like the Machine Intelligence Research Institute and the Future of Humanity Institute (MIRI and FHI) consider the main danger (and have considered the main danger for over 11 years) is that the AI will not care about any person at all.

For the AI to do an adequate job of protecting human welfare it needs to understand human morality, human values and human preferences -- and to be programmed correctly to care about those things. Designing an AI that can do that is probably significantly more difficult than designing an AI that is so intelligent that the human race cannot stop it or shut it down (although everyone grants that designing an AI that cannot be stopped or shut down by, e.g., the US military is in itself a difficult task).

The big danger in other words seems to come not from a research group using AI research to try to take over the world or to gain a persistent advantage over other people, but rather from a research group that means well or at least has no intention to be reckless or to destroy the human race, but ends up doing so by having an insufficient appreciation of the technical and scientific challenges around protecting human welfare, then building an AI that is so smart that it cannot be stopped by humans (including the humans in the other AI research groups).

I fail to see how changing the AI-research landscape so that more of the results of AI research will be published helps against that danger. If one team has 100% of the knowledge and other resources that it needs to build a smarter-than-human AI (and has the will to build it) and all the other teams have 99.9% of the necessary knowledge, there might not be enough time to stop the first team or (more critically IMHO) to stop the AI created by the first team. In particular, if the first AI is able to build (e.g., write the source code for) its own successor -- a process that has been called recursive self-improvement -- it might rapidly become smart enough to stop any other smarter-than-human AI from being built (e.g., by killing all the humans).

Rather than funding a non-profit that will give away its research output to all research groups, a better strategy is to give the funds to MIRI who for over 11 years have been exhibiting in their writings an vivid appreciation for the difficulty of creating smarter-than-human AI that will actually care about the humans rather than simply killing them because they might interfere with the AI's goal or because the habitat and the resources of the humans can be repurposed by the AI.

Any effective AI -- or any AI at all really -- will have some goal (or some set or system of goals, which for brevity I will refer to as "the goal") which may or may not be the goal that the builders of the AI tried to give it. In other words, everything worthy of the name "mind", "intelligence" or "intelligent agent" has some goal -- by definition. If the AI is powerful enough -- in other words, if the AI is efficient enough at optimizing the world to conform to the AI's goal -- then all humans will die -- at least for the vast majority of possible goals one could put into a sufficiently powerful optimizing process (i.e., into an sufficiently powerful AI). Only a very few, relatively complicated goals do not have the unfortunate property that all the humans die if the goal is pursued efficiently enough -- and learning how to define such goals and to ensure that they are integrated correctly into the AI is probably the most difficult part of getting smarter-than-human AI right.

That used to be called Friendliness problem and is currently usually called the AI goal alignment problem. The best strategy on publication is probably to publish freely any knowledge about the AI goal alignment problem, while keeping unpublished most other knowledge useful for creating a smart-than-human AI.

I will patiently reply to all emails on this topic. (Address in my profile.) I do not get a salary from FHI or MIRI and donating to FHI or MIRI does not benefit me in any way except by decreasing the probability that my descendants will be killed by an AI.

thallukrish 2 days ago 0 replies      
One example of a evil AI is the fact that the power of all the data out there about people,things and the ability to learn from it allows connections to be made like never before. With the right algorithms it should be possible to even make policy shifts that can impact human lives in a manner that is favourable to the 'super power' Govt. or company who owns this.
keithwhor 3 days ago 0 replies      
So, capitalism is all about exploiting unfair advantages, right? First mover's advantage on AI developments (regardless of whether they're made publicly accessible eventually or not) seems like a pretty big unfair advantage.

Good for them. I expect some great work to come out of this. :) I'm most excited to automate travel as quickly as possible --- too many people die each year from automobile accidents.

oneJob 2 days ago 0 replies      
Here is my prediction, after watching "Terminator Genisys". The dangerous AI is not the AI humans "invent". It is the AI that runs away. But, what AI will humans allow to "run away"? What scenario presents an opportunity for one group of humans to attack an AI and then provides the opportunity for said AI to be released from its lease ("run away") and become that which we all fear?

So, you have 'red team' and 'blue team'. Blue team is super rich and builds itself an awesome AI. Red team needs some "rally round the flag" pick me up and so, looking around for targets, decides that attacking a bunch of machines is a safe bet. If they win, awesome. If not, then they didn't kill any persons, just made a bunch of junk.

Blue team's response is to internalize the threat (as is only natural, or is at least politically expedient to some subset of blue team) and frame the situation as follows: "This is what we built our AI for. This is an existential threat. It has the capacity. We only need to let it off the leash. The choices are 'destroy' or 'be destroyed'. This is nothing less than an existential moment for our civilization."

And, with that horrible, non-technical, propaganda riddled rationalization the AI developed by the most well meaning of people will be let off the least, will run way, and nothing that we know about the AI up to that point will be worth diddly squat.

I respect anyone that tries to tackle this issue. But, the nature of the issue, the kernel of the problem, is nothing less than Pandora's box. We won't know when it is opened. But, the AI will.

xiaoma 3 days ago 2 replies      
Is the Steven Levy who wrote this the same Steven Levy who wrote Hackers?
kordless 3 days ago 0 replies      
> unconstrained by a need to generate financial return

AI should definitely be constrained by financial means. Computing, unbounded by financial constraints, will eat everything.

Animats 2 days ago 0 replies      
The near term danger of AI, as I point occasionally, is a Goldman Sachs run by an AI. Machine learning systems are already making many investment decisions. We're not far from the day when society's capital allocation is controlled by programs optimizing for return over a few years, and nothing else.
codeulike 3 days ago 0 replies      
So all this will end with the red open-source AI Jaeger mech battling against the grey corporate AI Jaegers among the ruins of our cities. Thanks, Elon.
zobzu 3 days ago 0 replies      
I want computers to take over.
perseusprime1 3 days ago 0 replies      
Fix my autocorrect
phlandis 3 days ago 0 replies      
At the same time couldn't this just make it easier for rogues to fork?
theklub 3 days ago 0 replies      
Computers already took over.
coldtea 2 days ago 0 replies      
>How Elon Musk and Y Combinator Plan to Stop Computers from Taking Over

Well, for Y Combinator is easy: by ensuring funding goes to "Uber for X" and "Facebook for Y" startups instead of real technology advancing businesses


gist 3 days ago 1 reply      
"OpenAI is a research lab meant to counteract large corporations who may gain too much power by owning super-intelligence systems devoted to profits"

As opposed to (almost) the entire startup ecosystem which is focused on ... profits.

Edit: And what does "to much power" even mean other than trying to use hyperbole to make some kind of point.

Can't sign in to Google calendar on my Samsung refrigerator productforums.google.com
433 points by lelf  9 hours ago   273 comments top 48
codemac 7 hours ago 10 replies      
The problem here is not the refrigerator, or the software on it.

I repeat.

The problem here is not the refrigerator!

We're building a shitty internet, not an internet of shitty things. If integrating with an online calendar means having to have a software team at the ready at all times for the whim's of some service's API updates.. we have failed society in a deep way.

The API that Google should be releasing for Google calendar is CalDAV, because it's a standard that people can depend on. Google - by choosing to not use that standard AND to not get a different one standardized with their peers, shows they don't want to be part of an Internet.

VMG 9 hours ago 2 replies      
Ubik, 1969

> From the drawer beside the sink Joe Chip got a stainless steel knife; with it he began systematically to unscrew the bolt assembly of his apt's money-gulping door.

> "I'll sue you," the door said as the first screw fell out. Joe Chip said, "I've never been sued by a door. But I guess I can live through it.

begriffs 8 hours ago 3 replies      
I guess we know our place socially don't we?

"The upper-class kitchen, designed to be entered only by servants, is identifiable at once: it's beat-up, inconvenient, and out-of-date, with lots of wood, no Formica whatever, and a minimum of accessories and labor-saving appliances like dishwashers and garbage disposals. Why tolerate these noisy things when you can have a silent servant do precisely what they do? The upper-class kitchen does have a refrigerator, but so antique that it has rounded corners and a big white coil on top. Neatness and modernity enter as we move down toward the middle class, and the more your kitchen resembles a lab, the worse for you socially." -- Class, a Guide to the American Status System

makecheck 9 hours ago 11 replies      
I remember seeing a commercial for those things where they drafted the Top Chef brothers, and even in the commercial it looked extremely awkward to use. "Oh, let's just look up something...(walk over...bend down...glance at screen...slowly punch things in...)."

Even if it were the best thing ever, I know technology and I know that fridges last years longer than any gadget ever has. Any choice of touchscreen, OS or even network connection technology would probably be a "bad idea" in 5-10 years.

atestu 9 hours ago 2 replies      
If you like this, you should check out http://twitter.com/internetofshit I am not the author
h00k 9 hours ago 2 replies      
"I bought the fridge so my wife could keep up with my calendar. I hope this gets fixed soon. That is a lot of wasted money if no one can use this anymore."

If only there were other ways of checking or sharing an electronic calendar! The future is now.

ohitsdom 9 hours ago 3 replies      
"Samsung support only advised to restart the refrigerator and check timezone..."

What a time to be alive.

vvpan 9 hours ago 4 replies      
I have a feeling that internet of things is going to be a disaster...
vkou 8 hours ago 1 reply      
Welcome to the internet of things that don't work.

My apartment's fridge looks to be at least a decade old - somehow, it's not clear to me that spotty integration with AltaVista Calendars, or whatever it is we used back in the dark days of 2005 would be any kind of value add.

pjc50 6 hours ago 0 replies      
Coincidentally I just saw this on twitter: https://www.techdirt.com/articles/20151214/07452133070/light...

"The lights are going out all over Europe; we shall not see them go on again in our lifetime. Or until we jailbreak the firmware."

gjkood 9 hours ago 2 replies      
I believe we need to measure ourselves on how close we are to the imaginings of the The Jetsons[1].

Do we need to create a universal standard to measure how close/far we are to/from The Jetsons?

[1]. https://en.wikipedia.org/wiki/The_Jetsons

exelius 9 hours ago 3 replies      
I have a Samsung refrigerator that is supposed to be Internet-connectible. I wasted about 5 minutes on it before giving up. The shoddy quality of the software combined with the low marginal utility of having my refrigerator connected to the Internet just didn't make it worth more than 5 minutes of my time.
ihsw 8 hours ago 2 replies      
Refurbished/used home appliances -- yet another way to leak credentials.

I look forward to a day when the internal components of home appliances are salvaged on the off-chance that a user's credentials could be found in them.

You can bet that off-brand devices will ask for a username/password and store it locally in plaintext.

cbsmith 9 hours ago 0 replies      
We really are living the dream of first world problems.

That said, integration with calendaring software in particular is way harder than it should be.

djhworld 8 hours ago 0 replies      
This is why I hate proprietary systems like this, too many options for failure, you're at the whim of the manufacturer.
Eric_WVGG 8 hours ago 2 replies      
I was impressed and a little amazed by how deadpan and serious the messages in the discussion are nobody stops to reflect on the sheer insanity that were all giggling at over here.

I assume, anyway, eventually my attention wandered and I skipped to the end, where a charming young lad does indeed go unhinged over samsucks and heroin-addicted mothers.

wooptoo 2 hours ago 0 replies      
This underlines that the level of newly launched and already abandoned hardware is alarming.

I'm looking at companies like Motorola who almost stopped (if it weren't for a very vocal user base) updating Moto E about half a year after launching.

And many other phone OEMs who just don't care about updating their software.

Companies came to expect fidelity from customers while providing the minimum of commitment in return.

OEMs should stop wasting money on crappy addonware and invest in quality (and timely) software updates, and better customer relationship.

AdmiralAsshat 8 hours ago 0 replies      
This highlights my biggest concern about the Internet of Things: security. If you can't update it, you can't patch it in the event of security holes. Most of these companies can't even be bothered to patch their routers. How am I possibly going to trust a company to patch their refrigerator?
tdkl 8 hours ago 0 replies      
"bickerdyke Top Contributor250 Best Answers

Sounds like your fridge needs a software update to use the new API version."

> Samsung

> software update

Seems someone needs a new fridge.

wklauss 9 hours ago 0 replies      
This is actually a very good example to have around every time we go into the IoT discussions. We need new interaction models and new platforms that solve the issue with updating several things in the house. I have 6 or 7 devices that I now update frequently (computer, tablet, phone, set top box, watch,..) and it's starting to be a really annoying and time consuming process. Soon I'll have to add my car and who knows what else.
nkrisc 8 hours ago 0 replies      
What's sort of funny is that the only people who could raise a real stink in the right place, the technologically literate, would probably never buy such a fridge. Meanwhile, purchasing this looks like a good idea to the technologically illiterate, and when it doesn't work they complain to Google instead of Samsung. Winning strategy for Samsung!
CptMouse 8 hours ago 0 replies      
I'm shocked -shocked, I tell you- to hear that Samsung is absolute trash. Yet people fall for their bullshit over and over again, despite them openly not giving a fuck about customers, laws, competition, or common sense and basic decency. Yet try to argue with the average user who will keep praising their new Crapsung phone, while in the same sentence telling you how it was being repaired for 3 weeks because of a random hardware failure* . It's hard to contain my frustration towards people who buy Samsung (and thus "encouraging" them) when they should know better. I draw some consolation from their eventual inevitable fall into irrelevance (despite their ties to the corrupt SK government) - the shared fate of most inane, incompetent, and evil behemoths.

TL;DR fuck Samsung; stop keeping those assholes in business

* admittedly a bad example, but whatever you choose it will fail to convey their true awfulness anyway

sophiedeziel 8 hours ago 0 replies      
IoT problems...I fear that some hacker groups from communist states DDoS smart thermostats. Well call it Cold War 2.0
yk 8 hours ago 1 reply      
And this is, why there sits a Linux PC next to my "smart tv" in my living room. The thing about "smart tvs" and I imagine freezers is, that the friendly easy to use interface creates problems that were solved in the early 80ies for computers, and it interfaces with a ridiculously underpowered processor.
ed_blackburn 6 hours ago 0 replies      
I am surprised Samsung don't point them at their own service. Said service can then redirect to google and act as an anti-corruption layer. No doubt they could also find out how many devices are online and wether people are updating.
rconti 7 hours ago 0 replies      
I bought a Samsung fridge last year. The documentation referenced internet features that my fridge did not have.

As I posted on Facebook at the time: "I'm not sure if I'm more appalled that there are fridges with WiFi, or upset that mine doesn't have it."

ArekDymalski 7 hours ago 1 reply      
"Very anxious to get this fixed as this is how my young daughter knows her daily schedule."

This comment terrifies me. It sounds like that I kind of future I've read about in dystopian literature.

ksk 7 hours ago 0 replies      
More and more software development is becoming a constant update treadmill (especially on web/mobile), with companies having no sense of software reliability or stability. "Move fast and break things".. way to go! /s
gvb 5 hours ago 0 replies      
Oh my, "Rawson" is really Eliza! https://en.wikipedia.org/wiki/ELIZA

Rawson: I see that you are facing issues with the calendar app.


Rawson: I understand that must make you feel upset.

Rawson: If there was any other option, I would have definetely provided you with that. I hope you understand.

daveguy 8 hours ago 1 reply      
What's scary is there are hundreds of messages from people seriously trying to resolve this. I expect to see a bunch of snarky "first world problems" comments, but its mostly "me too! my many $,$$$ fridge isn't working!"
daigoba66 8 hours ago 1 reply      
It's cheaper, and probably a better UX, to duct tape a tablet to the fridge door.
tlrobinson 8 hours ago 0 replies      
izzydata 5 hours ago 0 replies      
Smart refrigerators exist? I don't know why I'm surprised. Even more so that people rely on said refrigerators for their daily workflow.
mikerg87 6 hours ago 0 replies      
I think this tells me that web api's and their versions are going to need to live much, much longer. Far longer than anyone realizes or is currently planning for.
jkrp 8 hours ago 0 replies      
"Samsung support advised...to restart the refrigerator and check timezone"
drvc33 7 hours ago 0 replies      
Thanks for this thread. Totally made my day.

Waiting for future slew of news stories regarding "rogue fridges"... some goof drives by a house, jacks the wifi, compromises the fridge.

SixSigma 8 hours ago 0 replies      
Welcome to the Internet of Things

20 November 2014 - problem reported on fridges costing in excess of $2,000


23 Feb 2015 - problem on some models resolved


4 June 2014 - original poster from 20 Nov has working fridge


Dec 2015 - still not fixed for all fridges

smegel 7 hours ago 0 replies      
If ever there was a headline made for Daring Fireball...
AndrewKemendo 8 hours ago 0 replies      
Sounds like your fridge needs a software update

What a time to be alive

sliverstorm 9 hours ago 3 replies      
Well, at least a calendar is fitting for a refrigerator, as opposed to a twitter client or whatever.
rasz_pl 6 hours ago 0 replies      
Samsung employs tens of thousands of software engineers, all working in small isolated groups. One hand has no clue what the other is doing. Hardcoded, outdated, proprietary and broken are just a few selling points!

TV that lets your neighbour remotely enable buildin webcam in the middle of the night? Wifi enabled digital camera running ancient linux emailing pictures in the clear? Thats Samsung.

andrewvijay 8 hours ago 1 reply      
First world problems
unixhero 8 hours ago 0 replies      
vicfcs 8 hours ago 0 replies      
did you reboot your refrigerator ? maybe you need a firmware upgrade.
JustSomeNobody 8 hours ago 2 replies      
Really? You don't think <insert popular black, asian, latino, etc name here> is rockin' a fridge that speaks internet?
smoreilly 9 hours ago 1 reply      
carterh062 8 hours ago 0 replies      
This is the kind of stuff that should be number on HN everyday.
Anki a program which makes remembering things easy ankisrs.net
390 points by sonabinu  1 day ago   202 comments top 35
gjcourt 1 day ago 12 replies      
A friend and I decided to build an Anki replacement a few years ago, it's now located at https://www.memorangapp.com and has a loyal following of 10s of thousands of users. Anki is a great tool and has a wide variety of content on Ankiweb, however the ecosystem doesn't allow for any collaboration or the ability to keep content up to date (aside from manually editing decks, but where for example can you suggest a correction?).

Memorang was designed from the ground up to be the next generation adaptive learning platform. In fact many hard-core Anki users have switched over to our platform and love it! The main difference is in the ease of use of both creating and consuming content. Anki has the underpinnings of supermemo, which is designed for the ideal learner. In a couple of experiments that we've done partnered with large institutions, one still underway, we have shown that most students are not "ideal learners" in that most still cram right before big exams or deadlines. The original study focused on efficacy, you can find the slides we did for an NSF sponsored presentation here http://www.slideshare.net/gjcourt/memorang-nsf-mooc-2014. The latest study is still in the works, but will involve significantly more data.

Anki is a fantastic program and many people love and use it everyday. If you have loved and used Anki as much as we have, then give Memorang a try and see what you think! (or come help us improve the future of learning https://www.memorangapp.com/jobs)

Edit: Read about our data-model here http://blog.memorangapp.com/post/108094496626/tags-more-how-...

cblop 1 day ago 14 replies      
I used Anki for 30 minutes to an hour each day for about a year to memorise around 1500 Japanese characters and their readings, with the aid of a mnemonic technique. So long as you have an hour a day to dedicate to the flashcards without fail, you will retain the memorised information.

My problem was that a job and location change altered my daily routine entirely, so I stopped reviewing the cards. One year later, I have forgotten most of the characters I knew. However, the characters I learned while living in Japan, in context, are still fresh in my mind.

I think that spaced repetition isn't the memory panacea it's always touted to be. It's a great tool for cramming, but soon becomes a pain when you have hundreds of cards to review every day. I've heard good things about the goldlist method, a much more low-tech pen and paper approach. Does anyone have any experience with this technique? It claims to be better for long-term memory: http://huliganov.tv/goldlist-eu/

tormeh 1 day ago 4 replies      
>It's highly recommended that you install Anki from this package instead of relying on the version distributed with your OS, as the packages in the official repo are often very out of date.

Why is this still a problem in Linux-land? Why can't we just make some but not all packages rolling-release? Making Anki rolling-release won't make the distro unstable by any sensible meaning of the word.

jhund 1 day ago 2 replies      
Anki is great. I've used it to learn the 100 most common Clojure expressions. You can get the Anki file by signing up for the newsletter at http://www.lispcast.com/100-most-used-clojure-expressions
reinhardt1053 1 day ago 1 reply      
I would like to recommend Memrise [1] as alternative to Anki.

[1] http://www.memrise.com/

sandGorgon 1 day ago 1 reply      
Anki is great - the original Supermemo algorithm on which Anki is based[2], makes for fascinating reading [1]

However, I daresay that this is a usecase that is tailormade for the mobile rather than the desktop in any which way.

I wonder why the core team is spending effort behind a desktop app, rather than go full mobile.

[1] https://archive.wired.com/medtech/health/magazine/16-05/ff_w...

[2] https://www.supermemo.com/help/smalg.htm#Anki_will_work_grea...

syntiux 1 day ago 0 replies      
Anki is great. I have been using it to prepare for my exams. Creating flashcards on my laptop and learning with them on my Android phone / tablet works like a charm. Also, the Android app just got updated. The only limitation I've found is that there is currently no way to self-host flashcards which can be accessed by the mobile app. If you're only using the desktop version you might want to look at https://github.com/dsnopek/anki-sync-server in order to self-host your flashcards.
markbao 1 day ago 0 replies      
I use Anki as a part of my college strategy (http://markbao.com/notes/college-strategies) by using the Cornell notetaking method to write up questions and answers for my notes and piping those 'cues' into Anki. Turns out self-testing is one of the best ways to learn, so this combination is surprisingly effective.

It's no replacement for actually learning the concepts, but they help in drilling down the foundational knowledge that helps me understand the bigger-picture concepts better.

omarish 1 day ago 0 replies      
I've been using Anki on my iPhone for the past 3 years and it's been quite effective. Highly recommend it for anybody who has a lot of content to learn. It's really useful if you can make time for it on a daily basis.
markm248 1 day ago 2 replies      
Remember that you read it here first, there will be a unicorn built on the concept of SRS. I use Anki daily and love the application but there's a learning curve and you have to build your own workflow to make it work. Someone is going to take the concept and make it accessible to the masses.
dorfsmay 1 day ago 0 replies      
I'm surprised there's no mention of Super Memo and P. Woniak's studies on spaced repetition in this thread:



hammerbrostime 23 hours ago 0 replies      
I love Anki, been using it for the last year.What has been most powerful for me, is that I no longer take standard notes when trying to learn a new topic. Instead, all of my notes are in "question" form, so that I can keep on top of it over time. The spaced repetition keeps my learning efforts efficient and keeps things from leaking out of my wetware memory.

The synchronization feature between devices has also proven to be very useful, as I use my laptop usually to put in new questions, but use the iPhone app to quiz myself whenever I have a few spare minutes during the day (waiting in line, walking to work, etc).

UI is a little wonky at times, but its features and flexibility more than makes up for it for a dedicated student.

Shank 1 day ago 0 replies      
I've used AnkiSRS and recommended them, but I still don't have a good solution to catch up on decks that you're behind on. It's very good for keeping memory in tact with lots of info.

Gwern has an excellent overview of spatial repetition software too: http://www.gwern.net/Spaced%20repetition

EvanL 1 day ago 1 reply      
Someone please create a browser extension that automatically adds every word you "Look Up" to an Anki stack. My vocabulary would be vastly improved if I had this, I always find myself looking up the definition of the same complex words over and over again that are for whatever reason slippery for me to remember.
bigethan 1 day ago 1 reply      
I'm loving learning Arabic with http://orangeorapple.com/Flashcards/

The website looks underwhelming, but the app is amazing. The more I use it, the more it's flexibility impresses me. There are lots of options to review the cards, and it'll use card data to make quizzes, etc for you.

tangled_zans 1 day ago 1 reply      
Ask HN: What's the coolest thing you've used Anki for?
xavi 1 day ago 1 reply      
Anki has many features but I found it too complicated and I didn't like the UI, nor having to sync across devices. So I decided to develop a basic, simpler alternative, a web app that can also be installed in the home screen of an iOS or Android device, https://omnimemory.com
antoaravinth 20 hours ago 0 replies      
Anki is an great tool. I used to read javascript Coercion algorithm from JS Specs using this little tool. It have immensely helped me to remember almost everything!
mtau 1 day ago 0 replies      
I would like to recommend Mnemosyne [1] as an alternative to Anki. It uses a very similar algorithm for spaced repetition (based on SM2), but has a simpler and cleaner UI.

[1] http://mnemosyne-proj.org/

aeris 1 day ago 0 replies      
There's also a web interface


stewbrew 1 day ago 3 replies      
The problem with every app like this is the lack of good quality material. For language learning, I prefer learning whole sentences as examples of an appropriate context. But creating good lists of sentences & translations is cumbersome and just throwing a tool at the "crowd" doesn't automatically get you quality content.

That said from all these tools I prefer anki since it has a web-based UI and an android app.

sriram_malhar 14 hours ago 0 replies      
I couldn't get past the awful UI, written using a generic portable UI toolkit which makes it uniformly terribly looking on all platforms.
currentoor 1 day ago 0 replies      
I've been working on a similar concept after using Anki for a year. http://www.looprecall.com/

I wanted to have a more modern UI and plan on adding the ability to share decks between users. I would welcome any feed back! :-)

a-dub 1 day ago 0 replies      
I used this for foreign language vocabulary when I was in school. Did it work? I honestly don't remember...
kiba 1 day ago 0 replies      
I spent using Anki, and all I have to say is that my graphs actually dovetail neatly.

Space repetition just works.

roflmyeggo 1 day ago 1 reply      
I've been using the Anki iOS application for a while now. I'm considering moving my stack over to org-drill.el to minimze context switching. Anyone made a similar switch or can compare the two at all?
thewhitetulip 22 hours ago 0 replies      
Anki has a nice android app too by the way
746F7475 1 day ago 1 reply      
I remember trying to use this for school stuff, but I didn't find it all that useful. There is after all very little you have to remember that isn't about schedules
ixtli 1 day ago 0 replies      
Anki is a very impressive project if for no other reason than its constant development. It basically got me a degree in Japanese seven years ago and it's still being polished.
wahnfrieden 1 day ago 3 replies      
The iPhone app came years after the free and open source desktop one; the boat had long since sailed on charging on the desktop. The Android one is unofficial.

I recall reading that because they didn't make the app friendly to casual newcomers, they didn't pick too low a price-point that would attract them and their support load.

That said, complaining about an app's price on HN is pretty disappointing. Though they might have found more success with a recurring/IAP approach that makes payments more palatable.

Seems development has largely stalled on the iPhone app though.

outlace 1 day ago 0 replies      
Anki is what's getting me through med school
pmoriarty 1 day ago 0 replies      
Also see the mentat wiki:


asimjalis 1 day ago 0 replies      
A simple screenshot on the landing page would make this a lot easier to immediately understand.
meesterdude 1 day ago 2 replies      
This is really great! I'm building a SaaS that is similar to this, but I was feeling bummed about not releasing it as open source; but now I can just point people to this if they want to run their own. awesome!
GeorgeMatthews 1 day ago 0 replies      
That's what everyone needs in a busy schedule
US to ban soaps and other products containing microbeads theguardian.com
293 points by benologist  3 days ago   219 comments top 17
SwellJoe 3 days ago 11 replies      
While it's cool and all that this has been done, it's many years after credible people started warning about the dangers of these products. The fact that manufacturers kept making this stuff with knowledge of the damage it was causing is pretty damning, and is the kind of thing that makes me mistrustful of arguments that the free market will sort things like this out. There's just no good mechanism to stop massive harm to common resources (even now, we don't have a good mechanism to stop this kind of thing, since it took years to do so, even for something as uncontroversial as this...our system barely even puts a dent in the ecological harm of fossil fuels, animal agriculture, etc.).
bhartzer 3 days ago 3 replies      
I am happy that small, but fairly substantive (and meaningful) policies like this still manage to get through Congress. And that not everything becomes like that "feds are confiscating our light bulbs" fight because of posturing or corruption.

This is a good regulation/law, and it's actually going to be passed. Glad it still happens sometimes.

krschultz 3 days ago 4 replies      
This was an obviously bad idea from the start.

One of the tenants of a reasonably modern mechanical engineering curriculum is at least passing coverage of 'product lifecycle', which basically means thinking about disposal/recycling during design. What did the CPG companies think was going to happen when they dumped a billion tiny little pieces of plastic directly down the drain?

merpnderp 3 days ago 1 reply      
Forget about these beads winding up in streams and our ecosystem, which is bad enough. Dentists warn that toothpastes with these beads can embed them in your gums leading to infections and tooth decay.

No idea how this became legal. Our regulatory body should be a whole lot more about transparency, so questionable stuff like this can be easily weighed by the consumer.


kileywm 3 days ago 0 replies      
I've used some of the products containing microbeads and was flabbergasted when I learned that they were plastic. It was my dentist who asked a pointed question during a routine cleaning:

"Do you use Crest toothpaste?"

I answered yes and was informed that some of the plastic beads from the tooth paste had gotten stuck in my gums. The dentist went on to explain that she'd sent in many complaints to the company but they still didn't drop that ingredient.

What weighs on my mind now is why a toothpaste company continued to use microbeads after learning of the externality. Did they have too much inventory to sell. Were they afraid that discontinuing that exact product would upset customers? Did they think that owning up to the mistake and correcting it would draw too much attention to the mistake?

Personally, my respect lies with any entity that can own up to and correct its mistakes before it is forced to do so.

bhouston 3 days ago 5 replies      
Easy solution, just switch to natural biodegradable abrasives like crushed walnut shells. These are already used a lot in the cosmetics industry.
xixi77 3 days ago 3 replies      
Just curious -- what is the main problem with these (as opposed to say sand and other natural objects of similar size) -- do they float in water instead of settling down, or look like food, or something else?
brokentone 3 days ago 0 replies      
There are a number of natural "beads" regularly used in alternative products. Likely a higher pricepoint, but clearly a better idea. e.g. http://www.kiehls.com/mens-oil-eliminator-deep-cleansing-exf...
castratikron 3 days ago 0 replies      
Microbeads were banned in Minnesota several months ago.


buckbova 3 days ago 1 reply      
What about pumice soaps, don't these particles float as well? Probably too abrasive to have widespread use outside of some hand soaps.
azinman2 3 days ago 0 replies      

How people aren't able to be put in jail for this kind of idiotic stuff in the first place I'll never understand. Our priorities in our legal system don't really accommodate the destruction that can be caused by normal everyday products at scale.

wiradikusuma 2 days ago 0 replies      
Holy batman, I didnt know (certain type of) microbeads are made of plastic. Everybody knows plastic in general is non-biodegradable etc etc, and they're making it in small particles? Who in their right mind?

What I don't understand is, they get the profit, everyone else clean their shit. And they can get away with this? Sigh

jackcosgrove 3 days ago 0 replies      
Not only are these environmentally harmful, but they get stuck in your ear after you shower with them and then your doctor has to refer you to an ENT costing time and money to vacuum them out. Not to mention the psychological stress when your doctor tells you you have a blue growth in your ear and they don't know what it is. Or so I've heard this sort of thing can happen...
sosuke 3 days ago 0 replies      
Why 2017? That seems very far off.
aaron695 3 days ago 0 replies      
Anyone who cares about science rather than religion want to link about issues with microbeads here?

Can't say I've ever seen any, other than the normal anti-science, stuff is 'scary' point.

That alone annoys me with this conversation, how about starting with a real reason this is bad rather than we can track it.

donatj 3 days ago 0 replies      
What's wrong with pumice soap that we need plastic?
adrusi 3 days ago 10 replies      
Great, but now what do I do when my hands are covered in tree sap and normal soap isn't working? I guess we'll just have to settle for a more environmentally friendly alternative like acetone.
How I wrote a self-hosting C compiler in 40 days sigbus.info
381 points by rui314  13 hours ago   103 comments top 17
bad-joke 10 hours ago 2 replies      
I really enjoyed reading this. It's informative, fun, and has a refreshingly honest tone. Too often, stories passed around by computer scientists entail clever solutions and elegant insight striking the protagonist like lightning in the hour of need. Rarely does the programmer express regret, make self-corrections, and confront fear and doubt along the way:

>I should have written beautiful code from the beginning, but because I needed to learn by writing a working code, the rewriting was unavoidable.

>I should probably change my mind to implement all the features from end to end. I may find it fun as I'm approaching the goal. Sometimes, I have to write more code than I want to write in order to achieve a goal.

>In a tough situation like this, I probably should recall the fact that the compiler was just in one file, to see how much progress I've made in a month. It just reads an integer with scanf() and prints it out with printf(). Really, I made so much progress in one month. Yeah, I think I can do this.

jaybosamiya 11 hours ago 1 reply      
> I suspect that he [Dennis Ritchie] invented a syntax, wrote code for it, which turned out to be more complicated than he had expected. And that was eventually standardized by the ANSI committee. It's hard to implement a standardized language because you have to get everything right. It's rather easy to write your own toy language.

Love those lines

draugadrotten 11 hours ago 0 replies      
Interesting, and quite funny to read with a sense of humour that reminds me of the movie PI. The author of the compiler goes from rational to something more... spiritual.

"Day 52

I was looking for a mysterious bug for three days: ..."


xigency 7 hours ago 2 replies      
Honestly, the most difficult, time consuming, and mundane aspect to this project would have to be the parser, which was apparently written in C by hand. So bravo.

Getting to some of the final notes:

> ... I'd choose a different design than that if I were to write it again. Particularly, I'd use yacc instead of writing a parser by hand and introduce an intermediate language early on.

That's why I found the LALRPOP post by one of the Rust developers interesting. Writing your own parser generator is actually much easier than writing a parser by-hand (depending on the complexity of the language, here not that complex and still difficult), and I think it's more instructive than using a free or open parser-generator or compiler compiler. The downside is that it is less practical, because almost none of the important aspects of language implementation involve the parser.

tomcam 12 hours ago 1 reply      
I was a little surprised that the author was able to manage both C11 and the preprocessor in that time. The preprocessor is hard. But there was existing code from a previous version of it, which makes sense. Still, a fantastic achievement! Congrats to the author!
peterkelly 10 hours ago 5 replies      
For anyone interested in compiler writing and looking for a good resource to start, probably one of the best is the "Dragon Book":


I highly recommend it, but it's heavy stuff. There are probably simpler guides out there that just cover the basics.

yeison 11 hours ago 2 replies      
What does it mean to be 'self-hosting' here? Does it just mean that it's a compiler that can compile itself?
jlappi 12 hours ago 0 replies      
This is really interesting, and I was glad to see the author go beyond just the stated 40 days and give insight into where they went after it was 'self-hosting.'
peter303 10 hours ago 6 replies      
Long ago UNIX had compiler writing tools like yacc and lex. I wonder if they are useful for exercises like this.
pagade 9 hours ago 1 reply      
Anyone tried using it? How do I use it to generate executable (as per the code it should fork the 'as')?

Getting following error:[ERROR] main.c:144: (null): One of -a, -c, -E or -S must be specified

-c, -E and -S are working fine. Couldn't figure out from code what -a does.

sdegutis 12 hours ago 4 replies      
Thought this was going to be an inspiration to me to continue with my pet project of writing my own little programming language. But it starts off on day 8 with him already having written a basic compiler, with no explanation of how he did any of the basics. Still interesting, just not what I thought it was.
sabujp 9 hours ago 1 reply      
1) write compiler2) get a job at google..4) profit
allannienhuis 9 hours ago 0 replies      
> I have a mixed feeling I learned a new stuff, but I could have learned that without spending this much time.

Story of my life!

andrewchambers 8 hours ago 0 replies      
rui314, thanks for posting this, you really were such a big inspiration to me writing my own. You were really quick!
andrewvijay 8 hours ago 0 replies      
Unbelievable Jeff!
pskocik 10 hours ago 2 replies      
Now repeat for C++.
pinn4242 8 hours ago 1 reply      
Self-hosting is pointless. Go has it--who cares. I wrote a program 1000x faster than the ruby one (at work) with zero bugs in Go, but I still don't want to use it(Go). Java is fine. Do I care if Java is self-hosting? No. I'll do (another) language in Javacc (my first one is still awesome) or ANTLR.
New Style of Police Training Aims to Produce Guardians, Not Warriors washingtonpost.com
278 points by tokenadult  4 days ago   210 comments top 29
darkr 4 days ago 6 replies      
My first experience of American policing in action was at a hotel in Gainsville, most of which was booked out to people attending the same punk rock festival. There was a whole bunch of people hanging out by the pool in the summer afternoon, playing music of a ghetto blaster, drinking out of kegs and just generally hanging out. I think someone may have let off a firework; but the general scene was on the mellow side of rowdy.

Around 8 or so police suddenly burst out into the patio, guns drawn, shouting at people, throwing them face down onto the floor and handcuffing them. The whole thing would have been farcical if there wasn't such an underlying threat of mortal violence.

Contrast to a scene in Whitechapel, London a couple of years later; a roof party on top of a tower block. Considerably larger sound system and bigger crowd. Unsurprisingly, around 3AM one of the residents made a noise complaint.

Couple of bobbies turn up, as usual, unarmed apart from the standard issue baton; ask to speak to the organisers, give it bit of "well well well, what's going on 'ere then". Some banter ensues and a polite but assertive request to turn down the music and disperse is made, which is duely acknowledged and the party winds down/moves elsewhere.

exelius 4 days ago 12 replies      
I see a disconnect between what the public believes the job of the police is and what police officers believe the job of the police is.

In my mind, police officers sign up for a dangerous job knowingly - that's why they're revered as heroic by many in society. They are willing to put themselves at risk to protect society from harm, and that's noble.

What most police academies teach is the opposite. They teach officers to put their own safety over that of everyone else. They talk in terms of "reducing officer casualties" and "overwhelming force". This is not a heroic police officer sacrificing himself for society; it's jackbooted thugs threatening, terrorizing and killing the very civilians they are supposed to protect. Turns out it's really hard to tell a bad guy from a good guy by looking at them, so they play it safe and treat everyone like a bad guy.

Not every officer thinks this way, but it's common enough that it leads to a lot of police shootings. IMO a police officer should be more willing to take a bullet than put one into someone. I realize body armor isn't foolproof, but the job is inherently dangerous and the officer goes into it knowing that - something a person on the street doesn't get.

gherkin0 4 days ago 1 reply      
> Gone, too, is a classroom poster that once warned recruits that officers killed in the line of duty use less force than their peers.

Wow. That was a thing that existed?

> Alexis Artwohl, a former police psychologist and consultant to the International Law Enforcement Educators and Trainers Association...is skeptical of some guardian-style training. Artwohl has co-written a book on deadly force whose promotional blurb begins: In a cops world its kill or be killed.


I hope this new-style training actually has an impact.

stcredzero 4 days ago 1 reply      
My ex-girlfriend's uncle is an honest man, has a stable job, is in a stable marriage with a wonderful woman, and has two kids. At the time of this story, his kids were in High school and on the junior varsity and varsity football teams, and were getting good grades. He and his family are devoutly religious. In his family, young men are expected to be celibate until they get married. He lives in a small river town in Louisiana.

The police officers of his town are almost all white, and this is how they treat him: If they see him and his sons conversing in a parking lot, it is assumed that they are planning to break the law, and they are always told summarily to leave. They could talk to him and be civil. They could even ask him for useful information. However, they only see young black men who might be perpetrators.

I've also seen the police of this town herd crowds of black people on 4x4 vehicles like cattle on the 4th of July. They were clearly afraid of the crowd, which mostly consisted of high school students. Kids, really!

There are American police who are afraid of their citizens, and only see them in terms of their ethnicity and potential danger. Something is very wrong here!

mcguire 4 days ago 1 reply      
"In 1986, two FBI agents armed with six-shot revolvers died in a shootout near Miami with bank robbers armed with more powerful weapons, including a semiautomatic assault rifle."

There's quite a bit more to that story.


The bottom line is that one suspect (the other fired only one shot causing no injuries), after receiving a fatal wound, killed two FBI agents and wounded five more.

These are parts 1 and 2 of an FBI training video on the incident:



This incident seems to be so obviously a fluke that it would be hard to find any generally applicable lessons, although as the article points out, lessons have been found from it.

(One was the introduction of the 10mm pistol round and pistols chambered for it---that didn't last too long as the recoil was deemed too heavy for accuracy and the pistols seem to have suffered failures---and then the move to the .40 calibre S&W round (based on the 10mm with a reduced powder charge).)

kelukelugames 4 days ago 1 reply      
Don't cheer yet. This is how Seattle cops react to training. Go to the 1:22 mark. Cop talks about sticking a gun in someone's face as his method of de-escalation.


Animats 4 days ago 2 replies      
This isn't a new style of policing. It's Sir Robert Peel's principles of law enforcement, from 1822.[1] The UK still tries to follow those.

The militarization of American police is a reaction to the rise in gun ownership, especially guns with higher firepower. US police have to assume when they approach someone that they may be armed. UK police don't.

[1] https://en.wikipedia.org/wiki/Peelian_Principles

logn 4 days ago 3 replies      
> Artwohl compares police work to defensive driving, which is about expecting something bad is going to happen. Its not about dealing with normal traffic flow.

> We should go out there and expect something bad will happen and watch for it, she said. If we are not paying attention, we could die.

If that's how she views defensive driving it might shed some light on her approach to policing.

I don't think defensive driving means expecting that at any moment something horrible can happen. It's more about being aware, anticipating events, preventing problems, and having a plan B. And it's very much part of dealing with normal traffic flow, responding to common road conditions in a defensive way that prevents problems or at least offers you a good response should a problem occur.

femto 4 days ago 1 reply      
Maybe each officer needs to be reminded of their own roots, in the form of the Peelian principles? It should be an easy sell, as the principles have been "their own" for nearly 200 years. Dating from 1829, they are the founding principles of the London Metropolitan Police Force. In summary, policing is a self-regulation function by the community being policed, not imposed by an external force.


zeveb 4 days ago 0 replies      
I wholeheartedly approve of this shift in focus. Yes, police do have to deal with violent and dangerous people, but most people are neither violent nor dangerous (nor ought they be criminals). The police force is not the military; one's fellow citizens are not one's foes.

> The officers were charged with felony assault but acquitted by a jury in 1992, sparking days of rioting and protests.

For completeness, the article really should have mentioned that they were then convicted by a federal court, the Constitution's prohibition on double jeopardy being held not to apply.

brandonmenc 4 days ago 3 replies      
"Policing" and "law enforcement" are different.

Policing: hey, you're not allowed to jaywalk, so stop.

Law Enforcement: here's a $100 fine for your first jaywalking offense.

We have too many law enforcers, too few police.

tokenadult 4 days ago 0 replies      
The report "From Warriors to Guardians: Recommitting American Police Culture to Democratic Ideals" mentioned in the article is available for downloading from the National Criminal Justice Reference Service.[1] The report includes a lot of information about police training.

[1] https://www.ncjrs.gov/pdffiles1/nij/248654.pdf

sremani 4 days ago 0 replies      
What I like about US or West is, reflect, retrain and retool. That is absolutely missing in other parts of the world.I know it is early and a bit congratulatory, being an Immigrant and having seen the other world, I am happy to call US my home.
stcredzero 4 days ago 1 reply      
I was watching a video, where the YouTuber, who is a cop, was trying out a video-wall computerized training tool for police. One thing that disturbed me was that the simulator was supposed to help police know when to deploy their firearm. However, every single scenario required the use of lethal force. The only question was when to use lethal force.
eagsalazar2 3 days ago 0 replies      
I have a friend who went through the Seattle academy training then left the SPD after a year. He said this attitude runs very deep. Citizens are seen as the enemy and it is a bunch of insecure, macho dudes who get off on intimidating people.

I think some of the reason people disagree on this subject is that police across the country vary a lot. In SF where I live now police are, to most people anyway, pretty good. Growing up in Seattle and living there until I was about 35, I was harassed and intimidated many many times.

jessaustin 4 days ago 0 replies      
I'm no expert, but reading TFA I can't help but think that though this new training is an improvement, it's not going to make much difference. It would take a saintly person not to menace and abuse the public when placed in the position in which we place cops. As long as we rely on the goodness of people, on the availability of "heroes", we'll be disappointed. We'll still have a system that rewards corrupt and brutal behavior while imprisoning people who smoke pot.

This thread is a perfect illustration why. We have numerous comments claiming that police should just put up with serious risks to their lives, which is ridiculous. The risks that e.g. deepwater oil workers face are inherent to current oil drilling technology. Though such workers are paid enough to accept those risks, safety technology is always improving. In contrast, every risk involved in police work is due to arbitrary decisions society has made, which could be changed at any time if we cared enough to do so. It's no wonder that police have responded with a bunker mentality.

Society has invented numerous victimless crimes, many of which attempt to counteract basic psychological and physical drives. Society has outlawed "risky" behaviors rather than punishing actual harms caused by those. Society has given municipalities, agencies, and contractors commercial interest in draconian enforcement. Society has decided to employ multitudes more police than we actually need, so that they are forced to menace the public in order to make work for themselves.

My suggestion for a maxim of policing would be to follow physicians: "first, do no harm". Actually living that rule would be a vast change to current LEO practice.

unabridged 4 days ago 2 replies      
The best way to get better cops:Putting multiple cameras on every cop & car, and prosecutors who make it clear they side with the people over the police. Seriously punish any amount of power tripping or harassment of citizens.

It may take a few years, but the people who sign up for police academy will be different. Being a police officer will no longer be attractive to those who want to wield power.

ska 4 days ago 0 replies      
Isn't this more like returning to an "old style"?
jessaustin 4 days ago 1 reply      
So far this year, police have shot and killed more than 900 people, according to a Washington Post database tracking such shootings more than twice the number recorded in any previous year by federal officials.

Surely this indicates the utter failure by any federal agency to track this statistic more than it does exceptionally violent LEOs this year?

hyperion2010 4 days ago 0 replies      
Far better for the authority of a policeman to come from citizens' belief in the importance of law and respect for their governmental institutions than from fear of deadly force. Even if the rule of law is ultimately guaranteed by force it is a really bad idea to continually confront generally law abiding citizens with that fact, they are going to follow the law anyway most of the time and pointing guns at them is just going to escalate the situation because the one pointing a gun is seen as an imminent threat and 'other.' Dangerous for everyone involved and stupid if you care about building belief in the importance of rule of law.
sahreeg 3 days ago 1 reply      
Although I am no authority to properly comment on the American police system; having never lived there. I think we should not boil it down to a black-white issue of police training. I definitely agree that training police should not be in the same fashion as military, I think this needs to be coupled with anti-gun laws, and proper education to ever see a true decline in police brutality. After all, being confronted with an armed suspect, will lead to escalation, remove that game changer, and a properly trained police should be able to subdue them.
kevinpet 4 days ago 0 replies      
I remember when the Abu Ghraib and similar revelations came out, I was relatively unsurprised. Military training is to prepare people to fight a war, against a defined enemy, where the general gist of the available tactics all come down to "kill the other guy first". And if you put someone with that kind of training in a situation that calls for police, you're going to get bad results.

I had naively assumed at the time that police were still trained to a different outlook and set of priorities.

jqm 4 days ago 1 reply      
This is great, but I wonder if without a pay raise if they will be able to fill the ranks.

Cynical me thinks maybe a large number of current cops would do the job for even less, so long as they were allowed to carry a gun and a stick and order people around and maybe even get to shoot them.

So with that approach we get a less expensive police force that is fairly effective (for the privileged class) as it ruthlessly beats and suppresses the unprivileged class. Win-win for the privileged class. Keeping the thugs down at bargain prices.

So now policing gets more expensive? Many cities are strapped for cash at the moment. (Heaven forbid we consider cutting unneeded bureaucrats to pay for better police officers). So higher taxes it is? I'm certainly ok with paying for quality. That is... if I actually get it.

calibraxis 4 days ago 0 replies      
This article shows how far ahead some people (in the Mideast) are:

"They number roughly 6,000 [police] officers, all of them elected; a women-only force deals with sexual assault and rape. (All recruits receive their weapons only after 'two weeks of feminist instruction,' according to Cengidar Mikail, the director of the Qamishli police.)" (http://www.nytimes.com/2015/11/29/magazine/a-dream-of-utopia...)

What happens when people dare to think deeper. And they're not only in ISIS' backyard, but the ones beating them.

blhack 4 days ago 0 replies      
Police need to just behave more like firefighters.

Not only that, but I think that a lot more cops need to be out on foot. Not bikes, not segways, none of that. They should be out walking around, talking to people. IMHO, I should know who the cops assigned to my neighborhood are.

Here's a thing that happened last week that highlights a problem I have with the cops:

At about 1:30 in the morning, my dog starts doing her dog thing and informing me that IMMINENT DOOM is upon us in the form of somebody being outside of the house. I get up and walk to the kitchen to find that, yes, indeed somebody is outside of the door trying to get in, which is a scary feeling. I don't have a peephole on that door, it's the middle of the night, and they aren't knocking, they're trying to get the handle to open.

Now, luckily I'm a pretty huge guy, so while this was scary, it didn't really seem life threatening, (they weren't trying to bust through the door, just trying to come inside, so probably just a very disoriented person). Eventually, they left, and I went back to bed.

A few minutes later, however, we heard a car alarm go off next door, indicating that the person had just moved on to the neighbor's house, which is really sad, because there is an old lady that lives next door, who might not brush off the idea of somebody trying to come into her house as readily as I could.

I go outside and find out that the person trying to come in was a ~20 year old girl who couldn't have weighed more than 110lbs soaking wet. Basically the least threatening person imaginable, but she was trying to get into the neighbor's house.

One part of my life involves volunteering for a group of people who deal specifically with this sort of thing at a big dessert party that lots of people in SF have probably heard of. My mode switched from being worried about the lady next door, to being worried about the obviously confused kid trying to get into somebody's house.

I loudly convinced her to come and talk to me away from the lady-next-door's house (so as to make sure that the lady inside, who I am sure was scared, could hear that everything was alright), and we started trying to figure out where she was supposed to be, and a plan to get her to that place safely. Sidenote: turns out she just has some really shitty friends who more or less ditched her and went home.

While I was talking to her, the cops showed up (presumably my neighbor called them), and I got to see how they would have handled the situation.

There was a guy just riding his bike by the neighbor's house, and the cops started YELLING at him

"What are you doing?"

"Just riding home."

"Huh, why are you here? Why are you riding here? Huh? Why here? What are you doing? Is this your house? What are you doing? Do you know you can't be here? This is an alleyway you can't ride here [EVERYBODY rides their bike in the alley, which is practically a bike path], why are you here? This is illegal. What are you doing here?"

Just started machine-gunning questions at this poor dude who happened to be riding by at the wrong time.

Eventually, they figured out that the girl I was talking to was the person that they were looking for. I explained to them who she was, what she was doing, and where she needed to be.

The thing that absolutely FLOORED me was that they refused to give her a ride home. They wanted to stick this obviously disoriented, possibly drugged, girl into a cab (a fucking CAB! Yikes!), and make her into the cabby's problem (hey, get into this random car and hopefully this drugged up girl will make it home safe!)

Luckily they did NOT do this, because the girl said she didn't have any cash. The ended up calling some sort of non-police-police van who gave her a ride to [hopefully] her house. (It was their Crisis Intervention Team, I think. Like people who show up and talk to people who have just had a traumatic experience, I guess they weren't busy, and had time to give this girl a ride).


Watching the whole thing was just sad to me. Not only was the FIRST response that the cops had to start yelling at some dude, but when presented with a REAL opportunity to improve somebody's safety (this girl), they either didn't want to, or were not allowed to.

Keeping some disoriented girl safe in the middle of the night seems like the cliche of what cops are supposed to be doing, and here when presented with the opportunity, they wouldn't do it.


allworknoplay 4 days ago 0 replies      
I'm a fan of this video that tears down lousy police training during a tough encounter: https://www.youtube.com/watch?v=F4VeHOkt_o8

I believe they have others, too. Our cops are simply not trained for de-escalation and disarming like they should be.

fapjacks 4 days ago 1 reply      
Holy smokes we need this so badly in the States!
mindslight 4 days ago 0 replies      
This is akin to trying to stop crime by changing the high school curriculum. We already have civil and criminal legal systems that even police are supposedly bound by. Why not fucking apply them?
rms_returns 4 days ago 1 reply      
Reminds me of the movie "Demolition Man". For those who don't know, this is a Sylvester Slallone futuristic movie where a thug named Simon Phoenix wakes up from a cryo-prison somewhere around 2032. But LA is a pacifist utopia now, and the cops are all "nicey nicey" without any weapons with them, so who is going to stop Simon now?

I think, as a society, we are progressing towards that kind of pacifist utopia.

Atlassian gets IPO share price of $27 businessinsider.com
285 points by prostoalex  4 days ago   149 comments top 17
rifung 4 days ago 6 replies      
It's refreshing to see a tech company IPO that isn't VC backed and actually can sustain itself with its profits.

Maybe I'm just naive but I find it strange how so many companies focus so long on acquiring users and not on making money, thus forcing them to continue to get outside funding. For something like Facebook I can understand how this makes sense, but it seems as though this strategy is currently the norm rather than the exception.

richardlblair 4 days ago 4 replies      
Has anyone looked at the prospectus? The founders are going to do very well.

Michael Cannon-BrookesShares - 69,732,090Percentage - 37.7%

Scott FarquharShares - 69,732,090 Percentage - 37.7%

Well done, gents. Well. Fucking. Done.

myth_buster 4 days ago 0 replies      

 Atlassian was founded in 2002, but it hasn't taken any outside investment. The last two funding rounds from Accel and T. Rowe Price were done to let employees sell some of their shares, and Atlassian says the cash wasn't used for operations.
That's impressive!

Pre-IPO discussion https://news.ycombinator.com/item?id=10708908

phantom_oracle 4 days ago 3 replies      
This is interesting because the valuation is:

- Either driven partly by what Slack is valued at (and then you add the other parts of Atlassian to get its total value)

- It really is this valuable, which would make Slacks billion-dollar + valuation a lot more legit than when people were nay-saying about it

- Just investors throwing money around because money is still 'cheap'

I still think that some savvy retail investors understand the tech market a lot more better now. This company is not burning through cash and shooting for super, super long-term growth (Twitter and it's ilk) so this feels more like a Microsoft listing than a Facebook listing.

Good on them for building something legit and something people would pay for (instead of advertising revenue).

ghuntley 4 days ago 1 reply      
There's a global hackathon going on right now at Atlassian. Keep a eye on https://twitter.com/hashtag/shipit33

Memorable tweets so far:

1) Founder of the company last night calling it quits and heading out for a beer. https://twitter.com/mcannonbrookes/status/674734326571921409...

2) Insanely clever marketing by the jira team: https://twitter.com/Atlassian/status/674964465717260288

Reply with your favourites! ;)

medlazik 4 days ago 1 reply      
Maybe HipChat will finally have chat notifications on iOS [0]. A messaging app. Probably one of the many reasons people migrated to Slack.

Meanwhile, latest update reads: "New: Scumbagify support! Prefix any emoticon with (scumbag) to show how you really feel"


neals 4 days ago 1 reply      
Tech IPO's of any kind always have my interest, but I have a hard time finding out what IPO is happening and when it is happening.

How do some of you track these things?

azinman2 4 days ago 9 replies      
How much could they grow? It's not clear to me, unless they suddenly have much more ambitious plans and new categories of product, how big the market is that can sustain infinite growth over time for them. The market is very specialized for them, and if they try going after non tech their products directly then go up against many other players already entrenched (and it's not like JIRA is some shiny beacon of ease and apple-like perceived simplicity -- if they rebuilt from scratch maybe they'd have a better chance)
gohrt 4 days ago 3 replies      
On the one hand, it's great to see a company do so well making solid basic development tools.

On the other hand, I recall that everybody hated using JIRA. It was sooo slooow and so many little input boxes in the UI. Has it improved in the past 5 years?

cooperadymas 4 days ago 1 reply      
As someone who knows relatively little about how the stocks works, and this stage of a business' lifecycle... what benefit does Atlassian get by going public?
michaelcampbell 4 days ago 2 replies      
The ticker is TEAM for those wanting to play along at home.
omginternets 3 days ago 1 reply      
I'm 28 and I've never invested so much as a dime. I quite like Atlassian, and I think I could afford to lose $27 if it came to that.

Forgive me for the really elementary question, but how do I actually go about buying shares?

pkaye 4 days ago 2 replies      
Interesting that it seems they IPO'd in US though they are mostly located in Australia? Can employees in other countries cash out on their stock options or are they compensated in different ways than in the US?
jbrad7354 4 days ago 1 reply      
I like Atlassian and we use JIRA, Confluence and Stash.

Do I think Atlassian is worth $6bn? No. I planned to buy ~100k in stock at the open, saw the $27 open price, and cancelled my trade.

Is this a nice result for the founders? Yes. :)

Starting price on IPO day means little. It was flat the entire day. The way to judge these is in a couple of weeks (2-3 weeks) to see where the market really values it. See RACE for an example. That had a stronger IPO day "pop" and... here we are. :)

jarjoura 3 days ago 0 replies      
As a side question, I wonder if this means GitHub will go for their own IPO.
bjacks 4 days ago 1 reply      
Once a company has gone public is it common for lots of staff who have stock to quit?
rrtwo 4 days ago 0 replies      
Can someone please explain what would be the reasons for Atlassian to IPO? Is it to return money to investors? fund new things (such as?)?
Hi, Im from the games industry. Governments, please stop us positech.co.uk
332 points by smacktoward  1 day ago   226 comments top 37
adrtessier 1 day ago 17 replies      
> "We need to reign this stuff in. Its not just psychological warfare, but warfare where you, the customer, are woefully outgunned, and losing. Some people are losing catastrophically."

I strongly disagree with this statement. I am very aware of the psychology behind marketing and advertising, and anyone with half an interest in it has probably read Vance Packard's The Hidden Persuaders, which was released, oh, 58 years ago. People know what pushes their buttons, too, and we have a name for it: interest. Not "manipulation". Certainly not "psychological warfare".

Marketing and advertising might seem to be refined in this dopamine-heavy, short-attention-span social gaming age, but this shit has been around as long as humanity itself has existed. Some people have better impulse control than others; it's not the state's job to regulate the impulses of their populaces, or regulate systems that know how to manipulate them. It's a slippery slope that would take down the entirety of marketing and advertising as an industry, an industry which is pretty much necessary for the vast majority of the citizenry to even know what is out there to purchase.

Furthermore, how would one even decide where the line between manipulative consumption and social signaling is? Some people want to own all the capital ships; they have some social hierarchy in some subculture standing upon it. Should I regulate that, too, and force them to have the same consumption values and preferences that I do? Should we take this a step further and ban all vices? Let's see how fast American society would fall apart if we banned alcohol again.

FWIW, I hate consumerism and advertising. I block ads everywhere, I hate big branded items, I don't know the last time I bought something from an ad. Just because I feel this way, I don't go around trying to violently force that opinion upon others by begging the state to do it for me.

kozukumi 1 day ago 3 replies      
Hmm this is a tough one. Firstly I should make it clear that my wife works in the games industry. She works on PC, Console and Mobile titles. She has mentioned several times how there is a lot of pressure put on developers and publishers to "catch the whales". They employ psychologists and gambling specialists to make games, mostly mobile but it is also creeping into PC and Console, super addictive.

Personally I am of the opinion that people should be able to do what they want* but my wife is genuinely concerned about just how they go after certain types of people. They do want to get the gambling types. They are the people who spend money. Lots of money. I have seen numbers where people who obviously have more money than we can imagine spending $50-100k on IAPs. Some games even have "limited" IAPs that were designed for just one player as they spend so damn much.

However I can't help but feel there is something else behind this post than what the author seems to spin it as. What I am not too sure but it seems quite the ramble and he doesn't actually have any solid sources or data to back up his claim. Of course neither do I outside of what my wife tells me :)

* Okay so there has to be some limits.

Edit: I would like to make the point clear that while I think people should be able to do whatever they want, I do find this kind of predatory game design unethical. As does my wife. Thankfully nothing she has had to do so far has been totally deplorable.

mschuster91 1 day ago 1 reply      
As someone who worked in the customer-facing ends of the gambling industry (hell, I haven't been able to totally let go, I just ordered a true casino machine for my home... for reverse engineering and pentesting)... that online shit is far worse than "offline" gambling.

With gambling offline, well you have to leave your house, gonna dress up, and people will notice if you fuck up your life by gambling every day. Also, maximum losses per machine and hour are usually regulated and you can have yourself banned from gambling venues.

Online gambling - and online playing of the "free to play" or "pay to win" sort - however, does not offer any of these advantages. Instant pay-ins of five-figure sums (and instant pay-outs, publically shown)... or just the pleasure of 0wning an opponent because you paid for the "by-cash-only" battleship.

No limits at all, disgusting. And no one will see you going down and down, except the pizza delivery guy and the coroner when it's too late.

LukeB_UK 1 day ago 3 replies      
A lot of people here seem to be missing the main point of this post. It's not about crowdfunding or advertising, it's about using hugely manipulative tactics within the games to get people to basically throw money at them.

The reason they're called microtransactions is because they're small, usually priced at a point that's negligible. Using these tactics, they entice people to buy over and over again, each time the spend is a negligible amount but over time it adds up to hundreds or thousands.

TazeTSchnitzel 1 day ago 2 replies      
This post is a bit rambly and doesn't make its case particularly well.

I think the case is better made implicitly by reviews of games like Game of War: Fire Age. They are designed to be addictive and get players to spend literally thousands of pounds (or dollars) on microtransactions. They have ruined lives.

Particular attention should be paid to games which advertise microtransactions to children.

eveningcoffee 1 day ago 0 replies      
We take Facebook stalking to a whole new level. You spend enough money, we will friend you. Not officially, but with a fake account. Maybe its a hot girl who shows too much cleavage? Thats us. We learned as much before friending you, but once you let us in, we have the keys to the kingdom.

Discussed previously https://news.ycombinator.com/item?id=10239931

brohoolio 1 day ago 3 replies      
I was totally and completely addicted to WoW for two years. Almost lost my girlfriend, friends, etc. When I finally quit I had the shakes. Literally.

One thing though is I've never saved more money than when I was playing WoW because I never went out. These newer games do seem to have a different financial model that exploits addiction in potentially a much more dangerous fashion.

wturner 1 day ago 0 replies      
I was momentarily good friends with someone who went on to become a well known entrepreneurs in this space. Right before he got his business off the ground he emphatically told me :"I am going to exploit the living fucking shit out of these people". He wasn't just talking about his users, but the investors and the entire silicon valley ecosystem. It kind of blew my mind when he actually did it. He went on to be extremely successful in the free-to-play space.
marknutter 1 day ago 1 reply      
I really don't understand the opening rant against Star Citizen. This is a game that quite literally would not even have started production if it weren't for crowd sourcing because the big boys are too busy cramming the next Call of Duty down everyone's throats than taking chances on interesting new IPs. If Star Citizen never materializes it will still have done the gaming industry a huge service by signaling to the other companies out there that a huge, verifiable demand exists for that style of game.
jnbiche 1 day ago 2 replies      
Yeah, pretty doubtful of the author's initial claim of being "not a big fan of government regulation in general" and "not a friend of regulation". I can only imagine the views on gambling and drugs of anyone who proposes regulation for video game advertising.

Suffice it to say this guy is a friend of regulation if he's making this proposal. It's a nice little rhetorical device but most of us are smart enough not to be fooled by this.

sandstrom 1 day ago 0 replies      
I think what some are missing here is whom the regulation (of gambling, alcohol/tobacco advertising, lending regulation etc) is ment to protect.

Pulmonary physicians didn't need regulation to cease smoking[1], people with basic financial schooling probably won't get tricked by usury (which, interestingly, has been banned since medieval times but is now permitted in many countries), and so on.

If you're an educated, conscientious person you don't need this (and are sometimes needlessly constrained by it, which can be frustrating). But it was never for you.

It's for the struggling single-parent who would otherwise burn their paycheck on an online casino -- and get through the month with 300% payday loans.

[1] http://onlinelibrary.wiley.com/doi/10.3322/canjclin.36.1.2/p...

rwallace 1 day ago 0 replies      
OP is rolling two separate claims into one.

The first claim is that there is a problem.

Yes, I agree. There is.

The second claim is that the solution is prohibition.

The results of applying that solution to alcohol addiction and then drug addiction were... 'more harmful than beneficial' is much too mild a way to put it. 'Society-wrecking catastrophe' would be more accurate. Given those precedents, any argument that another round of prohibition is the right answer here requires far more justification than OP gives.

marcoperaza 1 day ago 0 replies      
The solution is to teach people to be savvy consumers, to have a skeptical eye to the world, to be able to sort out fact from fiction from opinion, to understand incentives and identify ulterior motives. This skill will never become obsolete. The government can never protect you from everything, but politicians are happy to take your freedom away while they convince you that they can. No consumer protection law can ever supplant caveat emptor.
gus_massa 1 day ago 6 replies      
This idea to stalk customers in Facebook is scary, but why don't other companies (let's say the home appliances manufacteres) do the same trick? Is it effective only for some niches?
JabavuAdams 1 day ago 0 replies      
I'm glad someone called out this "addicting" as a positive adjective nonsense. First of all, the word is "addictive". Second, how is this a positive for the user?

I got addicted to Clash of Clans after starting to play for market research. I spent $400. It didn't hurt me financially, but it was very interesting to observe myself engaging in classic addict behaviour.

"I can buy another gem pack because I'll just come in by transit tomorrow, so I won't have to pay for parking."

"If I bring my lunch tomorrow, then that will cancel this other item."

... of course, knowing that I wouldn't do any of these things...

pandaman 1 day ago 0 replies      
I might be reading this wrong but this gentleman's problem seems to be that some companies make way more money than he thinks is "fair". Because all his sources are about who made how much. If this was similar to gambling he might be able to show a story or two of somebody mortgaging their house and/or losing job, taking a loan from mob and unable to pay it etc. etc. over addiction to Candy Crush as we have plenty of such stories about gambling.

I have never seen anything similar about f2p games. At most you learn about kids dropping out of school to play DOTA/LoL/WoT/etc but this had been happening ever since there had been video games. Same kids dropped out over EQ/UO/WoW, which are not f2p. Heck, a dude dropped out of my school over a single player PC RPG (was either one of M&M or Wizardry games). I never heard of anybody becoming homeless trying to get their Farmville's dog to lv85. All there is are stories about somebody spending thousands of dollars at once to by some magic crystals in some stupid cell phone game. While it looks pretty stupid it does not seem any more danger to society than people buying pieces of painted canvas for millions and even tens of millions at art auctions.

DaveSapien 14 hours ago 0 replies      
I have talked in front of rooms full of my peers about abusing the vulnerable with predatory IAP's. Rooms full of artists (and students) applauded me. While rooms of more 'professional' developers booed me. I found more often than not, the 'professional' people that have a family to feed defended there moral very strongly. I could boil their argument (unfairly?) down to this, its ok to trick the vulnerable out of their money because I need to make money to feed my family. Where the more idealist artisans where more interested in the craft of games and earning the right to make money with good content. Its a weird rational to base your moral actions on.

I count myself as an idealist, I care about people before my own ability to buy things. I don't think I have a right to make a living at what I love, an opportunity for sure, but thinking you have an entitlement to make money at making games seems to lead us to a dark place for us all.

oldmanjay 1 day ago 0 replies      
That this person is putatively in the games industry does not bolster the point of the article, so I have to assume that used as a manipulative aid to help sell the idea therein. Mildly ironic, I suppose, but it's okay to manipulate when you are doing "the right thing" in some minds.
njharman 1 day ago 1 reply      
> have an incredibly fine tuned and skillful marketing department bent on psychological manipulation.


This is not specific to games industry. In fact they are rather late bloomers and amateurish. This is consumerism. This is production driven economy (in which goods are produced and post facto demand is manufactured). This is inevitable in any market with unequal ownership of resources (that is any market system were it's possible to profit or "get ahead", i.e. all of them).

All advertising is this. Almost all marketing is this. It only varies in it's sophistication and subtleness.

I can not fathom why people watch commercials, don't ad block, read advertising based periodicals, go to shopping centers, etc. Why people willingly participate in being psychologically manipulated into beliefs, decisions, and actions.

JabavuAdams 12 hours ago 2 replies      
Terrible article, but raises some good issues. Would like to see a better discussion of those issues.

One problem that I haven't seen mentioned elsewhere is that in my experience a large proportion of people in the games industry are essentially game addicts.

They've forgone other opportunities, higher pay, IRL social interactions, family duties, etc. due to their passion for games.

It's hard to have a serious conversation about games and addiction without examining this further.

erik14th 1 day ago 0 replies      
I'd draw the line at the point where the user/consumer can't easily recognize advertisement as advertisement.

For example, paying people to write blog posts or to friend your client on social media without making the "I'm being paid to do it" part explicit should be illegal.

I know nothing about US law, but shouldn't that be some kind of fraud crime or "confidence trick crime" already?

falcor84 1 day ago 1 reply      
I wonder how i is that South Park did a great episode[0] on this subject of addiction to freemium games, and neither the post nor anyone in over 100 episodes mentioned this. Is South Park completely pass?

[0] https://en.wikipedia.org/wiki/Freemium_Isn%27t_Free

petke 1 day ago 2 replies      
I think there would be a better case for regulating in game purchases. Some kind of limit, or a at least a warning at the beginning so people know what they are in for. "This game contains purchasable items. You may spend more than 10 000 dollars in this game."

Marketing on the other hand is a different story. We cant really ban that.

abalone 1 day ago 0 replies      
He brings up a bunch of things, only one of which applies to Star Citizen: the "addictive" element of collecting things. Star Citizen is a game with genuinely exciting concepts and visuals exhibited in their marketing. The "addictive" thing that appears to have driven part of their $100M fundraise is prepurchasing cool-looking spaceships.

But how would you regulate that? And what would be the impact on Lego kits, Star Wars action figures, Beanie Babies, coins, stamps, and other real-world collectibles? They have the same issues with OCD. Wouldn't it be arbitrary to limit it to software?

If found it interesting that the author left the most potentially regulatable thing about Star Citizen untouched: consumer protections for crowdfunding investments.

xacaxulu 1 day ago 0 replies      
I'm always wary of people who beg for more government control .
swiley 1 day ago 1 reply      
I don't think I've ever thought of "addictive" as a positive attribute to a game. Reddit and HN are addictive, I want entertainment that I can leave when I want and that has depth.
flink127 1 day ago 1 reply      
"You think you are not manipulated by ads? Get real, read some of the latest books on the topic."

Any book recommendations? Sounds like this would be really interesting to read about

foxhop 1 day ago 0 replies      
Where do erotic web camera sites that allow tipping and rake money off the top fit in?

I'm pretty sure lots of people are addicted to these shows.

hackuser 23 hours ago 0 replies      
The article the author links to is more revealing:


meanduck 1 day ago 0 replies      
Only if the regulation get 90+% votes.

I am so waiting for the future when one can switch (mobocratic) government like a house.

m0nty 14 hours ago 0 replies      
NSFW tag would help.
andrewclunn 1 day ago 0 replies      
> "I guess at this point we could just say A fool and his money are soon parted"


microcolonel 1 day ago 2 replies      
Cliff's comparison of Star Citizen's preorder process to gambling, painted in royal we, is obscene.

I don't like to be dismissive, but it sounds like he's mostly upset that their slimy marketing actually works; while meanwhile people haven't even heard of his company.

For kicks I looked at his company's website, and their top two titles are "Democracy 3", and "Big Pharma". i.e. he makes video games about regulation and a highly-regulated industry. After seeing these titles, the attitude makes a lot more sense.

mahouse 1 day ago 0 replies      
This has absolutely no sources. And if you compare what "an anonymous developer" says he does (which may as well be made up) with what a company that spends $40,000,000 in an ad campaign does, you see how this is delusional.
vinceguidry 1 day ago 1 reply      
Look, this is just commerce at work. People are going to use whatever edge they can get to make more money. These edges only go so far. If you think the only reason Game of War is popular because of the million dollar ads, then why isn't every single product, everywhere using these ads?

These games fulfill more than just some need to mindlessly click on shit. Just because doesn't do anything for you, that doesn't make the people it does work for stupid or rubes. As soon as something better comes along, all of these games companies are going to be toast, way, way deader than disco.

Assuming that any massive success is due to nefarious attention-gaming is pretty much categorically wrong. Video gaming, by the standards of global commerce, is a ridiculously tiny niche. There are billions of people who have never played a video game in their entire life. Game of War is the largest player in a sub-niche of this small niche. Big enough to buy ads on the Super Bowl, sure, but that says a lot more about the declining power of television and advertising in general than it does about the size of the gaming market.

That this market needs political attention to curb abuse is utterly laughable. How about we fix oil drilling, or pharmaceutical production first?

PaybackTony 1 day ago 1 reply      
This is jealousy, at the core of it. Not just the gaming industry uses these tactics. I work in an industry that does that all the time (I hate it).

Certain kinds of games shouldn't be regulated, though I would agree that creating fake profiles on social media in an attempt to gather more information from users should be regulated to some extent.

At the end of the day, you have to ask yourself, are you in favor of people having a free will, or do you want to mitigate what a person can and can't do with their free time because in your opinion, it's bad for them?

pascalxus 1 day ago 2 replies      
I don't see any problem with games or any other product spending that much on advertisement if it's beneficial for them. I don't understand what people have against targeted advertisement? So what if the advertisements are tuned for your interests. Wouldn't you rather see relevant ads, rather than the something completely irrelevant to your life? I actually wish, they had more information and were better targetted. Right now, the ads really don't have much of the right information or enough intelligence at all.

Also, since when is it to much to ask people to spend responsibly? I mean, we have licences for driving and licences for fishing. Should we also have a licence to spend? Perhaps, all these so called 'victums' you speak of, should go to the a financial therapy class and their spending should be limited by the government.

If you dont know how to spend your money, you shouldn't be allowed to have a wallet or money.

People need to be responsible for their own spending if they want to grow up and live on their own.

I am Peter Roberts, an immigration lawyer who does work for YC and startups. AMA
308 points by proberts  3 days ago   306 comments top 117
kemitchell 3 days ago 1 reply      
As a fellow (non-immigration) attorney, I'd just like to stress how remarkable and generous is Peter's willingness to do this via HN, even with the very obvious disclaimer. It speaks volumes about his confidence in his expertise and communication, as well as a genuine desire to spread good information around to those who need it.

Neither the dated, often fuzzy rules about practicing law nor lawyers' developed risk consciousness encourages this kind of "innovative" altruism. Instead, they create anxiety that keeps many community-minded attorneys from doing anything like this.

Bravo, Peter. Inspiring.

mtrpcic 3 days ago 1 reply      
I'm currently on an H-1B in California working at a startup. In my spare time, I work on side projects that might provide value to somebody somewhere, but have an operating cost that I would have to cover if I wanted to offer the project as a free service. I'd like to be able to charge for this (or at least provide the option of a "Premium" plan) to supplant the money that I will lose in hosting the platform. I am NOT trying to make this a high revenue generator, and I am NOT trying to supplant my personal income. I'm more than happy to have an LLC or corporation (with a bank account), and all revenue stays within that ecosystem to cover costs. Is this possible?

1. Can I set up a company with zero employees? Since I am on an H-1B, I am not allowed to work for this new company that I would create to house the service.

2. Is there any legal implications for me of doing this? Most of what I have read claim that any additional work is illegal, but I am not trying to get paid. I am just trying to make the service self sufficient so it's not a cost to me. I will not take a paycheck or salary, and will not remove revenue from the account of the Corp/LLC.

3. What other avenues would you recommend for doing something like this? I've heard from many other engineers in the field that they have similar ideas. They want to create things to benefit others, but are not willing to do so if it is a literal cost to them.

billconan 3 days ago 3 replies      
What are the options for an h1b who wants to start up?

we can't lose our jobs to maintain the h1b status. will yc care that we are not working on the idea fulltime by the time of applying to yc? (will certainly quit the job if accepted to yc.)

what are the common attitudes of companies, like google, microsoft, apple, facebook, toward employee moonlighting?

tinbad 3 days ago 3 replies      
Not a question but wanted to point out the L1 visa that is often overlooked by foreign startup founders. It allows for founders/workers of foreign companies to be transferred to a US subsidiary that is majority owned by the foreign entity. I found it a fairly simple and straightforward process that got me from nothing to L1 to Green Card in about 13 months (although with help of immigration lawyer of course). The main requirements are having worked for at least 12 months for the foreign entity before transferring and the person must be in a managerial/executive position. Also the foreign entity must own the majority of the US subsidiary.
leroy_masochist 3 days ago 2 replies      
Say for the sake of argument we're a seed-stage startup and we've identified an engineer with a very specific set of skills -- skills necessary for the growth of our company -- who would need an H1B to work here legally. All-in, about how much will it cost us to get that H1B visa processed through the system in a timely fashion?
ojbyrne 3 days ago 1 reply      
I am curious about the approach YC takes for foreign founders who are accepted into YC. How do they come to the US for the initial incubation period? What happens after demo day?
jeevand 3 days ago 2 replies      
Can founders of a startup who have majority ownership & with appropriate board having the power to fire them sponsor green card through their startup? Founders are currently on H1b with approved I-140. Thank you
newuser2016 3 days ago 2 replies      

USCIS recently approved my EB1 visa I-140 petition. Since I'm abroad my process will go thru the NVC and then consular processing. What kind of questions should I be prepared for at the consular interview? And about how much time should I have to wait for my green card?


ancarda 3 days ago 4 replies      
I live in the United Kingdom but I've always wanted to move to America. I don't know much about the process as I find it very hard to go through the volume of information online. Do I need to apply for a visa, then find a company in the U.S. to hire me? Is there a good website for finding green card jobs?

I'd be grateful for a pointer on doing this or even just an FAQ as a starting point.

franciscop 3 days ago 1 reply      
I won a NASA contest as a programmer and I'm interested on working in the USA (also as a programmer). Am I elegible for an O-Visa? My degree is on Industrial Engineering which I'll finish in January in Spain, my home country. I also have about 1 year of work experience in two startups as an internship.

PS, thank you so much for the help so far.

shekispeaks 3 days ago 0 replies      
How can people on H1B Visa be founders. What is the best way for them to say spend 6 months figuring out what the product is without actually having an actual company?
mindvirus 3 days ago 1 reply      
What are the typical visas for a Canadian who gets accepted into YC, and afterward/during raises seed funding and sets up shop in the USA?
mahyarm 3 days ago 1 reply      
How does one with a H1B, TN, H1B1, or E3 visa move from their current US employer and start their own company legally while living in the USA?
sadok 3 days ago 1 reply      
Hi Peter. There have been several times where YC companies wanted to hire me (designer) but couldn't because they can't sponsor work visas at the moment.

How hard is it for a YC company to be able to sponsor visas? Have you had experience with this? And, as an applicant, is there something I could do to ease the process? Thank you.

alantrrs 3 days ago 0 replies      
Hi Peter, thanks for taking the time for this AMA. I have a ton of questions, but here's a summary:

1. Can I incorporate a company and look for funding under a B1/B2 Visa? 2. Once incorporated and funded, what type of Visa could I get for myself to work for my own company? 3. Would my two-year home-country presence requirement "212(e)" affect getting those visas? 4. If I'm unable to get any other Visa, could I be living in the US with a B1/B2 Visa working for the company I founded but without receiving a salary? How long could I stay? How about a TN or TD Visa?

lfittl 3 days ago 0 replies      
What are your experiences with going from a successfully issued O-1 visa to an EB-1A?

Any lessons learned / things one should watch out for? (specifically around required evidence or RFEs that you got issued)

Thanks for your time! :)

cagenut 3 days ago 1 reply      
This isn't really an immigration question so much as an avoiding-having-to-immigrate question:

What are the challenges in having co-founders in other countries and being able to grant them meaningful chunks of equity. Say example someone with 10% in Hungary and another with 10% in the Netherlands.

RohrerCarlos 3 days ago 1 reply      
Thanks for you time.I'm a chilean entrepreneur developing a startup here in SF. I'm one of the founders and we have already incorporated as an LLC.

1-What's the easiest path for me to get a visa that will allow me to work and receive a salary here in the US?2-Can I do that through the company we just established?

I'm fully dedicated and focused on our company and growing as fast as we can and I need to come to a solution to my visa so I can continue working here with no problems.

Much appreciate your help Peter.

randall 3 days ago 0 replies      
My cofounders are from Finland and Pakistan respectively. We want them to be able to move them and their families temporarily to the us for a year or two. Is h1b the best option?
disbelief 3 days ago 4 replies      
How would you rate a senior engineer's odds at qualifying for an O-1 visa? Can they get by on career/work history alone or does it require a level of public notoriety?
disbelief 3 days ago 1 reply      
If someone is on a visa tied to a specific job at a specific company, what is the legality of working on personal side projects (that may turn a profit)?
throwaway333349 3 days ago 1 reply      
Questions regarding international companies being able to sponsor H1B visas in America.

1/ How long does the process take for a company to be eligible to sponsor H1b visas. 2/ How much does it cost ?3/ Does the company need any minimum funding ? 4/ Does the company need to hire a certain number of American citizens/Green card holders before it can hire H1B visa holders ?

pboutros 3 days ago 1 reply      
We hear a lot about the limited # of H1B visas available, about how it functions as a lottery, etc... What are common issues with the H1B application process that don't receive as much public attention?
mydpy 3 days ago 3 replies      
As US citizens, how can we help our international friends trying to get H1-B support? It is really hard to watch friends get denied, and I really wish policy makers would admit more very talented people from highly competitive countries.

One of my good friends from China is gay and if he goes back home, he could actually be in danger.

I feel helpless and I want to do more.

iktl 3 days ago 1 reply      
Hi Peter as H1B / E3 visa holders are only allowed to work for the company sponsoring them, are these holders able to provide contract work (separate to their regular work) to clients either in or out of the US provided the work is conducted and billed via a registered business entity in their home (or another non-US) country?
haydenlee 3 days ago 0 replies      
When working for your own company (a Delaware C-Corp) on OPT there is some language in the policy about having to be an employee, but that you can also work for yourself. Is being a founder enough to stay in status without technically paying yourself minimum wage and being on the payroll? And does this apply to the extension too?
baristaGeek 3 days ago 0 replies      
1) Is winning an ACM-ICPC national/regional contest enough to be considered a top programmer and be able to apply for an O-1?

2) If my B1/B2 visa allows me to stay in the US for 6 consecutive months; can I do programming, sales, fundraising, etc. for my Delaware C Corp in the US?

michwill 3 days ago 1 reply      
Hi Peter!

I am a citizen of Australia and I am going to switch on ZeroDB [http://www.zerodb.io/] fulltime pretty much now. For that, I have to leave my employer with whom I have an E3 visa (and I have a wife on E3D). Also I need to travel right after that.

Would there be any problem for us to enter back under Visa Waiver? Should we just fill an ESTA form online and have back out-of-US tickets on hand when we enter back? Any possible caveats here?

Another thing - my employer could technically terminate my employment very close to our date of re-entry (due to some corporate stuff). Would it cause problems in getting ESTA (when you are still technically on E3 visa but in a couple of days you're not)?

Thank you!

shekispeaks 3 days ago 1 reply      
Can I drive for lyft part time on an H1B?
d--b 3 days ago 2 replies      
I currently hold a green card, but am temporarily abroad (2 times 6 months). How long can I stay abroad and retain the green card, if I periodically come back to the U.S.? And how frequently should I get back to the U.S. ?
jason558 3 days ago 1 reply      
Hi Peter, thanks for doing this session! My question relates to techniques and probabilities of getting H1B visa for potential hires. We are a 5 year old profitable start-up with more than $1 million in revenue...how hard would it be for us to sponsor a potential new engineering hire for the visa process? I understand that it can cost $4k in the application and X in legal fees ($5k?) which we would be ok with.

My questions are (a) what are the actual chances of success given the lottery system process for sponsoring an employee for the H1B visa, and (b) are we limited in the # of applications?

focus986 3 days ago 0 replies      
I was married to a US citizen for some years and recently got divorced over infidelity/financial issues (have proof). I have since filed an appeal as "Abandoned Spouse" which has yet to be acknowledged by USCIS so I am yet to have a new case number at all. My current work permit has run out and I have received notice to appear for Removal proceedings in Sept 2017. Is there a way to get my work permit renewed in the interim? I am yet to receive acknowledgement of receipt from USCIS about my abandoned spouse appeal
bobfunk 3 days ago 0 replies      
Hi Peter,

I have a question about E2 VISA's and what to do when you raise enough funding that you loose majority ownership?

The situation is company with 2 founders on E2 VISAs with majority ownership of a company, who'll most likely not be able to keep majority ownership after a series A.

Is there a good way to prepare for this and a good alternative strategy to not end up with a series A funded company where the founders can't stay in the country?

And do the E2's stop being valid once the founders loose majority ownership, or is it just impossible to renew them?

erehweb 3 days ago 1 reply      
Is there a good source of information for Presidential candidates' proposed changes to immigration laws? If you were a betting man, what (if any) changes would you bet on post-election?
kylnew 3 days ago 3 replies      
In your experience, how necessary is it for Canadians and Mexicans applying for a TN Visa, to be accepted for Software Engineer or Computer Systems Analyst jobs without a degree in computer science or engineering? For example, I have a B.Comm degree.

I've heard it's a bit hit and miss and if you don't have a good lawyer working on your side getting through might be tough. I'm not sure if it's a different story for H1B Visas though.

infocollector 3 days ago 1 reply      
I am currently on F1 (Alien from India, getting my PhD in early 2016, Computer Science) and am planning to apply for either the EB-1 or NIW. I have one publication (and multiple submitted), and my work has mostly gone in supporting Department of Defense. Do you recommend EB-1 or NIW route, or perhaps something else? I do have strong letter writers both in the DoD and Academia/Industry.
OSButler 3 days ago 0 replies      
There was a post here a while ago about s.o. coming to the US on a tourist visa asking to do volunteer work in return for a place to stay.I'm just curious if you've ever dealt with similar cases, where people came into the country with the wrong visa, found a place to live/work, but then had to get their papers sorted out to be able to stay?Were they actually able to stay or did they have to go back and apply from outside the country (US) again?

And more of a personal anecdote than a question, but during my own immigration process I've noticed that there appear to be mostly people who are either extremely over-prepared (have all the documents filled out in advance with additional papers/proofs/documents for every single step), or they are not prepared at all.My fondest memory was a man walking into the embassy asking to immigrate right now. No papers, documents, nothing. Just walked in, went to the clerk's window and asked to immigrate today. Even the clerk was a bit dumbfounded by the demand.

erispoe 3 days ago 1 reply      
Could the administration decide to lower the bar for some visas, like O-1, without the need for congress to approve it?

For instance, could the administration decide that anyone with a PhD, or even a master's degree, is eligible for a O-1 visa? If that's the case, why is the focus some much on statutory reform and not on the administration which could get results much more quickly?

jensv 3 days ago 1 reply      
Can you recommend some immigration resources for self-service? I am a Canadian who is seeking better work opportunities State side, with a Bachelors in Computer Science and 3 years of experience. I wonder if flying down with the intent of networking and finding companies to meet is a good and realistic way of meeting employers and lining up interviews.
h1b_transfer 3 days ago 0 replies      
Thanks for doing this! I have a question regarding time off between jobs while on an H1B.

I've been working at a startup for 2 years that sponsored my H1B. I've just accepted an offer at a big tech company, and they are transferring the H1B in the coming weeks. In the meantime, is it OK if I take 2-4 weeks off in between the two jobs without pay?

morriswong 2 days ago 0 replies      
How does a startup know if what they are doing is breaking the law or not? Usually ideas are cool until they realized that there might be legal issues that aren't intuitive enough or straight forward to those who does not have a law background.
keyblock5 3 days ago 0 replies      
Hi Peter, thanks for your time.

I have job offer to work in US, reliant on immigration.

I haven't completed my bachelor's degree, and my final exams are after the April 1st 2016 deadline. I do not have more than a year of professional experience. UK citizen.

Am I right that an H1B won't be applicable? Would any other visa types fit (Other than work abroad, then L1)?

octopus00 3 days ago 1 reply      
Hi Peter, thanks SO much for doing this

- Is it ok to form a side company while on H1B?

- Is it ok for me to develop free or paid apps through my own side company (just me doing everything, without hiring anyone else)? If not, what do I need to do to not violate my status?

- What are the minimum criteria for an O visa and is that a viable solution if the side company is going really well?

haydenlee 3 days ago 0 replies      
What's the current status of the OPT extension? I'm a co-founder of a startup that I started during my 12 months of OPT and its about time to apply for the extension, however I recently realized there'd been some changes to it.

Should I apply now for the extension or do I have to wait until further policies are put in place?

ic10503 3 days ago 0 replies      
I am moving from a big company to startup and I have initiated my h1b visa transfer. I want to take a break between the two jobs. Is it OK to go outside US after leaving the current company and come back to start working for the startup ? Will carrying the approved h1b petition for the startup be enough to re-enter US ?
CAThrowAway 3 days ago 2 replies      
Hi Peter,

Thanks for taking two full hours to do this - I've learned a lot.

I am a US/Canadian dual citizen, my cofounder is Canadian. We're currently running our business as a Canadian corporation, but would like to set up shop in San Francisco full time over the next year or two, preferably incorporating in Delaware.

My cofounder has a BSc and has done some impressive things in her career, but the O-1 looks difficult from the outside. We're in a position to raise ~1M of funding from US investors over the next 6m - would that make her eligible for an E-2? The L-1 looks like a reasonable fallback if we can get nothing else setup over the next year, but we've been told not reincorporating as a Delaware corp will make fundraising more difficult.

Is there an obvious standout option here? Are there any that I'm missing?


anarazel 2 days ago 0 replies      
I think it'd be awesome if somebody with actual clue, and without the primary intent of getting new clients, would start collecting information about the US visa situation at some permanent location.

Looking for information about US Visas on your own right now is made very hard by all the immigration lawyer's homepages. Those mostly seem to contain copied and low quality content. Often with conflicting or outdated information.

Given the obvious desire, by US companies, of hiring non-residents, it seems that there'd be a rather big collective interest in providing quality information.

hamhamed 3 days ago 2 replies      
Hi Peter, thanks for doing this. I've been applying to YC for the past 4 years and never gotten an interview, I'm starting to suspect it's because of my background (hence they never reply to you with the reason of rejection). I'm born in Canada, meaning I'm Canadian, but I never managed to finish my college CompSci Degree so I'm not eligible for a TN-1 visa. However I do have around 6 years of professional web dev experience, founded a couple of startups, raised money and exited. I am 23 now. Any tips?

This might also help, but I did not finish my degree because I was kicked out of college: https://news.ycombinator.com/item?id=5090007

Eridrus 3 days ago 2 replies      
Is it possible to transition from an E-3 to a green card directly?

I had a person at the US consulate remark on the fact that I was applying for a third E-3 visa with the comment "you can't keep doing this indefinitely", I didn't challenge him, but this by understanding was quite the opposite, that there was no limit on E-3 visas issues; can you provide any insight into this?

If I obtain a green card by marriage, but then split up before the 2 year deadline, does that have negative repercussions on your ability to get employment-related visas? I've already been dating my girlfriend for 2 years and we've been living together for most of that, so it's something that comes up as a reason to get married, but I'm not sure if it's a good idea.

yranadive 3 days ago 1 reply      
What are the top 3 things required to make a strong case to get EB1 for startup founders on H4 EAD?
izzosismyfav 3 days ago 1 reply      
I'm graduating senior(F1 visa) in this December. I'm waiting for my OPT card. Can I work in between that? Once I get my OPT can I apply for H1B on year 2016 or will I have to complete H1B? What other legal things I need to be aware of?
fawaz 3 days ago 0 replies      
Canadian starting a startup in the US:

I haven't launched my startup yet, and I reside in Canada. I've never been employed in the US.

I'd like my startup's HQ to be based in the US. What's the best way for a Canadian to set up base and launch in the US?

SeoxyS 3 days ago 0 replies      
H1B, L1, O1, etc. are all non-immigrant visa. You must be able to show non-immigration intent. (Except for O-1 which allows dual-intent.)

Edit: I may be wrong about this. IANAL.

immiques 3 days ago 1 reply      
Hi Peter,

I have a very specific question I think will apply to many people here. Me and a buddy who is from another country are building a product. We will soon be done with the product and we are thinking about registering the company here in the USA just because it is very easy to get funding here. The company will be a registered in both of our names, (even though he is a foreign national, I am flat-out assuming this is possible). Eventually, if the company does well we would want to stand up an office here. At that point, what are his options to get to USA ?

an4rchy 3 days ago 0 replies      
Great topic.. just out of curiosity.. Has YC directly sponsored any H1B visas (if not for founders but people who actually work for YC)? I tried the usual h1b websites and couldn't find anything...
patrickddaniel 3 days ago 0 replies      
Let's say you're working toward getting an O-1A, and you fulfill the three categories out of eight, how broad can you make the scope of work that you can do? (since the category includes sciences, education, business, or athletics)

For example, if you are not set on one career, and have pursued 3-4, and you get the O1 for one career (where you can show extraordinary proof), can you still do work in other areas? In other words, how broad can you define the O1 so that you could do almost any type of work as you could do with a greencard.

KAdot 3 days ago 1 reply      
Can I sell my own software as a H1B worker? E.g. my own apps in App Store?
diogenescynic 3 days ago 1 reply      
How do you feel about companies and law firms gaming job postings to disqualify qualified workers in the US so they can hire a candidate on a visa for much less? Employers are posting jobs that dont really exist, seeking candidates they dont want, and paying for bogus non-ads to show theres an IT labor shortage in America. Here is the law firm Cohen & Grigsby advising other employers in running classified ads with the goal of NOT finding any qualified applicants, and the steps they go through to disqualify even the most qualified Americans in order to secure green cards for H-1b workers: https://www.youtube.com/watch?v=TCbFEgFajGU Do you consider this abuse or fraudulent? Also, how much of your firm's work is done by paralegals using templates and boiler plate support letters?
ameen 3 days ago 0 replies      
Is it possible to start a company on a B1/B2 Visa? This would an extension of my startup in India. Is any investment required? We're bootstrapped and yet to launch our product.
geoka9 3 days ago 0 replies      
Would you recommend a Canadian wishing to work (remote, from Canada) for a US employer to get a TN visa. The work may require short (2-3 days) onsite visits several times a year.
throwawayforlaw 3 days ago 0 replies      
Hi Peter! Thanks for doing this AMA.

I had a question about H1Bs. My F1 OPT expired Feb 2015 and I had a grace period of 180 days to apply for STEM extension. But in the meanwhile (April 2015), I heard that my H1B got picked in the lottery. So I googled it and read someplace that I wouldn't have to worry about the OPT STEM extension anymore, so I didn't go forward with my STEM extension application.

Is this something I have to be worried about going into my visa interview in my home country?

graeme 3 days ago 1 reply      
What are the odds of getting an O1 visa in a very small niche. I'm legitimately one of the top experts in the field of LSAT preparation. I've published several books, run a popular website, moderate a major forum, and have written guest articles for most major sites about the LSAT.

However, it's a small field, and not one that attracts much press coverage. How does this balance out?

I run my own business. All online, mostly US customers, soon will be a Canadian corporation.

rdc12 3 days ago 0 replies      
Is getting a visa to work at a U.S at startup something unique to the YC program? I am/was under the impression that the company had to be accredited to be able to employ foreign nationals.

Is there any advise you can give for a current undergrad (for me personally citizen of NZ and UK if that matters) to improve the odds of being able to accept a job or PHD study in the US (on the visa side of things), both at application time and now til then (~2 years away).

anindyabd 3 days ago 1 reply      
The 17-month OPT extension has recently been terminated by a court. What are the chances that a new rule will be implemented regarding the OPT extension?
sul4bh 3 days ago 1 reply      
How does H1B and remote work? Say, can I work in Nashville remotely from home for a company in San Francisco and have a valid case for getting H1B?
poerkladsfl 3 days ago 1 reply      
L1 related. What should I do if I want to go work for a different company but am currently on an L1-A visa (been here in the USA for 3 years).
laxinger 3 days ago 1 reply      
Hi Peter. I'm currently preparing for O1 visa. I'm on B1/B2 visa now and planning to extend 2 months so I can stay total 8months while I prepare for O1 visa. My question is if I ever get denied for extending B1/B2, can I have any disadvantages when I apply for O1 visa? I met a person who told me this but I'm not sure whether this is true.
tosseraccount 3 days ago 1 reply      
Critics have charged that H visa guest worker programs are subsidies to the already rich holders of enough capital to influence inside the beltway politics. They say that the program is designed to keep wages down and facilitates outsourcing. Are the current laws also a subsidy to the legal industry who get to charge for an overly complicated process?
n00b101 3 days ago 0 replies      
What is the usual or best process followed by Canadian citizens who get into YC and then relocate from Canada to Silicon Valley?
shpx 3 days ago 1 reply      
If I were to take a year off school right now, can I still get a J1 for an internship this summer?

Also just wanted to say thanks for doing this.

sn0v 3 days ago 1 reply      
Hi Peter, thanks for doing this.

How would you recommend an H1B holder go about transitioning to founding/working for their own startup?

ojbyrne 3 days ago 0 replies      
I'm curious about gambling while on a work visa. It seems obvious that spending your vacation in Vegas is acceptable, but what if I think I'm good enough to become a professional poker player and decide to pursue that part-time while working full-time. At what point (if at all) does it become an immigration issue?
erispoe 3 days ago 0 replies      
What is the best way to transition from a J1 visa (visiting researcher) to a visa allowing to work one's startup?
roadbeats 3 days ago 0 replies      
The new startup I joined is filing a new H1B instead of transferring (because it took so long to transfer due to company's registration progress). Previously, I was filed H1B two times (2011 and 2014). Is third time possible ? Especially now, since visa regulations are getting strict for security reasons.
homakov 3 days ago 0 replies      
What's best way to move to US and not work for anyone and not invest much money? O-2? I work in infosecurity
gemmakbarlow 3 days ago 0 replies      
What process would you recommend for a startup looking to relocate software developers immediately from the UK to the US?

The H1B process officially kicks off in April, so am interested to hear about types of contractual agreements that might allow employment from now for the next twelve months whilst processing is underway.

a-zA-Z0-9 3 days ago 1 reply      
Hi Peter,

I'm a Canadian and I had an H-1B several years ago. I used about 2.5 years of it and left US in summer 2011 before using up the full 3 years.

Am I eligible to come back on H-1B without lottery by claiming the remainder time? I read something about this online saying that I can come back on H-1B before 6 years past the date I left US?


danieltillett 3 days ago 0 replies      
Peter a basic question about the L1 visa. If you are the owner and manager of an established foreign business can you apply for an L1 to establish the USA branch? Does the USA branch need to be established for some length of time? Does being the owner of the foreign business cause problems?
PureSin 3 days ago 0 replies      
Hi Peter,

My wife and I are Canadians working in California on TN visas. I'm at a small startup that doesn't sponsor H1-B but I might start my own business in the future. Should I look switching to a larger company in order to get H1-B so I have the freedom start my company?

Thanks for doing this AMA.

cpenarrieta 2 days ago 0 replies      
I am from Peru and I have a software engineer degree, I'm currently taking a Dev Bootcamp in San Francisco and will look a job here after that. I am currently with my Tourist visa. What are my real chances to get a H1B visa?
kur158 3 days ago 0 replies      
Hi Peter,

I want to know how can a founder and a co-founder who are on F-1 and F-2 Visa respectively start a company. What are the requirements for the company to sponsor their own Visas at a later date if and when required? Do investors have a bias against investing in such companies.



judge 3 days ago 0 replies      
Can you complete an H-1B transfer (moving from one job to another) while outside the US (traveling for 2 weeks), so that upon your return you can join the next employer? Or do you have to physically be in the US while the request by the new employer is filed and accepted?
susiemielekim 3 days ago 0 replies      
I'm currently under OPT visa until next August. How would getting a resident visa work after incorporating the company in my home country work? (the company has already been incorporated in United States under the other co-founder). Thank you!
arunbahl 3 days ago 0 replies      
Are there options for a foreign national to move from an E-2 visa (treaty investor) to permanent resident status? I've heard that it wasn't possible previously but now may be, making the E-2 a possible "startup visa" for many.

And thank you for doing this!

throwaway_nj 3 days ago 0 replies      
Can you share some advice on how to build a prototype / proof of concept while working as an employee? I have read this is not an issue in places like California as long as you do not use company resources. But what about states like New York?
_fabio 3 days ago 0 replies      
Hi Peter, thank you so much for doing this!

I'm a student on F-1 visa. Am I allowed to form an LLC and sell products / offer services, while revenue from said products or services will be kept in the company bank account, without me pulling a salary?

Thank you!

ulobabacan 3 days ago 1 reply      
In these days, how long does it take for a H1b holding engineer from a "rest of the world" country to get green card via EB2(or EB3 if faster) from the day the current company starts the progress?

Also at what stage he/she can change the job?

msvan 3 days ago 0 replies      
I'm told that it's easier for musicians/artists to get O-1 visas than for software engineers. Are software engineers disadvantaged from getting the O-1, simply because the visa wasn't designed for software talent?
henkel 3 days ago 1 reply      
As a competent and above average software engineer from Morocco, not holding a university degree, what are my options for a work visa in the US, assuming I get a job in a company willing to put every possible effort into this?
crorella 3 days ago 0 replies      
Hi, right now I'm processing my perm, in particular, the PERM application was sent to the DOL last September. I would like to know if there is any problem if I change jobs now. Will this cause delays in the process?

Thank you

miciah 3 days ago 1 reply      

Is it possible to do YC, if the founders are initially registered as 'tourists'?

shpx 3 days ago 2 replies      
Canadian, recent high school grad. What are faster ways to working and eventually living in the US than doing a degree then getting H1-B?

F-1 and OPT then H1-B? O-1? Making some money in Canada then starting a company and E-2?

nathanvanfleet 3 days ago 1 reply      
If I come over from Canada and work somewhere, is it easy to change my job to another company if things don't work out? Do I have to leave and come back? Is there a deadline in finding a new job?
manuelgodoy 3 days ago 0 replies      
I have 5 years of experience and a BS and MS degree in Electrical Engineering from a top school in the US. How easy is for me to get an EB2 visa if a company is willing to do the process?
nathanvanfleet 3 days ago 1 reply      
How does the process work. If I (a Canadian) get a job in the US, what is the timeline for me to be onsite working? What kind of help settling (finding a place to live etc) is there?
alinspired 3 days ago 0 replies      
What happens with L1 visas of a company that is acquired by another US company?

and related: How long until you have to leave US if the L1 issuing entity is acquired (and disappears as an entity)?

tty7 3 days ago 1 reply      
1. What is the best course for an E3 Visa holder to move to a Greencard? (or something similar).

2. If an E3 Holder would like to found a startup, how does one go about self sponsorship?

raitom 3 days ago 1 reply      

Is it possible to apply directly for a green card through employment while being on J1-Intern visa?How long do you think it takes to receive it if approved knowing I'm French?


edko 3 days ago 0 replies      
I had an H1B visa granted to me in 1998, but have never used it. Would that have any influence, either positive or negative, on me getting a new H1B?
golergka 3 days ago 1 reply      
How important a degree is for H1B? I think given amount of self-taught engineers in the profession, you have to answer this kind of question a lot
shpx 3 days ago 0 replies      
I will need a J-1 visa this summer, but I applied for the diversity visa this year. Could my J-1 be rejected for having an intent to immigrate?
throwaway-apg 3 days ago 0 replies      

Can you describe in practical terms how the requirements between an O1 and an EB1 differ? If I got my O1 recently, can I reuse the reference letters directly?


BradRuderman 3 days ago 0 replies      
What is the average cost you recommend for a new the H1-B petition? What about an h1-b transfer? (Legal fees not including filing)
erispoe 3 days ago 1 reply      
How hard is it to create an entity that is H1B cap-exempt and can this entity be related in any way to a for-profit company?
susiemielekim 3 days ago 0 replies      
Currently under OPT visa co-founding a startup. How would a resident visa from my home country work?
patrickddaniel 3 days ago 0 replies      
Can you get a J-1 visa even if you have already had 2 OPTs and studied for undergrad and grad in US?
goodcall 3 days ago 0 replies      
If a H1b holder participates in a Hackathon with cash prizes and wins. Can he claim the money?
chill_bro 3 days ago 0 replies      
Is it possible to own stock in a start-up and work on it without getting paid while on an F1 visa?


gobr 3 days ago 0 replies      
What are the most common difficulties for immigrants? Any Brazilian examples?
PameVls 3 days ago 1 reply      
Do you need a visa to attend a 3 month program like YC or Techstars?
franze 3 days ago 1 reply      
Meta question: Do the US immigration laws make sense?
tosinaf 3 days ago 0 replies      
How does the J1 to H1B visa work?
gozo 3 days ago 1 reply      
Uhm. I don't want to sound ungrateful, but I was slightly disappointed by the result of this. They might want to change the format in the future to promote fewer longer answers. "You might try X Y Z visa" doesn't really use his expertise very much. We also didn't really get an answer to the most obvious questions like if you can actually attend YC legally or if a temporary worker in the US can also run a company. Still a good thing of course, but a bit more structure would go a long way.
treasuresque 3 days ago 5 replies      
DO NOT WORK WITH HIM. Can't believe he made it to HN!!He somehow became a thing but I can't stress out enough how much money he had cost me while providing either no service -even declined to work on my first case- or really shitty service, where i ended up writing all docs myself. He does not think out of the box at all or provides any value bigger than digging into Google. He didn't have any plan B or even replied to my emails asking what we should try next after we had lost. The only thing he ever did for me was sending the invoice. Everything else was taken care of by his assistant, sending information and documents i had drafted to the government, trying things that i had researched myself.Please feel free to reach out to me for a curated list of good immigration lawyers. I would have taken my return flight back to Germany more than a year ago if i listened to his advice, which has been "i don't see any options here" when there WAS an option for another year.
rorykoehler 3 days ago 0 replies      
In your personal opinion how far away are we from your job becoming obsolete?
RjCharm 3 days ago 1 reply      
We're shortly going to be opening an office in a European country and employing several local employees. Once the organization is established, what would the process be for inter-organizational transfers of employees between countries? For example, if someone were to relocate from primarily working in Europe to primarily working in the USA?
Django Awarded MOSS Grant from Mozilla djangoproject.com
301 points by cdnsteve  3 days ago   98 comments top 20
mladenkovacevic 3 days ago 2 replies      
I know purists and more experienced developers will disagree, but for a clueless beginner like myself with no real-life connections to the developer community, it's awesome how external projects have been integrated into official Django releases as they become widely adopted and production ready. From staticfiles, through migrations (developer of South took this one to Kickstarter too) and now channels and DRF. Python 3 followed a similar path with virtualenv.

Does any other framework ecosystem work the same way? I don't follow Rails or other ones as closely as I do Python and Django.

hackuser 3 days ago 2 replies      
Good for Mozilla, putting their money where their mouth is and setting an example. It would be great if they or someone with sufficient reputation setup a way to make it easy for others to contribute financially to FOSS projects, a central payment clearinghouse.

In my for-profit organization, we thought we should contribute to the free / open source projects that we were using and which were helping us make money. We wanted to pay our share.

We started by putting together a list, and that's pretty much where it ended. The list was enormous. Try it yourself, and don't forget the components of other products that are FOSS. EDIT: Here's Mozilla's attempt at making such a list, which they label "incomplete":


On one hand, it reflects the incredible contribution of FOSS software. On the other there was no way, in a busy workplace, we were going to spend days tracking down all those FOSS developers, finding ways to send them money, and implementing it.

bjt 3 days ago 4 replies      
On one hand, awesome to see Django getting first class Websockets support.

On the other hand this feels a bit like "Build the web applications of today, tomorrow!" I think that's an inherent problem with kitchen sink frameworks; new things that come along have be shoehorned into the old way of doing things, except when they can't, and then you have to restructure your foundation. I stopped using Django a couple years ago because the kind of apps I wanted to build were REST APIs, websockets, a little static file serving, and connections to realtime backends like redis pubsubs or RabbitMQ. Django wasn't great at any of those, yet carried a lot of baggage from stuff I didn't need (e.g. form handing, templates).

I would recommend Flask and/or its underlying Werkzeug library (my personal preference) over Django.

mangeletti 3 days ago 0 replies      
Congrats to Django, and thanks to Mozilla!

This is the best news! Thanks @ Andrew Godwin (since I see him in here) for all the work you've done on Django over the past N years. I've contributed small amounts of money to the fellowship program 3 or 4 times now, but the sum of that program's entire fundraising for 2015 is almost tripled by this single grant, and the purpose of this grant money is so exciting. Channels is awesome; so is Django REST Framework. Adding the best of these things to the core and tightening some of the architecture sounds exciting. It helps to position Django for continued relevance, web services and real-time applications become more popular.

dudus 3 days ago 5 replies      
Don't get me wrong, these are amazing news love the project, but I'm a bit confused on how Mozilla works. Just the other day they were requesting donations, now they are donating money?

Why ask for donations if you are expecting to donate the money away instead of using it to improve your products?

I donated to mozilla a few weeks ago, but now I wonder if I should have just donated directly to django instead, skip the middleman that is not adding anything.

hackuser 3 days ago 1 reply      
Here's a full list of Mozilla's first round of MOSS grants:


austinjp 3 days ago 1 reply      
Congratulations to the Django team.

In practical terms, what does this mean for those of us who use Django and Django REST Framework? Will there be a graceful transition path to the planned all-in-one model? What timescales are likely?

Also curious about which version(s) of Python are likely to see continued support.

I appreciate that some of these questions probably can't easily be answered yet, but a steer would be appreciated.

Edit: typo.

icn2 3 days ago 3 replies      
Could some company support phoenix framework as well? Phoenix's channel feature is already there :)
samskeller 3 days ago 1 reply      
Very exciting news. The Channels project within Django is one of the most anticipated future projects and the Django REST framework has been a essential add-on to Django for a long time now
travelton 3 days ago 0 replies      
This is great news! My Django installation feels pretty bloated with various 3rd party libraries to handle post-request tasks and serving data via web sockets. Can't wait to shed those dependencies. Congrats Django team!
dethos 3 days ago 0 replies      
Awesome news, I'm even more excited after reading about how they decided to spend the money. I haven't yet digged into it, but "channels" look promising.

I would also like to congratulate Mozilla for this program, several other great projects are being funded.

victorhooi 2 days ago 1 reply      
Curious if anybody here has experience with Django versus Flask versus <other> on AppEngine?

I'm new to AppEngine and I'm familiar with Django, so that would have been my first choice - but it seems like Flask, Werzkeug, Webapp2 are more popular - just curious why, or how good/bad Django is on it.

dormanted 3 days ago 0 replies      
> Buildbot: $15,000. Buildbot is a continuous build and integration system which has been immensely valuable to Mozilla over the past few years. Their award will be used to remove the term slave from all documentation, APIs and tests

Oh good. $15,000 well spent.... sigh.

rayalez 2 days ago 0 replies      
Oh, this is amazing!! I'm a huge fan of Django, and I'm really happy to see that the project keeps improving. Recently we had news about update to Django CMS, and now this.

Really great stuff.

astaroth360 3 days ago 0 replies      
I would love it if django were able to compete for more funding like this! Things tend to move much more quickly when there is decent funding involved :D
latenightcoding 2 days ago 0 replies      
As a perl programmer I wish someone would fund mojolicious
bedros 3 days ago 0 replies      
awesome, love django project
reinhardt 3 days ago 1 reply      
TIL there's a (big apparently) FOSS project called "Bro". Is this an acronym, humour, sarcasm or what?
asr1 2 days ago 1 reply      
Wow, channels...I will just go and learn go. But for other reasons too. I dont like django anymore, because I kept continuously fighting with the framework. Its fast to work with, if you do exactly what the developers had in mind, but it gets in your way all the time, if you use your own brain.
Cell-Site Simulators Arent Secret Anymore eff.org
262 points by pavornyoh  4 days ago   86 comments top 15
themartorana 4 days ago 5 replies      
Some version of these went up all over Philadelphia when the Pope came to town and they turned the city into a giant TSA checkpoint, replete with National Guard troops.

The Pope left.

The boxes stayed.

Edit: here's one of several pics I took: (see edit 2 below for link) anyone recognize it? Interestingly enough, the nuclear and chemical detection boxes were labeled...

Edit 2: direct photo link: https://s3.amazonaws.com/f.cl.ly/items/1X2f2i1M2P0e0n322r1X/...

ipunchghosts 4 days ago 2 replies      
Since OpenBTS launched 6 years ago they haven't been secret. All you need is a backpack with an Ettus USRP1, handful a D batteries, and a laptop. Walk into any starbucks, connect your laptop to the wifi and then SIP through Google and voila, you can snoop on everything and no one has any clue.
alexvr 4 days ago 6 replies      
What? Why doesn't it work like this:

Cell phones have SIM cards with an ID and a secret key. Cell service providers have a database of these SIM associations. Cell phones encrypt IP packets in their entirety with the symmetric key and send it as the payload of some cell protocol packet that might expose my ID, if anything. Assuming the cell provider is secure and not on the dark side, this is the safest part of my my packet's trip.

I don't understand how a cell-site simulator could see what websites I visit, much less the messages I send, without knowing my key. And it's not like one could trick my phone into thinking it's the actual cell site, because it won't be able to respond to my transmission with a message that my key can decrypt.

What the heck am I missing?

mirimir 4 days ago 1 reply      
There are apps for detecting these things. Maybe we need an app that plots locations based on anonymized submissions. Also, I wonder if it's possible to distribute blacklists. But I suppose that's buried in the radio firmware.
droopybuns 3 days ago 1 reply      
"Cell Simulators" is a bad name. We should be calling them "Phoney Base Stations"
mmaunder 4 days ago 1 reply      
Related: A video I took at blackhat 2013 demo'ing a hacked femtocell intercepting calls. Voice is intercepted before the call even starts. https://vimeo.com/71466006
codezero 4 days ago 3 replies      
Are there any standalone tools that can be used to capture the meta-information about cell towers?

It seems like it would be really useful to crowdsource known towers, their identities and strengths, so that simulators can be singled out.

ikawe 3 days ago 0 replies      
atallcostsky 3 days ago 2 replies      
I wonder if it would be possible to take the idea of certificate authorities and apply it to cell phone towers. Basically, each cell tower company would be a CA, and could generate a certificate for each cell tower. Major cell tower companies could then be trusted by other CAs, and cell phones could have a store of trusted CAs. Then, when a cell phone attempts to connect to a tower, a check is made to verify that the tower is trusted by a trusted CA. This way, a user could (at least maybe) revoke a certificate from a CA that has trusted a group that has set up a cell site simulator.

My knowledge of PKI is pretty shaky. Does anyone know if something like this would work and/or be an improvement?

malandrew 4 days ago 2 replies      
Is there any way to re-engineer infrastructure so that all cell-sites cryptographically identify themselves so that cellular devices can verify the identity of a cell-site before identifying itself to the cell-site?
ck2 3 days ago 0 replies      
So is wiretapping illegal without a warrant or isn't it?

Because they aren't collecting just meta data with this and unlike the NSA there is likely no discipline at all about how the data is used or shared.

Policing in this country has come down to "try to stop us from doing it" instead of asking first "is this even legal" on every aspect.

cm2187 3 days ago 1 reply      
I can see how it can be creepy, however my house was burglarized recently, and I would have loved to have a device that could catch the IMSI of all the mobiles in my flat at that time. I can't really do anything with the IMSIs myself but I could give them to the police after a burglary, like a CCTV tape.
KazWolfe 3 days ago 0 replies      
So, how long until schools decide to deploy this in the name of "student security?"
EFruit 3 days ago 2 replies      
I was speaking with a friend regarding cellphone jamming, and a question was posed:

Suppose there is a piece of equipment that strictly follows all the relevant cellular protocol specs and can route 911 calls, but drops all other traffic. Is such a system illegal?

mtgx 3 days ago 1 reply      
Why aren't the OS vendors such as Apple, Google and Microsoft protecting users against this?
Show HN: Ponzi.es Startup sticker ponzi scheme ponzi.es
331 points by zachlatta  3 days ago   124 comments top 29
codeshaman 3 days ago 1 reply      
From Wikipedia:

Ponzi's original scheme was based on the arbitrage of international reply coupons for postage stamps; however, he soon diverted investors' money to make payments to earlier investors and himself....

The profit that could be made by taking advantage of the differing postal rates in different countries to buy IRCs cheaply in one country and exchange them for stamps of a higher value in another country was the intended profit generator for a scheme operated by Charles Ponzi, which became the fraudulent Ponzi scheme.

natch 3 days ago 1 reply      
Why not ask for "one or more" stickers? (Still promising only two total in return). If you're not being purist about the ponzie part, that is. Then you might be able to carry it on a little longer.
thrownaway2424 3 days ago 1 reply      
Which of the examples is considered to be a "startup"? The featured one is a 15-year-old spinoff of a public company with hundreds of millions of dollars in revenue. The second one is an actual public company with 4000 employees. The third one might be sort of a startup, but they have over 100 million dollars in VC backing.
daveguy 3 days ago 1 reply      
Self sustaining version:

Send us 3 stickers and postage. Get 2 unique stickers back. You get all the profits of the excess postage (assuming bulk rate) plus the extra sticker can go to a bin / advertising page of ALL THE STICKERS.

ohitsdom 3 days ago 6 replies      
Why not ask for 50 cents or a new stamp in the envelope? Including a stamp is easier in an envelope, and it should make prepping a return envelope simpler. Require "forever" stamps to protect yourself from a postal price hike.
mydpy 3 days ago 1 reply      
I bet a lot of companies would just give you their stickers, simply to take advantage of your distribution network. It would be funny if this became a legitimate business idea (while the trend is alive).
mmcclure 3 days ago 1 reply      
The nit picking in these comments seems, for once, to actually be in good spirit, but this is a really fun idea!

I would love to do this with startup / event t-shirts so I could (eventually) clean out my drawers...too bad sizing clothes is a little more complicated than stickers.

fortytw2 3 days ago 1 reply      
This is awesome in every way. Talk about a fantastic project
djsumdog 3 days ago 2 replies      
I wonder if they'd honor the deal if I sent them a letter internationally...with an Australian 50 piece
timdorr 3 days ago 1 reply      
Can I just mail you postage instead of heavy quarters? I think that would be nicer to the USPS.
thecuriousone 1 day ago 1 reply      
Inspired by what you guys done. I created one based on favours :). http://ponzifavour.com/
devNoise 3 days ago 0 replies      
That moment you see this and then wonder where your pile of stickers went.
fluxsauce 3 days ago 2 replies      
In case anyone was curious, mailing cash is legal according to this Snopes Article - www.snopes.com/legal/postal/sendcash.asp

Can't find a current USPS FAQ, though.

highace 3 days ago 2 replies      
If I want to just buy a load of different startup/saas/tech stickers, is there a place I can do that?
cpdean 3 days ago 0 replies      
Given how many useless stickers I've been given at conferences this might go on for quite a while.
pierrebazoge 3 days ago 0 replies      
Looks like they ride on the Notifuse successful initiative: "Free dev stickers, worldwide shipping" https://notifuse.com/stickers
melvinmt 3 days ago 0 replies      
Curious to know what the privacy policy is with this thing.
plesner 3 days ago 1 reply      
Nit, but: it's not actually a Ponzi scheme. What made Ponzi's scheme a Ponzi scheme was not the postal coupons. It was pretending that investors were making huge profits to make them keep their money invested, and paying them with each other's money. At that point the coupons was just an alibi.

Of course I don't know ponzi.es' long-term plan, maybe they'll start taking investment and promising huge returns. That'd be pretty funny. Though I don't know if you can legally run a Ponzi scheme, even an ironic one.

benwaffle 3 days ago 1 reply      
If I send 2 stickers will I get back 4?
collinmanderson 2 days ago 1 reply      
Are you going to do all this work by hand?
pearjuice 3 days ago 0 replies      
Can I also send you a dollar and get back four stamps for return postage?
wfbarks 3 days ago 0 replies      
the amount of time and effort I'm willing to spend to get start up stickers is probably about equal to the effort it takes to type in the address to mail them to. probably less.
srameshc 3 days ago 2 replies      
Always wanted that github sticker, never knew where to get from :)
mariocesar 3 days ago 0 replies      
You got something here
gtpasqual 3 days ago 0 replies      
Would you send it to the EU and take euros :)?
rogeryu 3 days ago 0 replies      
I want PONIES!
jkrejci 3 days ago 2 replies      
austinstorm 3 days ago 0 replies      
personjerry 3 days ago 4 replies      
32 points, 35 minutes ago? Is this a voting ring?

Edit: As mentioned below, also with 0 comments at the time (this was the first comment)

UK citizens may soon need licenses to photograph some stuff they already own arstechnica.com
253 points by jfasi  1 day ago   118 comments top 29
RobertoG 19 hours ago 2 replies      
What I miss in this kind of articles is more details in how this kind of laws are cooked.

They say "the UK government", OK, but who in the UK government, and who is really behind. Not that we don't know, but making the process explicit would be helpful.

Aloisius 20 hours ago 1 reply      
Won't this severely affect movies and television? I imagine a huge number of movies and television shows have at least some piece from a classic designer and I can't imagine Hollywood is going to track down who owns the licensing rights for every set piece, building, article of clothing, etc. ahead of time just to show it in the UK.
buserror 18 hours ago 2 replies      
It's just what happens when most of the policies are made up by 'sponsored' lobbies -- the ones with the deep pockets.

It's completely bonkers really, most of the decisions that people care about are made up in a gray area that doesn't involve them at all; you got an office somewhere making up policies straight out of briefs they've been fed by lobbies (who feeds it to every party too, just in case)

A good recent example is Heathrow's third runway. Despite millions of people living underneath, well, someone somewhere made a 'study' that 'proves' it's a good idea, so despite the fact it impacts millions of people -- in fact, millions of the most influential and rich people in the country even (London and the Thames Valley) well, it's likely to go ahead anyway...

WalterBright 22 hours ago 2 replies      
This may run head on into the ubiquitous surveillance state. After all, you can how prevent anyone from legally photographing you if you wear some classic designer objects.
blowski 17 hours ago 2 replies      
OK, seems like a storm in a teacup. The items covered by this ruling ('industrially exploited artistic works') are artistic items that have been reproduced more than 50 times. They are already under copyright for 25 years, but due to a judicial review based on EU law, that section has been repealed so they are now covered by the same copyright as 2D artistic works (such as paintings).

Therefore, it's not everything that I own, but only items that a court would consider to be a work of art. Since Apple would have a hard time convincing a court that their laptops are first and foremost works of art, this will have no effect on my ability to run a blog featuring photos of Apple laptops. However, if I wanted to sell a book featuring photos of Busk and Hertzog chairs, I would now need to wait for 70 years instead of 25.

IANAL, but I really don't think this is a big issue.

js8 18 hours ago 1 reply      
I understand the reasoning of the UK government as follows:

It's important to preserve soul and essence of things. When you photograph someone, you take part of their soul; similarly, when you photograph an object, you take part of its essence. The modern technologies created lot of soul-less people and essence-less things; to restore the world to its former magical beauty, we need more protection against soul- and essence-taking.

Now, I am not an expert in psychology of governments, but the UK empire is actually quite old. We have to consider the possibility that it may be getting senile.

netcan 19 hours ago 0 replies      
To me, the long term trend in copyright legislation is the perfect quite example of how our system is broken.
userbinator 18 hours ago 0 replies      
This sounds like it'd run counter to the requirements of sellers to provide clear pictures of the products they're selling, and gives an opportunity for scammers to weasel around them by referring to this ridiculous law. What's next, illegal to describe something you own without license?
carlob 17 hours ago 0 replies      
I think they're trying to curb websites like voga.com. The UK now is in a weird position in terms of copyright law of designer objects: basically a physical copy of say a designer chair is considered the same as a photo of it and so it can be mass produced without paying royalties to the designer's estate if they've been dead for more than 25 years (as opposed to 70).

Wouldn't it be better to rephrase the law making a distinction between unique (or small batch) works of art and works of design that are going to be mass produced?


walterbell 11 hours ago 1 reply      
There was a recent HN thread on the world's highest-grossing photographer, https://news.ycombinator.com/item?id=10689889. Would this impact Prince's business model?
im2w1l 18 hours ago 2 replies      
DMCA has been used to accomplish this in the US. A man who took a photo of a star wars action figure he had bought got issued take down requests for the photos. Also relevant is that the figurine was apparently accidentally sold ahead of schedule, but still.
bsenftner 12 hours ago 0 replies      
Clarify: does this impact fashion, as in clothing, hair and so on? The Fashion industry treats their work as art. Can this be used to block fashion critics from using imagery in the criticism of an outfit, shoe, hair or makeup design?
wildefyr 22 hours ago 1 reply      
Yet again I am ashamed of my government.
Joakal 18 hours ago 2 replies      
I have not read the laws but mostly likely this applied to commercial use already for years and they wanted it applied to private use.

However, it's a racket. To give an example, the happy birthday song is illegal to 'broadcast without permission of copyright holders'. But, the restaurant owner can buy a license from a copyright group for a fee.

Second example; Imagine being a school and dealing with students who submit assignments with copyrighted content. It's illegal to accept such submissions without approval. So, all schools usually have to pay for a license from copyright groups (Music, photography, video, etc).

Here's a really good article on it from Australia: http://insidestory.org.au/the-copyright-cops/ Most, if not all copyright groups in Australia are Hollywood-based).

Here's a copyright group website: http://www.musicrights.com.au/fact-sheets/usingmusicinschool...

> How do I know if I am doing the right thing at school?

> To make using music in schools easier, there are a number of licences in place between copyright owners and schools. These are set out at the end of this Guide. A reference to (Licence # ) is a reference to the licence applicable in each instance.

mcv 12 hours ago 0 replies      
How do you determine whether an object you own requires a license to photograph? It sounds to me like you'd want every object you buy to come with an automatic license to make photos of it.

And of course you need to store all those licenses somewhere. What an awful overhead.

jackgavigan 17 hours ago 0 replies      
The Act that that includes these changes also includes provisions to allow compulsory licensing of "orphaned" copyright works: http://www.legislation.gov.uk/ukpga/2013/24/section/77/enact...
blowski 17 hours ago 1 reply      
I'm trying to understand what problem this law is trying to solve. Without this law, would it be legal for me to buy a Damien Hirst painting, photograph it and sell the photographs? Or is it that I could run an unofficial information website (say, on Apple laptops) on which I list detailed photos, and the brand owners want to limit that?
vezzy-fnord 21 hours ago 5 replies      
A lot of technologists here have certain dreams or aspirations they want to see materialized in their lifetimes. AGI, transhumanism, space colonization, etc.

My dream is that I hope to see the emergence of a viable minarchist or anarchist society (beyond Mennonite and kibbutzim arrangements). Thus far the closest was a brief period of Catalonian history amidst a civil war.

pwm 17 hours ago 1 reply      
I would love to see a statistical analysis on how many times an average citizen, assuming zero maliciousness, breaks laws like this unknowingly in his/her lifetime. It wouldn't surprise me at all that literally everyone breaks one at one point.
tantalor 11 hours ago 1 reply      
Do I also need a license to sit in a designer chair?
iofj 22 hours ago 0 replies      
mhandley 17 hours ago 1 reply      
So if a celebrity wears a designer hat or jewelry and they own the copyright to it, they can sue the gossip magazines for publishing unauthorized photos taken in a public place (which would normally be fair game)? Could be fun to watch how that plays out in court!
Nano2rad 11 hours ago 0 replies      
If there is a copyright on a chair, you cannot manufacture the chair without permission. Where is photographing it mentioned?
whibble 11 hours ago 0 replies      
No more photos of celebrities in designer clothing then. It would get too expensive for the rags to do it.
jkot 17 hours ago 3 replies      
I do not think this can pass. I think it would be incompatible with EU law, which has higher priority than local country law.
jevgeni 17 hours ago 2 replies      
Is this some contrived plot to repel people from taking UK citizenship?
tempodox 14 hours ago 0 replies      
A prime example of how law creates the criminals in the first place. We could live in perfect peace without this brain fart.
gambiting 18 hours ago 1 reply      
I mean.....it already sort of happens in the US. Every single TV show censors out any brand names, car logos, restaurant names out of fear of I don't know what. Being sued I guess? It's ridiculous.
zoner 15 hours ago 0 replies      
Not a surprising act after the rest of the population pays "TV license" for decades to watch BBC lying.
April Zero v3 aprilzero.com
302 points by aprilzero  4 days ago   93 comments top 30
hunvreus 4 days ago 10 replies      
Looks sharp, definitely a lot of great work on design.

I just can't seem to see the value in it; what actionable or information do you get out of this? I've tracked my movements with OpenPaths [1] in the past and had fun building a couple data visualizations out of it (I love the fact that others can request access to anonymized data collected by OpenPaths).

I just have no anecdotal or empirical evidence of users putting this data to good use. If anything, folks I've seen using Fitbit/FuelBand could use a bit more exercise and a better diet. People tracking their sleep activity tend to stay late and have poor sleeping habits (for example).

Do you really need a fancy dissection of your every move and interaction to even know what you should be doing with yourself?

[1]: https://openpaths.cc

vessenes 4 days ago 2 replies      
Super cool theme. The company seems to be at https://gyrosco.pe/. They lost me at the facebook login, but it made me think about privacy and user-defined privacy a bit.

Gyroscope's pitch says you can share or instead keep everything private. I think a service like this would be a lot more appealing to me (and it is undeniably cool) if the data were kept encrypted on server, and only available on my end device.

I suppose the same thing could be done with some sort of multiparty destination encryption for sharing.

Until then, my data is just getting monetized for any party that will pay gyroscope. Any comments from your team, aprilzero?

artursapek 4 days ago 2 replies      
This is the most San Francisco thing I've ever seen
Cshelton 4 days ago 3 replies      
I was just thinking about the UI on a lot of shows/movies and how rarely we see any UI like it in the real world. Even those 'FBI databases mainframes...' that a character hacked into do not look anything like that in real life.

This is pretty cool and fits in well in 'Hollywood'. Now I want to find a project that I can actually use a similar UI on haha.

nkozyra 4 days ago 1 reply      
I love the UX but I still find the self-quantification data shallow. Time as a measurement of efficiency is inherently flawed, for example.

And there's (as best I can tell) no goal setting nor recommendations for meeting goals based on prior successes and failures.

That's still the big gap in self-quantification. In the mean time we're just making prettier charts.

vikram360 4 days ago 1 reply      
I vaguely remember seeing the homepage of a quantified self nerd that showed all his live stats - ecg, hydration, activity and stuff like that. I can't seem to find it though. It was absolutely beautiful. Then again, like the other arguments made here, I don't know what one can do
googletron 4 days ago 0 replies      
Here is some of the data collected during my work out this morning in the new mobile app coming out soon.


I have been using this product as soon as I could join, then joined the company few months after.

There is definite improvement in my health and focus. As a weightlifter, it's especially useful and awesome to see after a work out. Or whatever you do to keep you active. And if you haven't started make a goal of taking a walk or light jog! That's streaks keep your motivated to improve!

Check out other stuff here at https://gyrosco.pe/myusuf3

Give it a shot and use the data to improve, set short term goals and strive towards them, the differences you will see are incredible.

jm4 4 days ago 0 replies      
Why in the world would I want to connect this to Facebook? You really need an option to sign up with email.

Other than that... Very cool looking design. Looks like it would be interesting to try. I like the integration with other apps and sensors. Lots of potential.

fieldforceapp 4 days ago 2 replies      
Big change since v2, congratulations. Any plans to commercialize? Would like to see this applied to athletic team performance monitoring.
daok 4 days ago 0 replies      
The UI is very nice, very interesting. I am curious if this could fit with the Microsoft Band.
wppick 4 days ago 0 replies      
I love the direction that this project is going in. I think a future where you can monitor your body in real time and use that data to make decision about what to eat, treat medical conditions in their very stages, maximize energy, etc. I think body chemistry is still a huge unknown area that I would love to see explored. Can you monitor the body and sense DNA damage and possibly be able to attribute it to something that you can change. Can you measure the levels of various chemicals in your body to find an imbalance? Fuck 140 characters, this is the direction the minds of our brightest people should be going in
anant90 3 days ago 0 replies      
"What you measure is what you get."

Just wanted to say thanks for building this out. It'll be great to be able to see custom metrics in the dashboard which the user can enter themselves, and yes, the app can remind the user to enter them. This came to my mind since I can imagine people wanting to track body fat% or other specific body measurements which can't just be picked up by a tracking device, nor can be found in Apple's Health App.

0x54MUR41 3 days ago 0 replies      
DISCLAIMER: I don't mean to promote an application

This application reminds me about a mobile application that is similar with this.

An application that called "Exist" [1]. Exist tracks everything in one place. It tracks your fitness data and other services connected with yours.

[1]: https://exist.io/

bl4ckdu5t 3 days ago 0 replies      
Splendid interface. I'll never stop saying I'm scared of the future but we have to embrace it.I am paranoid to use this as there isn't a TOS or maybe it's just me that couldn't find any but what if early sign-ups will just be experiments for them to work on? what if there's some way this affects the users later?
sktrdie 4 days ago 1 reply      
What tech did you use for developing the interface? Any framework, library, architecture (thinking React/Flux/Angular) or was it done by scratch?
pookieinc 4 days ago 1 reply      
Really beautiful work, I'd love to see a technical write-up on how this was achieved and would definitely follow something like that.

I'm interested in joining this, however I don't have such a wrist device. Which tracker would be best for receiving the most accurate data? Is there a recommended tracker (Fitbit vs. Jawbone vs. etc.) that provides more data, so this app can track more things for the user?

desireco42 3 days ago 0 replies      
It looks cool, but I still don't see how this site will give me more info then in my example my Basis website. It claims it improves productivity but I fail to see how it would do that, aside me analysing graphs, that tends to waste time...

Anyhow, if anyone has good info that would help me understand this, I would appreciate it.

It is really impressive site.

Rodeoclash 4 days ago 0 replies      
Make it work in realtime, add a camera feed and you would have realised the Colonial Marines mission screens from the APC in aliens.


ThomPete 4 days ago 0 replies      
This looks good, but the real value would be in an AI monitoring you in the background, rather than yourself having to look at your stats.

In other words it suffer a little bit from the hacking in movies syndrome. The real interesting application is the continues monitoring by the system not by me. That kind of defeats the purpose.

Glyptodon 4 days ago 0 replies      
Looks cool and useful, but I'd never want a 3rd party to have that dataset. Could care less about privacy policies. (They're meaningless.)

On a different level, it's very much not clear from the site itself what hardware you need to have to get all the various metrics it's displaying.

_arc 2 days ago 0 replies      
What's with the UI here? Sure, it looks nice, but do you need all that fanciness for just a health dashboard?
avitzurel 4 days ago 0 replies      
Looks awesome.

I am not sure this is your target market, but using TrainingPeaks as an integration will solve a lot of problems for athletes and coaches.

TrainingPeaks is the number one tool today for serious athletes, especially ones that have coaches and teams.

robbles 4 days ago 0 replies      
> 1.4 hours on Gyroscopedev:5000

Looks like someone's been working on the dev version :)

ilurk 4 days ago 0 replies      
Do you still have the previous versions of the site?

I did like the places/travel pages.

DiabloD3 4 days ago 0 replies      
My only problem with Gyroscope is there is still no alternative logins. I don't use Facebook, and I don't want to make a login for them just to use it.
namelezz 4 days ago 0 replies      
The UI is nice but it runs very slow on my FF.
KenanSulayman 4 days ago 0 replies      
I like it and I upgraded. Thanks!
chejazi 4 days ago 1 reply      
How will you measure Glucose levels?
Raphael 4 days ago 0 replies      
Oh, another mystery-meat menu.
Fastidious 4 days ago 0 replies      
Facebook account required.
Facebook to open-source AI hardware design facebook.com
197 points by kungfudoi  4 days ago   59 comments top 8
vonnik 4 days ago 3 replies      
I'm glad they're open-sourcing this, but I have to say that making 8 GPUs work together is not that big of a deal. Companies like Cirrascale are making up to 16 GPUs scale linearly with a blade.
scott_s 4 days ago 3 replies      
Big Sur is twice as fast as our previous generation, which means we can train twice as fast and explore networks twice as large.

Can a machine learning expert comment on this claim? There are many domains in computing where doubling your compute power does not halve your execution time, or allow you to double the size of your input. (For example, your algorithms are worse than O(n), or you have hardware communication bottlenecks.)

daveguy 4 days ago 5 replies      
Can we please not call open sourcing a build plan of off the shelf components, open-source "hardware design". I don't think backblaze called their case designs open source hardware. It just doesn't fit when we have open source circuit board designs. How about open-source build specification instead?
sawwit 4 days ago 5 replies      
How come they aim for GPU technology rather than for neuromorphic computing ASICs?
chimtim 4 days ago 1 reply      
Is this a machine with GPUs? You can order a similar looking design from acmemicro since last 4 years atleast?
nickpsecurity 4 days ago 0 replies      
Good work FB on another great deliverable for open HW configurations and improved datacenter design! :
guelo 4 days ago 0 replies      
I was hoping it was the AI that was designing the hardware.
Show HN: FuckFuckAdblock github.com
348 points by mechazawa  1 day ago   266 comments top 18
jbb555 18 hours ago 6 replies      
As far as I'm concerned advertisers had their chance but they totally abused their chance turning the internet into an ad-laden hellhole and now I just block everything I possible can. If a site decides it doesn't like this then it's literally no use to me.

They had their chance, they abused it beyond all possible tolerance. I hate advertising on the internet now so much that if anything does manage to get past my blocks I make a conscious attempt to avoid buying anything from the scumbag organisation that thinks forcing their crap on my is anyway acceptable.

SCAQTony 20 hours ago 2 replies      
I would not mind ads at all but ads are not really ads: they are trackers, beacons, analytics, and privacy busting cookies with none of the above asking for permission. I am blocking about 2,000 different versions of all of the above according to Ghostery. EFF's Privacy Badger is blocking a bunch to. We are being digitally assaulted. We need this stuff!
atom_enger 23 hours ago 12 replies      
At some point I really think we'll resort to blocking ads at the network level. There's no way to detect a block at that level and we'll truly be able to control the content that way. I've seen a few http/dns ad blockers here on HN already so we know it's possible and being thought about. These browser based detections are just a game of cat and mouse at this point.
dantetheinferno 20 minutes ago 0 replies      
I hate to ask this, but could someone explain where this stops the FuckAdBlock? I can't see it in the code.

EDIT: Oh nevermind; it calls a FuckAdBlock instance and then overrides it. Ha.

ainiriand 18 hours ago 1 reply      
Hi guys. I guess I'm one of the bad guys. I developed the foxplay, natgeoplay adblocker detectors. I used a very simple approach because we know that anyone that wants the content and not the ads is going to have it anyway.The ads in the pages are something that is needed. What is not needed are, as other commenters said, 3rd party cookies, trackers and analytics beacons. Keep up the good fight.
codezero 23 hours ago 1 reply      
I'm very pleased they include the link to the Trace Buster Buster: https://www.youtube.com/watch?v=Iw3G80bplTg
archibaldJ 21 hours ago 0 replies      
Or functionally: https://github.com/0a-/FuckFuckAdblockFunctionally

 var _ = function(){}; var fuck = function(fn,fns){ fns.push(fn); //side effect here since this is OO. Otherwise could have written: fns = fns.concat(fn) var partiallyFuck = function(fn){ return fuck(fn,fns)}; var thunk = function(){ fns.map(function(a){a(); window.fuckAdBlock = fuck(_,[]); //forced to do side effect here due to OO window.blockAdBlock = fuck(_,[]); //forced to do side effect here due to OO }); return true}; return { onDetected: partiallyFuck, onNotDetected: partiallyFuck, check: thunk, emitEvent: thunk } window.fuckAdBlock = fuck(_,[]); window.blockAdBlock = fuck(_,[]);

wtallis 20 hours ago 0 replies      
It looks to me like this is just a special case of what NoScript's surrogate scripts feature has been providing for years. It allows substituting your own no-op scripts for whatever scripts you'd rather not run, but want to fool other scripts into thinking have been loaded.
waynecochran 1 day ago 6 replies      
The license is even more flexible than the MIT license.
crystalclaw 23 hours ago 1 reply      
I went to the linked http://fuckadblock.sitexw.fr/ and their publicity example didn't load. Looks like my adblock rules are doing their job!
wldcordeiro 1 day ago 1 reply      
Make it work with the beta as well to preempt its release. :)
mukundmr 21 hours ago 1 reply      
How about this instead? Plugs into U-Block Origin: https://github.com/reek/anti-adblock-killer#instruction
Orangeair 17 hours ago 0 replies      
jayzalowitz 19 hours ago 3 replies      
A part of me really wants a system that asks the browser for a "hey, you wanna give money instead" token that lists a payment system and a unique id. I don't wanna really hurt content creators.
manigandham 1 day ago 3 replies      
But why?
simfoo 21 hours ago 0 replies      
I have Ghostery and ublock origin installed, after adding this to my custom blockers it still tags me as "adblock enabled"
recroad 21 hours ago 8 replies      
As a content creator who runs two websites and relies on ads to pay bills because my actual job doesn't cover the costs, I say fuck anything that's trying to block ads.
Show HN: Cloudcraft Create AWS diagrams cloudcraft.co
280 points by Rezo  2 days ago   72 comments top 31
lbotos 2 days ago 2 replies      
Some feedback: "In the delta of your current understanding and the reality of the cloud, lies cost savings and insight." This sentence was a little confusing. Maybe:

Cloudcraft make it easy and clear where your could is and where it could go.

Something a little simpler and more direct.

cddotdotslash 2 days ago 1 reply      
This kind of tool was sorely needed. I've used countless online diagraming tools with support for AWS icons and every one takes an unnecessarily long amount of time to get working. I ultimately settled on making my own diagrams in PowerPoint using the AWS PPT template. This tool is much easier to use and makes things look great without much effort.
Rezo 2 days ago 1 reply      
A word of warning, the UI doesn't work too well on mobile devices yet, it's sized for desktops. I hope to fix this in the near future.

Here's an actual screenshot kindly posted by Jeff Barr of what the UI looks like https://twitter.com/jeffbarr/status/674357220847443968

edjboston 2 days ago 2 replies      
This is awesome. Wouldn't it be cool is this were wired up to an orchestration tool like Terraform to visualize TF states?

Another level would be container infrastructure. Our new project is Docker/Kubernetes so the AWS infrastructure mostly just vanilla resources. It would be fantastic to have a similar tool for the internals of our micro SOA.

Great job!

joeyspn 2 days ago 1 reply      
Am I the only one who thinks that adding a CloudFormation export (json) to this tool would be awesome?
lobsterloga 2 days ago 0 replies      
what would be super cool if given AWS credentials, it'll auto draw the diagram for you?

The tool is awesome, btw! Thank you VERY MUCH.

jsz0 2 days ago 0 replies      
Looks good. I was able to put together a fairly complex diagram in about 15 minutes. Thats better than I can do with most traditional diagramming applications. In that way having limited options helps because there's less things to get distracted by. Anytime I try to make a diagram in Visio or Omnigraffle I spend (at least) 15 minutes just adjusting fonts, sizing, etc. It ends up being a very frustrating experience. So much so that I've mostly gone back to making paper diagrams.
bitwarrior 2 days ago 1 reply      
Static resources go right in the garbage!
narsil 2 days ago 0 replies      
Awesome! This is solving a real problem when it comes to visualizing infrastructure. I recently provided AWS with feedback at a focus group in SF on releasing a tool like this. So far, I've had to use Gliffy to create diagrams using Amazon's official Simple Icons. [0]

Minor note: my screen is fairly narrow, and the Preview/Export buttons hide the action buttons for EC 2 nodes. I notice the element has a z-index of 1. It might be better to ensure the Preview/Export buttons are always hidden by any boxes appearing when the user clicks a component.

[0] https://aws.amazon.com/architecture/icons/

rurounijones 21 hours ago 0 replies      
You have a "sign in with google" option. Can we get others? (sign in with Github would be nice, that is the nexus of all my dev related stuff and use it in a bunch of other dev related services.)
mariocesar 2 days ago 1 reply      
Something interesting to do from now on: Search in Google `site:https://cloudcraft.co/view` You will see the shared links from existing diagrams.

Right now Google had just indexed one https://cloudcraft.co/view/d3908616-4d96-4d5c-8747-6856e8801...

mej10 2 days ago 0 replies      
Looks cool! I am going to draw out some diagrams using this next week.
yranadive 2 days ago 1 reply      
FWIW react components are alerting :)"Warning: Failed propType: Invalid prop `value` of type `string` supplied to `DropDownMenu`, expected `object`. Check the render method of `SelectField`."

The app is beautiful. The most important you hit right on target is the perspective. Visually this perspective allows the best understanding.

awjr 2 days ago 0 replies      
This is so damn sexy (and really useful)! Kudos
mikejholly 2 days ago 0 replies      
This is excellent! Well done.

Would be awesome if it supported non-AWS (more generic) resources as well.

frik 1 day ago 0 replies      
How have you created the beautiful 3D icons? The look better than from the AWS icon template site. :)
chr15 2 days ago 2 replies      
How are you getting accurate pricing for AWS components? I know Amazon's Simple Calculator uses an undocumented pricing endpoint, but that endpoint could change and become inaccurate.
cypret 2 days ago 0 replies      
Great job with this, I can't wait to play around with it more. The bane of my life is creating graphics but this is something I will actually use.
redmaverick 2 days ago 1 reply      
Why is it free? Are you planning to monetize it in the future?
ceejayoz 2 days ago 0 replies      
This is sorely needed and really well done thus far. I've always envied the beautiful AWS diagrams they feature in their case studies.
omginternets 2 days ago 2 replies      
I like the idea a lot, but I'm not a fan of the fancy 3d visualization. Is there a way to have a simpler (2D) representation?
efm 2 days ago 1 reply      
This solves a real problem, beautifully.

Minor complaint. I couldn't find how to restart the tutorial.

teddythetwig 2 days ago 1 reply      
Amazon recently introduced a tool similar to this called cloudformations designer.
newman314 2 days ago 0 replies      
This is very nice. It would be great to have vCloud Director support at some point.
impostervt 2 days ago 1 reply      
Is the grid/menu/etc all custom, or are you using some UI libraries?
obulpathi 2 days ago 0 replies      
Wondering if there are any plans to support Google Cloud?
Svenstaro 2 days ago 1 reply      
Clicking "pricing" doesn't seem to do anything for me.
alexchamberlain 2 days ago 0 replies      
Just wondering what the motivation was for making this free?
sylvinus 2 days ago 1 reply      
Great tool! How do you extend the grid?
jafingi 2 days ago 0 replies      
Wow. Really awesome! Great work.
motles 2 days ago 0 replies      
Moving the Washington Post to HTTPS washingtonpost.com
206 points by joosters  4 days ago   106 comments top 15
n0us 4 days ago 10 replies      
The Washington Post and the New York Times both seem to have excellent engineering teams and both of their websites are in my opinion some of the best news sites that I've seen on the web. (Vox is also very nice in my opinion though they do not have ssl support apparently)

I would gladly pay a netflix-like subscription that gives me a "pass" to top news sites with no advertising. I just don't want to manage a dozen subscriptions to sites and it's difficult to choose between the different options that they offer like "tablet, web, paper" "web, paper" "tablet only" etc. Just take the difficulty away and give me access to the content in whatever format I want, with an addon fee for print delivery.

jakobdabo 4 days ago 0 replies      
Let's hope WP becomes a trend-setter among news websites in this direction. The only other one (that I know of) is RT.

SSLLabs grades WP's HTTPS support "A" [1].

I can see that they don't set HSTS headers and there's no OCSP stapling.

Also, their cookies are not set as "secure".

[1] - https://www.ssllabs.com/ssltest/analyze.html?d=www.washingto...

joosters 4 days ago 2 replies      
Why aren't big online publishers more concerned about the junk that the ad networks are attaching onto their sites?

If an expert like Google can't even ensure that ads don't auto play or include sound effects, what hope is there that they can protect against malware and other dangerous content leaking in?

cpeterso 4 days ago 2 replies      
Are there any ad networks that are 100% HTTPS? HTTPS ads would work in both HTTP and HTTPS content pages, but HTTP ads may cause mixed content problems. Are the ad networks worried about HTTPS latency? Or are they just lazy? :)
esaym 4 days ago 7 replies      
I am not sure how I feel about the general HTTPS-ization of the web. I've used squid (http://www.squid-cache.org/) and dansguardian (http://dansguardian.org/) for nearly 15 years now. It greatly speeds up my web and keeps smut away from my family.

Yet it is becoming more and more useless everyday because of HTTPS. I used to be able to quickly fly through google maps because most of the images were in cache, even with only 1mbs internet. Then it went HTTPS only. So I started using mapquest, then it too did the same. Bing maps still allows some non HTTPS images, so I now use that sometimes.

I can see how some sites might want to be more private in nature. But news and maps websites I am not seeing the point.

frik 3 days ago 1 reply      
Could you reduce the 114 requests and the 7.77s page load? Do you really need 6 analytics and 15 3rd party services?

Chrome DevTools timing said: 2.71s Scripting; 636ms Rendering; 210ms Painting

WashingtonPost.com crashes my iPad Safari, see: https://news.ycombinator.com/item?id=10697235

nandhp 4 days ago 3 replies      
> not only enable HTTPS on our site but also use our own custom EV certificate

Their certificate doesn't look EV to me. (I checked both developer. and www.)

darksoul 4 days ago 0 replies      
What's fascinating is how fast the washington post is, and how you get different image types depending on the browser you use. Off to check out that CDN provider they use.
MaxScheiber 4 days ago 0 replies      
Does anyone else have the following issue with the footnote hotlinking on this blog post? When I click the "return" icon on a footnote to jump to the point in the text where the footnote is, I end up several lines below where I should. The sentence with the footnote is concealed by the black WaPo header bar.

I can replicate this behavior in Chrome on Ubuntu, Chrome on El Capitan, and Safari on El Capitan.

autotune 4 days ago 1 reply      
> Today, more than 99% of our traffic is redirected to HTTPS.

So what are the .9% of traffic that aren't? Web crawler bots?

fiatjaf 4 days ago 0 replies      
I thought they used Wordpress.
scient 4 days ago 0 replies      
groan Not another site claiming blockchains are the solution for everything, please! I've yet to see an actual useful blockchain usage outside of payments.
foldor 4 days ago 0 replies      
As an aside, did the font on this page bother anyone else? The lowercase "w" in particular just kept confusing me into thinking it was a word in italics and my mind kept stressing it. I looked it up, and the font-family is "Ubuntu".
swagv 4 days ago 2 replies      
Sadly, like too much engineering writing, no attention is given to the "why".
kremmer21 4 days ago 1 reply      
Is this supposed to be an impressive project? Because if it is, I feel sorry for the developers working there.
Yahoos Engineers Move to Coding Without a QA Team ieee.org
230 points by teklaperry  3 days ago   178 comments top 49
ef4 3 days ago 10 replies      
Surprised to see the negativity here. I have worked in environments with traditional manual QA, and environments where all development is test-driven and nobody is allowed to merge a feature that lacks automated test coverage.

Both the productivity and the quality were higher in the places with fully automated testing. Which is not shocking at all: does anybody really think a human can run through 800 test cases better than a computer can?

It's not a magic way to save money -- the developers obviously end up spending time writing tests. But the long-term value of those tests is cumulative, whereas the effort spent on manual testing is spent anew every release.

Manual review is still good for noticing things that "feel wrong" or for helping think up new corner cases. But those bleed into product owner & design concerns, and aren't really a separate function.

chojeen 3 days ago 1 reply      
I worked at Yahoo before and during this period, first as a QA contractor and then as a full-time developer.

Before the switch, our team (advertising pipeline on Hadoop) used the waterfall method with these gigantic, monolithic releases; we probably released a handful of times a year. Almost without exception, QA was done manually and was painfully slow. I started to automate a lot of the testing after I arrived, but believe you me when I say that it was a tall order.

Soon after I moved into development, QA engineers without coding chops were let go, while the others were integrated into the development teams. The team switched over to agile, and a lot of effort was made to automate testing wherever possible. Despite some initial setbacks, we got down to a bi-weekly release cycle with better quality control than before.

Around the time I left, the company was mandating continuous delivery for all teams, as well as moving from internal tools to industry-standard ones like Chef. I left before it was completed, but at least as far as the data pipeline teams were concerned, the whole endeavor made the job a lot more fun, increased release quality, and weeded out a lot of the "that's not my job" types that made life hell for everyone else.

mstade 3 days ago 2 replies      
I don't think it's that cut and dry. I've worked in some places where the QA team was useless, meaningless red tape to get your stuff deployed. They wouldn't do much but sign off on deployment at some point, yet bore no responsibility if shit hit the fan. In these cases, they really were just an unnecessary cost and you learned pretty quickly to make sure your tests were in place, that you were testing for the right things, and so on.

But then there were the other QA teams. The people that would just reject your stuff outright if it didn't have tests (no matter if it worked) and when the tests passed they would look at things truly from a customer perspective. They would ask really uncomfortable questions, not just to developers, but to designers and business alike. They had a mindset that was different from those creating things; they were the devil's advocate. These people did much, much more good than harm, and they are few and far between. Unfortunately, while I believe they were incredibly valuable, business thought otherwise when cuts came around..

minimaxir 3 days ago 4 replies      
The stereotype that QA-is-pointless in Silicon Valley is persistent and actually annoying. There will always be issues that even the most comprehensive test suite will miss.

Startups still glorify Facebook's "Move Fast and Break Things" without noting that Facebook has backpedaled from that. After all, people expect startup software to have issues, so what's the harm? Technical debt? Pfft.

Engineers are not the best QA for their own code since they may be adverse to admitting errors in their own code. QA engineers are not as empathetic.

Disclosure: I am a Software QA Engineer in Silicon Valley.

smithkl42 3 days ago 0 replies      
Our approach - in a much smaller company - is that all stories should have automated tests before they head off to QA. QA's job is to make sure that the story in question works correctly, it's not to find regression bugs. If QA finds a bug in the story, we write a test to catch that before we resubmit it. Over time, we have enough tests at all levels of the system that QA doesn't generally need to worry about regressions: just making sure that the latest story works as advertised.

This approach allows us to stay agile, with small, regular releases, while also making good use of what QA folks are actually good at.

diivio 2 days ago 4 replies      
This isn't surprising.

Microsoft switched to this model a few months after Satya took over.

For the majority of Microsoft teams it worked really well and showed the kinds of results mentioned in this yahoo article. Look at many of our iOS apps as an example.

But for some parts of the Windows OS team apparently it didn't work well (according to anonymous reports leaked online to major news outlets by some Windows team folks) and they say it caused bugs.

First of all I think that argument is semi-BS and a cover up for those complainer's lack of competence in testing their code thus making them bad engineers because a good engineer knows how to design, implement, and test their product imo. But I digress.

I in no way want to sound like a dk but as an engineer it is your responsibility to practice test driven development but that's not enough.

Like reading an essay you usually can't catch all of your own bugs and thus peer editing or in this case cross testing is very useful.

You should write the Unit tests and integration tests for your feature


There should always be an additional level of end to end tests for your feature written by someone else who is not you.

Everyone should have a feature and design and implement it well including its Unit tests and integration tests BUT they should also be responsible for E2E tests for someone else's feature.

That way everyone has feature tasks and test tasks and no one feels like they are only doing one thing or stuck in a dead end career.

ergothus 3 days ago 0 replies      
I'm curious to find out if my expectations of QA are unrealistic.

I'd _expect_:

* Devs write automated unit tests galore, plus a smattering of integration tests

* QAs write some acceptance tests

* QAs maintain a higher level of broad understanding of where the org is going, trying to anticipate when a change in Team A will impact Team B _before_ it happens. They also do manual testing of obscure/unrepeated scenarios, basically using their broader knowledge to look for pain before it is felt.

The above hasn't happened anywhere I've been (though each point HAS happened somewhere, just not all together).

One thing in particular I've noticed is that a good QA is a mindset that a dev doesn't share. Devs can learn to be BETTER at QA than they are, but I honestly think it's not helpful for a Qa to be a Dev or a Dev to be a QA - they are different skill sets, and while someone can have both, it's hard to excel at both.

eecks 3 days ago 1 reply      
I can see why this would work in a place that uses QA as a crutch.

All developers should aim for no bugs and test their stuff themselves but of course when deadlines are looming its easier to just code and let the QA team pick it up.

steven2012 3 days ago 1 reply      
This is pretty much how things go in today's environment, especially in the startups that I've seen. More things are being pushed directly on devs, which is why we earn as high a salary as we do. Traditional QA is pretty much dead, no one should be doing that now if they want to have a career in tech.

Where I work, devs do the QA, and most of the devops work as well. It's the new reality, and anyone who thinks otherwise will be obsoleted.

projectileboy 2 days ago 0 replies      
If all your QA team does is run through a bunch of tests that could be automated, then by all means, automate the tests and get rid of your QA department. However, good QA folks have a valuable skill, which is that they think of interesting ways to break software. Not many programmers do this very well.
DanielBMarkham 3 days ago 0 replies      
Welcome Yahoo engineers to the year 2010. Or 2005. It's nice here.

The suckiest part of this story is the number of folks who are stuck with gated handoff processes that can't see how this would ever work. Some of those folks might be waiting 10, 20 years catching up to the other folks.

Just to be clear, QA the function isn't going anywhere. It's all being automated by the folks writing the code. QA the people/team? Turns out that this setup never worked well.

I work with tech organizations all the time. I find that poor tech organizations, when faced with a complex problem, give it to a person or team. Good organizations bulldoze their way through it the first time with all hands on board, then figure out how to offload as much of that manual BS as possible to computers. If they can't automate it, they live with it until they can. Same goes for complex cross-team integration processes.

devonkim 3 days ago 0 replies      
This reminds me a bit about how SREs and developers conflict on contradictory goals. With a self regulating system that gets established between development and operations, if your code is bad in prod, you'll spend more time in operations to try to take care of the mistakes made in development. If SREs really don't get along with how developers throw over crap and quit, the developers will get the pagers instead. This move to consolidate test and development seems to be consistent with recent trends to pile upon more and more work for developers in the efforts to reduce siloization.

So I'd have to ask how getting rid of QA has affected the pace of feature development.

alrs 3 days ago 0 replies      
"Some of the engineers really cared about system performance types of things, so they joined related teams. Some started working on automation [for testing], and they thought that was greatthat they didnt have to do the same thing over and over."

There is still QA, it's just automated QA. Welcome to the 21st century.

Xyik 3 days ago 3 replies      
Surprised that people are finding this unusual, in web/mobile anyways. In my experience most engineers do some level of QA themselves, particularly in start-ups < 1000 people. In what ways does an engineer being their own QA negatively impact the company?
BogusIKnow 2 days ago 0 replies      
Today QA is not manual testing.

Today QA is talking with product/UX, taking the end user and customer perspective, wearing a quality head end to end over features an cross devices, doing explorative testing for stuff that does not make sense to a customer (mostly what's created by the inference of different features or cross device interaction).

kemiller 3 days ago 0 replies      
I think QA has been misaligned all this time. They're not part of engineering, they're part of product management. They're the low-level eyes and ears for the product team. Automating checks for the issues they uncover is absolutely an engineering function, but user-oriented holistic testing is not.
Kabacaru 2 days ago 1 reply      
Manual testing is basically a 0 skill job. Can you click around this website and tell me when you see a bug. This is the most common form of QA but adds very little value that can't be added with more reliability using automation.

Given this QA can still bring value. The two roles that they really add value in are a Test developer specialist writing non-flaky automated tests, and a BA type role where they have conversations that expand a product owner's idea into an implementable feature.

Given that neither of these roles require manual testing, if a QA team has over specialised on manual testing, there's little value in keeping it.

TheAndruu 3 days ago 1 reply      
Totally buy they could reduce actual errors that matter.

I've worked with many a QA who would get bent up over a detail outside of the spec that didn't really matter, and where all QA testing was manual.

Coders (good ones) are well equipped to automate processes, and to do so quickly, and this extends to integration testing.

beat 3 days ago 0 replies      
Every agile story should have a testable completion point, agreed upon by development and the customer. If whether or not the story is "done" is vague and arguable, it's not good enough to do.

One of the big problems here, and where QA professionals can add real value, is defining that "done" point. Customers are often not very good at it. Their idea of what they want is too vague. They want developers to just build something, and they accept or reject it when they see it (and fault developers for not building it right).

But really, all story completion criteria should be testable, and developers should be able to demonstrate the tests. The job of QA shouldn't be to test, but to make sure the developers are actually testing what they claim to test.

loren_dunlop 14 hours ago 0 replies      
i'm in QA - have been for 12 years. Testers who can only perform manual testing, and organisations that only test manually are the product of companies realising they should 'do some qa', and managers who do not understand SW development signing off to build large, manual only test teams.

It's inefficient, there is a very slow rate of feedback to devs, not much can be done until there is a working UI - so it all lends itself to the broken waterfall model of code code code, then 'do some testing' right at the end of the project - which has already seen overruns from dev squeeze qa time out.

Manual QA testers are relatively cheap on paper - so managers don't see a problem with building a team this way.

I'm not sure this will ever go away, but as someone who tries to learn every year, and master his career, I welcome Yahoo's choice. I see a role for a highly skilled 'developer in test' role superseding the traditional, ineffective manual QA role. Someone who can build automation frameworks quickly, be responsible for maintaining them and test data, and provide rapid feedback to devs. Devs should still be carrying out unit testing, code reviews etc, but I do believe a role still exists for someone to focus on QA, just with a lot more skills, providing far more rapid feedback, with less dependencies on the devs for test environments.

throwaway_xx9 3 days ago 0 replies      
Historically, Yahoo! operated for the first 10 years of their existence without formal QA teams. The first USA QA hires were for Japanese product QA.

Nowadays OpenCV is used a fair amount, and they're migrating to modern industry-standard tools.

whistlerbrk 3 days ago 0 replies      
I agree in principle for most products, no QA, no testing. Why?

* Everyone should do QA and implement their features own UI/UX, by following the pattern the application and framework sets tuned by an actual designer* An environment where production issues and bugs are prioritized above everything else should be created and fostered* To paraphrase Rich Hickey's analogy on the matter: writing tests is like driving around relying on the guard rails to keep you in the lines. That is (my interpretation):* If your code is this fragile to constantly require testing you've chosen poor abstractions.

slothguy72 3 days ago 1 reply      
They removed only MANUAL QA testing. There is a big difference between removing the QA team entirely and automating QA work.
datashovel 2 days ago 0 replies      
In the past I've even loosely spec'd out a system that would build integration tests simply from crawling a website. In my head. I'm surprised this hasn't become a bigger priority from some of the biggest tech companies.

I know it would be a tough problem and a big project, but I think with only a small amount of human interaction you can build all the integration testing you would ever need simply by allowing the crawler to build them for you.

In fact the way I imagine it would work, the system would automatically build a framework and a user could (in a very structured way via structured UI) coerce the integration tests in small ways to ensure it understands what's going on. For example: "This form is used for registration". "This form is for logging in".

grandalf 3 days ago 1 reply      
Yahoo's main issue is UX and usability, not software quality, so this sounds like another way of saying Yahoo laid off its QA team.
ascendantlogic 3 days ago 0 replies      
I stand in the middle ground on this one. I fully believe that rote QA testing with huge volumes of test plans is a waste of everyone's time. However automated testing doesn't take into account the fact that people are almost always the primary users of your software, and so I feel somewhere there should be a person or persons who occasionally smoke test the application to make sure that things are working cohesively from an end-user perspective and just making sure everything makes sense. If this person is the prototypical product owner, that's great, if you can find one that's not in meetings all day...
transitorykris 3 days ago 0 replies      
Removing dedicated QA (whether they do manual testing like in the article or write automated tests) and forcing the developers to take this on themselves is okay. Alternatively, I've had a lot of success with having development teams take operational responsibility for their code. They are not only naturally incentivized to take on automating QA, they also move toward continuous deployment and become more involved in thinking about the product. The safety and speed that's gained is seeming to result in teams that stay small. It's not for everyone, and caused attrition early on, but talking about these practices during interviews has attracted the right people.
LoSboccacc 2 days ago 0 replies      
"What happens when you take away the quality assurance team in a software development operation? Fewer, not more errors."

And what happens when you close your eyes? Reality disappears?

Automated testing is a way to completely remove customer advocates out of the loop. Correct UX doesn't mean good UX and unless someone can automate the test of all the non-quantifiable qualities of good and intuitive they're gonna push loads of engineering driven interfaces to their users.


CodeSheikh 3 days ago 2 replies      
Dev writing automation tests for their code is kinda pointless. It would be better to have another dev or a different team such as automation engineers writing regression automation tests. Automating regression tests is definitely better than manual QA-ing the same 1000 tests over and over again. There has to be a good balance. Have extensive coverage of automated regression tests and let manual QA test new features. This will at least increase the frequency of release cycles. Getting rid of an entire QA dept is somewhat equivalent to shooting yourself in the foot.
RyanZAG 3 days ago 0 replies      
Ultimately, even with automated testing, someone has to do the manual testing of checking that it's actually providing the value it's meant to.

When you remove the manual QA team and switch to staged rollout, you are moving the manual QA burden onto your users. You still have that manual QA team - they're the first bunch of users in your staged rollout plan - you just don't pay them anymore and gather their feedback through bug reports. Users are used to buggy software because of other companies who do this (Google, etc) so they carry on being users anyway.

miles_matthias 3 days ago 0 replies      
Quality assurance is important whether you do that via humans or code, but the thing that always bothered me about Q&A was that the ones I worked with were mindless people simply looking at the feature request and the functionality on the page and comparing the two without any thought towards the actual product, business, or user.

And in that system, the developer is completely removed from the product and is just another factory worker. The closer engineers can be to users (with design to translate obviously) the better for everyone.

mixmastamyk 3 days ago 1 reply      
Great. I worked for a company that didn't invest in QA, it was consistently a !@#$% mess. When you do this, the need simply shifts to the customer. I wouldn't install our software until the 4th or 5th hotfix patch was available.

Certainly, I'm an advocate of a more responsible dev team sharing the quality tasks and continuous integration too. But no QA at all? Hahah... maybe if you're a web portal that no one depends on for business-critical needs.

Edit: I guess the truth hurts.

johnrob 3 days ago 0 replies      
When measuring the effectiveness via reduction of issues, how do you account the natural stability introduced by reducing the updates/week each developer ships? When devs are tasked with code reviews and/or QA, this is time that could have been spent on their own features. In other words, if the product is stable today, and everyone's on vacation (no new updates), the product will generally remain stable save for unforeseen usage patterns.
srp0010 3 days ago 0 replies      
Updated TL;DR: QA is changing - just like everything else.

The article makes the assumption that QA == manual QA which as a quality professional is false. Quality is about measuring risk across the development process. Immature team need manual QA while mature (in a process/quality sense) teams need much less (or none).

Quality professionals who want a sustained career needs to learn development processes, opts, documentation & monitoring. We make teams better.

xacaxulu 2 days ago 0 replies      
If you think about a QA team as your customer, as any downstream department in the work pipeline truly is, you realize that in order to make full use of them and to maximize your efficiency, you as a developer should write automated unit tests to cover the user stories or feature requirements, allowing QA to work on the nasty edge-cases.
plinkplonk 2 days ago 0 replies      
When I was working at ThoughtWorks, we had devs writing automated unit tests with close to 100 percent coverage, and also QA (who also automated as much of their testing as possible, using a variant of the FIT framework in those days) finding significant bugs and show stoppers,.

In my experience, one is not a substitute for the other.

dkopi 3 days ago 4 replies      
This is actually a story about the triumph of continuous integration and staged rollout.By shipping code constantly, but slowly rolling it out to users - bugs can be detected very quickly by the users themselves, instead of employing large QA teams.

Keeping a central code repository, automating builds, frequent commits and automatic tests for code are taking away a lot of load for QA teams.

stephenitis 3 days ago 0 replies      
Rainforestqa has a unique value proposition in this space (YC some year)

Their team and product are quite good if you want to explore QA as a service. Essentially humans(turks) preform outlined and preprogrammed steps.

Their tagline "We automate your functional and integration testing with our QA-as-a-Service API. Human testing at the speed of automation."

knughit 2 days ago 0 replies      
Is it odd that this article describes, but highlight , that yahoo is a decade behind the industry here? Continuous integration and skipping QA has been the web standard for years now.
steve371 2 days ago 1 reply      
I'd vote for having a QA team. Not for quality control purpose. But to have someone think outside the box. Sometimes, you will be surprise when you talked to the QA team. And you could not get those ideas from dev peer review.
thepaintedcow 3 days ago 0 replies      
Ultimately, a good development process is about building in checks and balances. Code reviews, QA, automated testing, etc. are all part of that. It's up to each to team to decide which pieces they want. There's no right way to do it.
Animats 2 days ago 0 replies      
Why is Yahoo still coding anything? They're focusing on dumping everything other than their stake in Alibaba.
alekratz 3 days ago 0 replies      
Based on Tumblr's past performance, it's a shock that they had a QA team in the first place.
nobrains 3 days ago 0 replies      
That is kind of like having an open kitchen policy (or a kitchen visible via a glass wall) in a restaurant.
hondo77 3 days ago 0 replies      
Nobody cares how Yahoo does anything.
Apreche 3 days ago 9 replies      
TL;DR: They fired the QA team to save money. Then they made the engineers do the QA work for no extra pay.
allsystemsgo 3 days ago 0 replies      
This will end well.
douche 3 days ago 0 replies      
There's no way this could end badly...
Bud 3 days ago 2 replies      
If this is working so well, why is there still no version of Yahoo Messengera core productfor either Mac OS X or iOS?
Japanese government dismantles freedom of the press freedom.press
236 points by sprucely  4 days ago   111 comments top 12
hackuser 4 days ago 8 replies      
Neo-nationalism seems to be on the rise everywhere. Look at the UK (Scotland trying to secede, the crazy extreme right-wing party getting seats in Parliament, etc.), France (the National Front did very well in recent elections), South Korea (IIRC, textbooks are edited to replace history with nationalist propaganda), Israel, Turkey, Russia, China, Donald Trump, etc. etc.

When is the last time you heard someone stand up and point out how dangerous nationalism is, how it's historically led to wars, massacres, and other oppression and brutality. I suspect the public, hearing no criticism or counterpoints, goes along with the nationalists.

Where are the liberal and moderate conservative elite? Asleep at the wheel? Cowed? Complacent?

pippy 4 days ago 0 replies      
It's happening in New Zealand to a similar extent. This current government is raiding journalists homes who publish things they don't want[1] [2]. Even more worryingly the government recently bailed out the other major broadcasting company, and replaced the CEO with a friend of the prime minister. It didn't take long before journalists who were sometimes critical of the government were replaced [3].

1 https://www.tvnz.co.nz/one-news/new-zealand/hager-police-rai...

2 http://www.radionz.co.nz/news/national/290989/police-raid-du...

3 http://www.nzherald.co.nz/entertainment/news/article.cfm?c_i...

meesles 4 days ago 1 reply      
For a country whose culture already stifles expression and speech, this is bad news. I'm proud that supposedly so many Japanese stood up to protest these legislative actions. Having lived there, I can say that the government comes across as way more untouchable and heightened than here in the US, and that it takes a LOT before action can be taken against them. The US messing in their politics in order to get more aid in their wars is infuriating and makes me worry for the future of such a rich country.
euske 4 days ago 0 replies      
So, as an average citizen in the country, what should I do?

I'm just a tech geek guy. I write software at work, and I teach sometimes (for CS related topics). I write OSS. I occasionally write a blog, but feel terribly helpless for spreading my view with that. I don't think going to a rally helps the cause that much (because I feel it only pleases the curious media). I'm probably a bit more politically aware than the average people here, but I don't want to devote myself to much political activities.

Probably I should continue to do the small things that I'm currently doing (e.g. occasionally bringing up the topic to friends and advocating more openness of the society, etc.), but what else? Does anyone have any idea?

alexandercrohde 4 days ago 0 replies      
I wish this post gave a better treatment to the discussion. By presenting such a stark claim on a blog, with little justification it leads to polarization that may make some who are open-minded to the issue actually take such concerns less seriously.

I don't have any answers myself, I'm learning a lot but here's what I glean:

- Reporters without borders (RWB) is a neutral cause dedicated to investigating freedom of the press in various countries, they explain their methodology on their website.

- RWB has moved Japan down to 61 on the list of nations with the most free press. However, the US is at 49 for comparison, so saying Japan has "Dismantled freedom of the press" because they are 61 seems to be an overstatement

- The piece presents a theory that America is trying to use Japan's military. Time will tell.

- The style comes across as vaguely rambly

If anybody who's not too emotionally involved in the topic has some recommended reading I'd love to hear more.

ilurk 4 days ago 4 replies      
I feel saddened by this this. But at the same time I see Japan in a very complicated situation with China expanding its maritime influence in the region.

If it doesn't flex its military muscle (needing the secracy for it) it may find itself in a vulnerable situation against China. Although on the other hand it really doesn't stand a chance against China's nuclear arsenal.

If China used a nuclear warhead would the US riposte in the same manner?

What I don't get is the "Japan to battle in the middle east". Is this a "you scratch my back and I scratch yours"? Or just a cover for not saying "we're preparing for an eventual conflict with China"?

Fando 3 days ago 4 replies      
If you could redesign a system of government free from major flaws we see today, how would it work? Current systems almost seem to have a tendency to "breed" the types of malevolent, inefficient, greedy and problematic power structures we see. I think that at the root of the problem are natural human flaws. It seems almost certain that given enough time, no matter how well intentioned people are, the cumulative effect of our natural flaws inevitably evolves the kind of system/culture that attracts and perpetuates these flawed tendencies. I think the process could be compared to natural selection. Has anyone thought about it like that? I'm curious. Is it possible to design a system of government which would not be influenced by inherent human flaws? A system which, by design, has a tendency to "select" characteristics that shape it to be more beneficial, free from its current flaws?
codingthebeach 4 days ago 2 replies      
"In preparation to join US wars, Japan dismantles freedom of the press". What US wars? Populist hogswaddle. Such assertions should be backed up if you're going to make them. As for the supposed rise of "nationalism", to the average liberal, ANY movement in the direction of national security, or any display of pride in one's country, is not just nationalism but fascism, racism, and worse.
mtgx 4 days ago 2 replies      
Why does it look like democracy is on a declining trend all over the world, even as the Internet is more used by more people than ever?
acd 3 days ago 1 reply      
I think that the Japan public debt house is starting to crackle and they do not want the public to know. Thus they are silencing the media.

Here is a graph of Japanese debt to GDP ratiohttp://www.tradingeconomics.com/japan/government-debt-to-gdp

Here is demographic Changes in Japan's population. It shows a top slope, population starting to fall.https://en.wikipedia.org/wiki/Demographics_of_Japan#/media/F...

So a retiring Japanese population is going to pay for rising public debt. Sustainable in the long run?

wtn 4 days ago 0 replies      
I have never seen the emdash overused this badly.
pm24601 4 days ago 1 reply      
Once again the US exporting "corporate democracy"
What Your Microbiome Wants for Dinner nautil.us
204 points by dnetesn  4 days ago   93 comments top 11
asciimo 4 days ago 6 replies      
This article opens with, "[New science] is also showing us that advocates of trendy paleo and vegan diets are missing the big picture of how our omnivorous digestive system works." But it concludes with, "Pick a modest-sized plate and make meals using vegetables, legumes, leafy greens, beans, fruits, and unmilled whole grains as the main ingredients. Add some meat if you want and dollops of healthy fats on the side or sprinkled through the plant foods. Desserts and sweets are special, so save them for the special times." Omit the optional meat an you have a vegan diet. What about the "big picture" are vegans missing?
kefka 4 days ago 10 replies      
That's an interesting article, for sure. It also applies to what I've been diagnosed with as well. Type 2 diabetes.

I'm applying data science to this disease. It's what I do, and can afford. Unfortunately, my doctor thinks that pushing pills is an adequate response, which I heartily doubt.

The more I understand this disease, it has to do with either: pancreas is dead (type 1), or your glucose response/insulin response is badly out of whack due to a metabolic carbohydrate problem (type 1.5/2). I was diagnosed with a fasting glucose of 161 and a1c of 7.1 . Met with doctor, and talked about it shortly. Was prescribed metformin and testing kit with 100 strips. I am currently not taking any drugs.

I keep reading that the problem is carbohydrate overload to my genetics. Can I validate or invalidate that? Indeed. I was told to test blood sugar 1x a day for 3 months. That's inadequate, as it's only an instantaneous sample. What about after I eat? What about when I go to bed and wake up? Can I deduce anything about my pancreas and its slow and fast response to glucose? Indeed.

I've started testing myself 4x every meal(1 before meal, 3 half hour increments after meal ends), and 1 when I wake up and go to bed. Turns out that yes, carbs has, in my body a direct correlation to my blood sugar. And it's pretty stark.

Now, if my hypothesis is correct, I can control my blood sugar purely with carb counting and eating to the glycometer. If after 2 weeks this does not show significant changes in my basal glucose rate, as well as weight losses, then I will go on drugs.

However, I already have 2 good side effects: I'm not craving for food any longer, and I no longer overheat. I'm not sure how to classify those effects, but not being slave to extremely annoying bodily issues does seem very beneficent.

_fs 4 days ago 3 replies      
One topic that was not mentioned in the article that I have found fascinating is that your gut microbiome can actually affect your cravings. (1) Your microbiome accomplishes this through a host of mechanisms including changing the expression of taste receptors, making certain foods taste better; they may release hunger-inducing hormones; or they may manipulate the vagus nerve (which connects the stomach to the brain) to control their hosts eating behavior.

Are you sure you really want that extra side of chips? Or has your gut biome decided it wants that extra fat, and influences you to eat it.

Additionally, you can "reseed" your gut biome by changing your diet. Do you crave greasy food? Go vegan for 2 weeks. The new gut biome that develops and takes hold may soon have you start craving healthier foods. A change in diet can change your biome in as little as 4 days. (2) You can witness this phenomena in the movie Super Size Me. At the start of the movie Spurlock is disgusted by eating McDonalds for every meal. But, by the end of his experiment, his gut biome has significantly changed, and you can literally see his mood darken when he is not eating greasy fries and burgers. He craves, or should I say, his microbiome craves those happy meals, and is influencing both his taste receptors, health, and general mood in order to manipulate him in to a trip to the drive though.

1. http://www.theatlantic.com/health/archive/2014/08/your-gut-b...2. http://www.scientificamerican.com/article/the-guts-microbiom...

blakesterz 4 days ago 2 replies      
Somewhat interesting and different look at what different types of foods do to us by focusing on our guts and those little things that help us digest. His recommendation seems to be more complex carbohydrates, less sugar... same old conclusion, different way of getting there I guess. It's still "Eat food, not too much, mostly plants."
jensen123 3 days ago 1 reply      
It's nice to see an article that explains that too much animal protein can be bad for your health, without advocating a 100% plant-based/vegan diet. We humans are apes, and when you look at other apes, they eat mostly plants, but not always only plants. For example, chimpanzees eat small amounts of meat.

I'm a bit skeptical to the claim that whole grains is a nearly perfect food, though. Think about it - we apes have not eaten huge amounts of grains for most of evolution. Also grains contain various anti-nutrients, otherwise they would have been eaten by insects. It does not seem inconceivable that some of these insect-harming anti-nutrients could also be harmful for humans.

Many dietary studies are done with rodents, but I think (I'm not an expert here) rodents have eaten far more grains than apes during most of evolution. So studies with rodents showing that high-grain diets are healthy, are perhaps showing just that - that high-grain diets are healthy for rodents (and not necessarily apes).

classicsnoot 4 days ago 1 reply      
Rolled out this idea in a thread long ago about Apple not including a Period Tracker in their health app and it was taboo then, but here is as best a place to try again...There should be a poop tracking app. It would help a user track their waste over time, identify important red flags, act as an incredibly detailed source of diagnostic data for health care professionals, and promote healthy and happy living.Two topics that people seem so afraid of discussing are shitting and masturbating which seems so silly to me, as those are two things all humans do. If you want to be healthy, start with your poo; understanding your waste will tell you how to eat, sleep, exercise, and live better.
scholia 4 days ago 1 reply      
It's a book extract: "Excerpted from The Hidden Half of Nature: The Microbial Roots of Life and Health by David R. Montgomery and Anne Bikl. Copyright 2016 by David R. Montgomery and Anne Bikl. With permission of the publisher, W.W. Norton & Company, Inc. All rights reserved."

Based on that section, I would look for an extract from The Good Gut (1) instead. It leans more towards science and less towards popularization....

(1) http://www.amazon.com/The-Good-Gut-Control-Long-term-ebook/d...

justinph 4 days ago 2 replies      
As someone who's going through a not very fun period of C. Diff, this is pretty relevant. There is still so much we don't know about what makes a healthy and un-healthy biome, it's maddening.
triangleman 4 days ago 1 reply      
What I would like to know is what happens to the colon when you go on a 20-day regimen of broad-spectrum antibiotics such as ciprofloxacin?
edem 3 days ago 0 replies      
> The amount of meat in the Western diet can also pose problems. When consumed in relatively large quantities, animal protein is not completely broken down by the time it reaches the lower end of the small intestine. Eat too much meat and your overwhelmed small intestine delivers partially digested animal protein to the colon. When bacteria in the colon encounter intact or partially digested protein, a different kind of alchemy gets underwayprotein putrefaction.

Do someone has any information on the "relatively large quantities" part? How much meat can I eat before protein putrefaction kicks in?

coldcode 4 days ago 0 replies      
When I was in graduate school in the early 80's I wanted to get a PhD in toxicology (but didn't, became a programmer instead). Today if I was at that same stage I would study the microbiome, it's a fascinating melange of chemistry, microbiology and food science, and we still know very little about it, plus it can make a huge difference in the world. Toxicology seemed too much like slaughtering rats for a living.
A Look into Machine Learning's First Cheating Scandal dswalter.github.io
240 points by metachris  18 hours ago   50 comments top 12
TuringTest 17 hours ago 1 reply      
Long story short:

- The LSVRC "visual recognition" competition has a rule in place, that limits to twice per week how often each contestant can run their entries to the contest against the ImageNet dataset.

- The Baidu team run their tests much more often, claiming that they understood the limit was placed on a person basis and not per team.

- This rule is in place because more frequent submissions somehow distort the quality of the test, by switching its emphasis to overlearning (adapting too much to the specific data included in the test dataset) instead of true algorithmic advances, which can influence the whole machine learning discipline given this contest's high profile. ( * )

- As a result, the whole Baidu company has been banned from the competition for a year.

( * ) ("The Baidu teams oversubmissions tilted the balance of forward progress on the LSVRC from algorithmic advances to hyperparameter optimization.")

not_that_noob 8 hours ago 0 replies      
To translate to a more familiar domain, think about the SATs. After a period of study, students take the SAT where they get back the composite score of how they did, but never the actual answers to the questions on the test.

Now imagine a student can take the test repeatedly over the space of a few days, and can use the score to reverse engineer the answers to the questions. They can put in random answers and note which ones cause the score to go up. Of course the real life SATs don't allow this, and they change up the questions to prevent this sort of cheating. If this were possible, our enterprising/cheating student can derive the complete answer key over time, noting the changes in scores for each run. And once they have the key, they can ace the test. No longer is it a test of their aptitude, but rather of their knowledge of the answer key.

This scandal is analogous to this albeit contrived example. With an ML testset, it's not possible to change the data because you want it standardized so you can evaluate improvements that new approaches may bring. It's the only way to have a meaningful yardstick to measure against. Thus, the only way to prevent such gaming is restricting multiple submissions, so that you can't do 'hyperparameter optimization' - i.e. overlearn on the testset.

That's why it's cheating - it's not a measure of how well your algorithm did, but rather on how well you reverse-engineered the answer key. It's a huge disservice to the field and the people who did this should be ashamed of themselves.

philh 14 hours ago 3 replies      
Idle speculation:

> "The key sentence here is, 'Please note that you cannot make more than 2 submissions per week.' It is our understanding that this is to say that one individual can at most upload twice per week. Otherwise, if the limit was set for a team, the proper text should be 'your team' instead," Wu wrote.

I wonder whether, to a native Chinese speaker, this really does sound like it's talking about individual people, and saying "you" when one means "your team" seems really bizarre. Can any Chinese speakers weigh in?

(Even stipulating this, the affair still sounds more like malice than incompetence on the part of Wu.)

dasboth 16 hours ago 0 replies      
Congratulations to the author for both of his last 2 posts making it to the HN front page!

This explains the need to drill home the train-test idea from the last post. I hadn't thought about this before but multiple submissions do amount to multiple peeks at your held-out test set, which is a huge ML no-no.

I don't know much about LSVRC, but doesn't the way Kaggle work prevent this? AFAIR you get a "public" test-score which is used for the leaderboards, but once the deadline for submissions is up, each submission is evaluated on a held-out test set giving you a "private" score. Now that I think about it, I'm not sure how that works, I guess the accuracy they show you as your public score is only on part of the submitted rows? Regardless of how that's done, could the LSVRC organisers not do something similar?

hbogert 16 hours ago 2 replies      
Couldn't the LSVRC just limit the amount of submissions? Why would you rely on the competence or good intentions for this?
HappyTypist 16 hours ago 1 reply      
> there are almost no papers focusing on 3 or 4-layer CNNs these days, for example

What's the 'best practices' for the number of hidden layers in a CNN? 1 or 2 hidden layers?

Flockster 17 hours ago 2 replies      
So how would a better dataset look like? Does bigger always equal harder? What are the criteria to measure that?

And wouldn't video datasets be somewhat easier to analyse given the fact that you have multiple frames of the same object?

jmount 10 hours ago 0 replies      
Nice article on why it is cheating (from a mathematical sense, independent of language) to get scores from the hold-out leader board too many times (plus some methods to mitigate the effect): http://arxiv.org/abs/1502.04585 .
mikeskim 9 hours ago 0 replies      
As long as you make the public leaderboard set small, and the private one shot leaderboard set very large, the number of submissions matters very little in the final rankings. The only real issue is hand labeling the public leaderboard set to augment training data.
TuringTest 17 hours ago 1 reply      
You can skip that part and start reading at "Conclusions". The first section merely describes an unrelated experiment, show for contrast, as an example of successful research that doesn't commit the sin of overfitting. In fact, the reasons of why this is such a scandal are explained in surprisingly understandable layman prose.

Edit: A MIT review linked from the article has a simpler and more detailed explanation of how the test system was abused, and why they did it.


jacquesm 15 hours ago 3 replies      
Happy to oblige. First: Chinese companies do not cheat on 'everything', that's a generalization that holds absolutely no water. Second, begging for downvotes is such a non-productive thing to do that it is a part of the HN guidelines.

"Please don't bait other users by inviting them to downvote you or declare that you'll probably get downvoted."


king_of_nouns 14 hours ago 2 replies      
Meh.. I'm not so sure about this.

Didn't people claim "cheating" back when the first compilers started doing data flow analysis too?

Dow and DuPont to merge washingtonpost.com
200 points by gotchange  3 days ago   119 comments top 18
giardini 3 days ago 7 replies      
The two companies cultures are incompatible.

Dupont has an "employee-first" attitude that permeates the corporation and dates from its early years when the manufacture of gunpowder proved to be so risky. The Dupont family owners recompensed families of injured/dead employees by providing housing and supporting them for the remainder of their life. Safety became paramount in all operations. The attitude remains today. Despite the recent indiscretions in the LaPorte, Texas plant, Dupont is probably the safest chemical company on the planet.

If Dupont culture dominates, then this will likely prove a Renaissance for Dow. OTOH should Dupont culture be subjugated, it will ruin the value of the merger and we will all lose a company (Dupont) that has been possibly the best-run, most forward-looking corporation that has existed.

TheBiv 3 days ago 3 replies      
>DuPont said it expects 10 percent of its 63,000-strong global workforce will be impacted, and the company is budgeting for $650 million in employee separation costs. Analysts also expect layoffs at Dow, which employs 53,000 worldwide.

>Dow chief executive Andrew Liveris also called it a seminal event for our employees.

Seminal indeed.

necessity 3 days ago 0 replies      
Fun fact about Dow:

In 1905, German bromide producers began dumping bromides at low cost in the U.S. in an effort to prevent Dow from expanding its sales of bromides in Europe. Instead of competing head on with the German producers, Dow bought the cheap German-made bromides and shipped them back to Europe, undercutting his German competitors.*

A fine example of "predatory pricing" gone wrong (which happens more often than not).

* https://en.wikipedia.org/wiki/Dow_Chemical_Company#cite_note...

metalliqaz 3 days ago 3 replies      
The resulting 3 companies would be in separate industries, and therefore have no direct competition between them. Therefore, I would say that regulatory approval isn't certain. Probably it will be approved, but with a long list of caveats and provisions. Among them will no doubt be some environmental concessions that DowDupont would never bother following.
RyanZAG 3 days ago 8 replies      
This kind of merger seems exactly what the regulatory system was designed to prevent, right? Have any recent mergers actually been blocked? Is the regulator completely toothless now?
jawns 3 days ago 1 reply      
Delawarean here!

DuPont is one of my state's largest private-sector employers, and beyond that, the Dupont family's influence on Delaware is hard to overstate.

Here are a few stories from my former employer, The (Wilmington, Del.) News Journal, about the merger, touching on its potential effects on the state:



By the way, did you know that DuPont's longtime CEO Ellen Kullman resigned just a couple of months ago?


Her exit came after winning a proxy battle led by activist investor Trian Partners:


Trian, led by Nelson Peltz, wanted to break DuPont into pieces, and Peltz said that even though he lost the proxy battle, he wasn't finished with DuPont:


It's hard to imagine that the timing of the merger so soon after her departure is coincidental.

Sounds like Kullman won the battle but not the war.

tptacek 3 days ago 5 replies      
... but will then break up into three independent publicly traded companies, one focusing on agriculture, one on materials, and one on specialty products.
godzillabrennus 3 days ago 1 reply      
Bubble deals are delightful. Time Warner and AOL merging marked the height of bad mergers in the last tech bubble. Since every trading company is now basically a tech company in disguise I wonder if this time around these mega deals will be seen as disastrous in the eyes of history.
calvinbhai 3 days ago 2 replies      
If "DuPont culture" mentioned in the comments is true, I hope they help Bhopal Disaster victims get the justice they deserve.

Bhopal Disaster happened because of Union Carbide(wholly owned subsidiary of Dow chemicals)

xixi77 3 days ago 0 replies      
Looking at the stock chart in the article, "cool reaction from investors" they mention is a downward adjustment reversing most of Wednesday gains, but still putting each company above Tuesday close.

Was there another announcement Tuesday night, that set hopes high, with this one being a disappointment? What was the new information -- something about the structure of the new company perhaps?

Edit: Wednesday's article: http://www.nytimes.com/2015/12/09/business/dealbook/dow-chem... -- mentions the future split into 3 companies; does not mention the layoff details or dual headquarters though :)

theophrastus 3 days ago 0 replies      
Farmers, or their lobbyists, would be well-advised to get very busy pressuring congress to block this. If Dow and DuPont successfully merge the control of major seed will be down to two mega-corporations; the other being Monsanto. Future prices and controls will become even less favorable for farmers. A quote from a CEO I knew: "Mergers benefit at most one CEO and a minority of stock-holders -- everyone else involved suffers, employees especially"
anon1mous 3 days ago 1 reply      
Dow and DuPont have been loosing a lot of customers to BASF in the last 5 years or so. Especially in the area of nano materials.
pvnick 3 days ago 1 reply      
>Their innovations include a sweeping variety of now-ubiquitous chemicals, including ... chlorine

Really? That's impressive, to have invented an element. Surely this must be a method of storing and transporting chlorine and not the halogen itself.

ClayFerguson 3 days ago 0 replies      
So we can expect the price of glassware and plastics to what, at least double right? Capitalism. Gotta love it. If you're rich.
bbulkow 3 days ago 3 replies      
We need the competition, I don't understand why as a society we would want a megacompany controlling the international chemical market.
kiriberty 3 days ago 1 reply      
Wow, DOJ will be busy.
NittLion78 3 days ago 1 reply      
Anyone else notice the headline was "jaw-dropping" but the URL was "job-dropping"?
littletimmy 3 days ago 1 reply      
Can't we stop this consolidation and keep the market competitive just by having a progressive corporate tax? Are there any arguments against such a tax?
       cached 15 December 2015 03:11:03 GMT