hacker news with inline top comments    .. more ..    10 Feb 2017 News
home   ask   best   2 years ago   
1
pfSense: Open source network firewall distribution pfsense.org
139 points by lobo_tuerto  2 hours ago   71 comments top 18
1
notaplumber 1 hour ago 2 replies      
Netgate/pfSense has been notoriously anti-community, with their own co-founder attacking other projects, including a recent fork called OPNSense.

They've also been very hostile towards the OpenBSD developers, and project. Despite the fact they've effectively built a business on OpenBSD innovations, like pf and CARP, even incorporating the name 'pf' into their trademark having not contributed a bit of code.. nor a dime (but perhaps some hw).

But feel free to keep using it.. no need to take my word for it.

https://www.openbsd.org/donations.html

http://www.openbsdfoundation.org/contributors.html

2
lima 2 hours ago 4 replies      
Last time I checked, pfSense was good at firewalling but bad at everything else security-wise.

- Web panel allows root code execution on the device (every XSS is full RCE!)

- Everything runs as root

- No ASLR or other hardening flags because FreeBSD

- Lots of XSS and CSRF opportunities (probably got better with the new UI)

- Did not replace SSL certificate after Heartbleed (on packages.pfsense.org!)

- No package signing, either (not sure if this is still true with pkgng)

- Did not even have SSL on packages.pfsense.org until one or two years ago

I'm also missing the fq_codel queueing discipline on my home network (prevents bufferbloat).

I still use it since it's awesome, but I hope their security posture has improved since.

Most of the commercial vendors are even worse, but still.

3
jstewartmobile 1 hour ago 2 replies      
I liked pfSense until it got too beefy for my ALIX board. That forced me to move to OpenBSD, and boy am I glad I did.

Once you grok the syntax, it is so much easier to directly update settings in pf.conf than the pfSense web GUI--especially traffic shaping rules.

That, and OpenBSD has great documentation, decent IPv6 support, and almost everything you need already baked-in.

Here's a basic set of config files for a home setup: https://github.com/BourgeoisBear/OpenBSDFirewall

If you need IPv6 or anything fancy, PM me.

4
notaplumber 2 hours ago 0 replies      
If you want a pf firewall, you should probably get it from OpenBSD.. they created pf.

https://www.openbsd.org/faq/pf/index.html

5
F00Fbug 2 hours ago 0 replies      
I can't say enough good things about pfSense. I'm running three of 'em and would gladly trash my Checkpoint firewalls to use pfSense, if our parent company didn't mandate them.

I've run it on everything from tiny, Atom-based machines to kitted-out HP DL385s, and even on ESXi virtual machines. It works great in all scenarios.

6
dstroot 2 hours ago 0 replies      
Happy user here. I have used this exclusively for the past eight years. It's gotten better as it's got more commercial better UI, more polished plugins, better features, etc.
7
sandGorgon 2 hours ago 2 replies      
I always had this question - can something like pfsense be built on linux with comparable performance or is there something inherent to the whole stack that makes this effective?

I have always wondered if building pfsense on a modern linux kernel + selinux + BPF with something like nginx/lua scripting (for addon packages) would make more sense.

http://www.brendangregg.com/blog/2016-10-27/dtrace-for-linux...

8
kiallmacinnes 2 hours ago 4 replies      
It's been a few years since I used pfSense (maybe 7 or 8 years?) - when did they become so "commerical"?

(I'm not saying thats a bad thing, I really have no opinion on that, I'm just curious!)

9
sschueller 2 hours ago 1 reply      
PfSense is awesome! It runs very well for very little money on your own hardware [1].

You can do so many things easily. For example force all local chromecasts via a VPN for netflix.

[1] https://pcengines.ch/apu2.htm

10
nailer 2 hours ago 1 reply      
Is this 'pf' like the BSD firewall? Anyone more familiar with the space want to provide some detail?

Edit: yeah it is:

> The pfSense project is a free network firewall distribution, based on the FreeBSD operating system with a custom kernel and including third party free software packages for additional functionality.

From https://pfsense.org/getting-started/ see also https://doc.pfsense.org/index.php/Installing_pfSense

11
kek918 2 hours ago 0 replies      
We use pfSense at a client of mine and it's great. I didn't have any prior experience with it but I only spent a couple of nights setting everything up. We use it to manage firewall rules and as DHCP server for 2 VLANs.

We've used it for about a year now. Had several power outages. The pfSense box is still running like a champ, no problems whatsoever.

Turns out my client actually wanted an email spamfilter though, but our mail is hosted in another country so I had a little trouble explaining why our internal pfSense couldn't help us there. Oh well.

12
mrmondo 1 hour ago 0 replies      
I've been using PFSense both at various workplaces and at home for many years now, it's seriously fantastic. Feature rich, rock solid and good 3rd party packages.
13
dcow 33 minutes ago 1 reply      
I've never hired a network security engineer or security researcher that said they use pfSense. Sometimes I'll keep entertaining them in case they repent, but it usually just means they don't actually know what they're doing. And yes, the pfSence team or whatever you call it are assholes. Don't give them your money. Use openBSD.
14
wslh 2 hours ago 1 reply      
I don't have such experience. I tried to configure IPSec and IKEv2 in pfSense following the instructions and it never worked with a Windows client. There are a lot of questions related to this on the forums without a concrete/canonical answer.

I like pfSense but I chose it to easily configure usually complex networking stuff, not a DHCP or DNS. We are trying now to configure IKEv2 on an Ubiquiti device using UniFi with a load balancer and doesn't seem to be trivial.

15
vxxzy 2 hours ago 0 replies      
I use pfSense as a vitual machine to firewall/NAT off my vitual environment. It works great. I am running it on KVM/QEMU and handing off the PCI device to the pfSense VM. Great for managing my environment through VPN.

I've also used this in a retail chain in DC/MD/VA. Each location used pfSense for site-site VPN (OpenVPN). We also used the asterisk package to handle VoIP.

Overall, pfSense is a robust UTM.

16
rdslw 1 hour ago 0 replies      
Guys, they started in 2004. Its 2017 today. hackersNEWS ?
17
kylegordon 1 hour ago 0 replies      
Been using pfsense for a while now, and with CARP and system redundancy, firewalling, and twin ipsec tunnels to AWS, it's been great and hugely reliable.
18
amadeuspzs 2 hours ago 1 reply      
Having used pfsense for a good few years, we recently switched to Meraki.

Global infrastructure and Meraki APs just made it easier to go with their ecosystem, despite the licensing costs.

2
Zelda: Link's Awakening: Kennel Glitch fobby.net
24 points by Ivoah  35 minutes ago   3 comments top 2
1
dec0dedab0de 1 minute ago 0 replies      
I played links awakening last year, and I hit this glitch by accident. I just turned my gameboy off though.
2
gdk 5 minutes ago 1 reply      
> This glitch can permanently ruin a game file and, in the extreme case, possibly even erase or damage your cartridge.

How can this glitch erase the ROM in a cartridge?

3
How the Flash Crash Traders $50M Fortune Vanished bloomberg.com
71 points by walterbell  2 hours ago   34 comments top 6
1
chollida1 1 hour ago 3 replies      
This guy isn't the first person to make money in one arena and extrapolate that he could do it in others only to get his ass handed to him.

I'm Canadian so the wealthy people AI know are either in finance or hockey players, and those that have lost money in areas outside their wheel house have all followed the same path.

1) Give money to a person they don't know very well to invest in a business idea they aren't an expert in.

There is no step two.

As a side note, i found this interesting....

> It wasnt until Sarao left Futex in 2008 and struck out on his own that he started to make serious money. Public filings show his assets popped to 14.9 million pounds from 461,000 pounds in the 12 months ending in June 2009, long before he enlisted a programmer to build a system that authorities say was designed to cheat the market.

Not sure what helped more in his rise:

The 6 years he spent learning at another trading firm or the fact that on his own he probably had alot less risk oversight that allowed him to lever up more than he would have in side of an investment house.

> That near-obsessive drive to hold on to as much of his wealth as possible can also be seen in the way he conducted his business affairs. Looking to minimize his tax bill, he was introduced by his accountant to John Dupont, a director at the London arm of an Isle of Man-based financial advisory firm called Montpelier Tax Consultants

The best investment adivce I ever got was from my father. and I quote.... "Don't fuck around with your taxes any more than an H&R block adviser would let you. Getting a million dollar tax bills 7 years after you earned the money isn't worth it."

2
KKKKkkkk1 54 minutes ago 6 replies      
I don't get it. This guy's crime is placing offers with the intention of canceling them before they are executed. Why is this a crime, and what does it have to do with the flash crash?
3
elastic_church 13 minutes ago 0 replies      
Just stick with bonds for wealth preservation. Keep your target yield around 5.5% and you'll be okay, even with a bit of leverage.
4
greenwalls 1 hour ago 2 replies      
Implementing something like Andrew Lo's encryption algorithm to avoid future flash crashes is a great idea https://www.technologyreview.com/s/512291/how-to-avoid-anoth....

It's also interesting the article refers to Sarao as frugal. If he was really frugal it seems like he would have said that X amount of money is enough and stopped risking it all. Scary stuff!

5
csomar 1 hour ago 1 reply      
Is it (-)safe to say that Sarao was a randomly lucky trader rather than a genius?

-By safe, I mean probabilistically possible in a sample of x traders.

6
soniman 47 minutes ago 0 replies      
It sounds more like this guy was a front for somebody else, and all of these shell companies are hiding whoever it is that's behind this.
4
Change at Buffer: The Next Phase, and Why Our Co-Founder and CTO Are Moving On buffer.com
26 points by colinscape  1 hour ago   4 comments top 2
1
CPLX 15 minutes ago 2 replies      
Summary: the cofounders disagreed so two of them left.

Maybe I'm an old guy but I don't really understand why this blog post exists, it's like hundreds of words of emotive rambling and vague talk of journeys and values and euphemisms for simple concepts.

Would it like have ruined anything to write something like "A couple key early people are moving on but we're doing pretty good, we make social media software and have revenue ok hey thanks for listening I'm going to get back to that now have a great Friday" and then hit save?

2
minimaxir 18 minutes ago 0 replies      
5
Google coding champion whose Cameroon hometown is cut off from the internet bbc.co.uk
61 points by SimplyUseless  1 hour ago   7 comments top
1
xrisk 22 minutes ago 4 replies      
What a clickbaity title... A Google Code-In winner is hardly a 'coding champion'.
6
Cryptographically Secure PHP Development paragonie.com
39 points by CiPHPerCoder  2 hours ago   1 comment top
1
sbarre 30 minutes ago 0 replies      
Nice to see this work being shared with others.
7
Watch pom-pom crabs fight over tiny anemones, which they hold like boxing gloves washingtonpost.com
52 points by mhb  2 hours ago   30 comments top 9
1
amelius 1 hour ago 2 replies      
> The pom-pom crab, the scientists speculated, is perhaps the only animal on the planet that controls another species growth, feeding and asexual reproduction.

Speaking about controlling another species, this story [1] where a wasp performs brain-surgery on a cockroach also seems pretty interesting.

[1] https://www.wired.com/2014/02/absurd-creature-of-the-week-je...

2
WayneBro 1 hour ago 4 replies      
Another interesting crustacean is the Mantis Shrimp - it's got a fast and powerful punch that reaches 73 feet per second (50 miles per hour) within 3 milliseconds - https://www.youtube.com/watch?v=DtNAqK_V-lg
3
pasta 27 minutes ago 0 replies      
Some years ago I had a reef tank. Most people start a reef tank by adding live rock. Those are rocks straight out of the sea.

It's amazing what live rock realy means. Every cm is covered with some form of live. Worms, crabs, starfish, anemones, you name it.

We know so little about this world. It's a shame we are busy destoying large areas of the ocean.

4
mfrykman 1 hour ago 1 reply      
"The pom-pom crab, the scientists speculated, is perhaps the only animal on the planet that controls another species growth, feeding and asexual reproduction."

Except for scientists, of course.

5
sosuke 1 hour ago 0 replies      
Better quality and longer videohttps://www.youtube.com/watch?v=vgWy0uOg54A

Boxer crabs induce asexual reproduction of their associated sea anemones by splitting and intraspecific thefthttps://peerj.com/articles/2954/

6
busted 26 minutes ago 0 replies      
Hacker News.
7
Dowwie 58 minutes ago 0 replies      
This was one hell of a David vs Goliath match
8
JoeAltmaier 1 hour ago 3 replies      
But... why?!
9
Unbeliever69 15 minutes ago 0 replies      
Am I the only one that read this as porn-porn crabs?
8
Bored with ho-hum cloud backups? Use Usenet (yes, Usenet) instead arstechnica.com
26 points by Tomte  1 hour ago   34 comments top 8
1
jasode 1 hour ago 2 replies      
>With access to a Usenet news server, you can simply upload your backup there, and it will be stored redundantly in news servers all over the world. Best of all, this approach typically costs considerably less than a cloud backup service.

I saw no mention about "retention policies" in the article. For newsgroups with binaries, the other Usenet peer servers can choose to download only text and ignore binaries. Or they only hold binaries for 30 days or whatever discretionary time period they choose based on available disk space.

With AWS Glacier, Backblaze, etc the data retention would be explicitly specified.

[1] example of limited retention period: http://www.giganews.com/blog/labels/retention

2
ashark 36 minutes ago 0 replies      
Related-ish-ly, I've had a notion bouncing around in my head that it should be possible to build a fairly complete social networking interface on top of email and ical (or similar), and some kind of contact-management standard, such that the whole thing (or at least enough of it) could be rebuilt from e.g. your (possibly dedicated to this purpose?) email account when necessary, possibly in stagessay, a new installation of the application builds what it can from the last 30 days in a few seconds, then displays a populated interface while it continues digging to some arbitrary point in the past (until it exhausts caching resources it has been allotted on a given platform, basically).

An initial search has revealed a couple abandoned partially-complete efforts, but that's it.

3
83457 44 minutes ago 3 replies      
Idea: An open source peer-to-peer backup service where data is encrypted and backed up in pieces across countless systems around the globe. No one person had your data but in some way it would be guaranteed your data would always be available. Everyone who uses the backup service would be required to also accept data pieces for backup.
4
thieving_magpie 1 hour ago 0 replies      
>Bored with ho-hum cloud backups?

Well I can't say that's something I've ever thought.

5
bb101 31 minutes ago 1 reply      
This has to be a joke, surely? A good example of the Tragedy of the Commons...

As ProfessorGuy succinctly commented:> Why not an article on how you can get a bed for yourself in the local hospital so you won't have to pay rent? Hey, if they're going to build a public institution like a bunch of suckers, they deserve to be taken advantage of!

6
dillonb 1 hour ago 2 replies      
While interesting, this seems like a bad idea. You're uploading your backups, no matter how encrypted, to a place where they will be publicly available to download.
7
therealmarv 43 minutes ago 0 replies      
Has anyone experience with that in practice? What are the data limits on this kind of backups?
8
apricot 9 minutes ago 0 replies      
Also, if you ever get bored putting junk mail in your recycling bin, why not mail it to the author of this article?
9
JavaScript Start-up Performance medium.com
190 points by chriswwweb  6 hours ago   80 comments top 13
1
hacker_9 4 hours ago 1 reply      
"Precompiling JavaScript?

Every few years, its proposed engines offer a way to precompile scripts so we dont waste time parsing or compiling code pops up. The idea is if instead, a build-time or server-side tool can just generate bytecode, wed see a large win on start-up time. My opinion is shipping bytecode can increase your load-time (its larger) and you would likely need to sign the code and process it for security. V8s position is for now we think exploring avoiding reparsing internally will help see a decent enough boost that precompilation may not offer too much more, but are always open to discussing ideas that can lead to faster startup times."

Surprised there was no mention of webassembly, which does exactly this.

2
spankalee 1 hour ago 0 replies      
I'm always shocked at how reluctant sites are to actually ship less code, and I think some of this comes down to a needed shift thinking in what an application is, what modules are and how to use imports.

One thing I've heard recently is "My app is big, so it has a lot of code, that's not going to change so make the parser faster or let me precompile".

The problem with this is thinking that an app is a monolith. An app is really a collection of features, of different sizes, with difference dependencies, activated at different times. Usually features are activated via URLs or user input. Don't load them until needed, and now you don't worry about the size of your app, but the size of the features.

This thinking might stem directly from misuse of imports. It seems like many devs think an import means something along the lines of "I'll need to use this code at some point and need a reference to it". But what an import really means is "I need this other module for the importing module to even _initialize_". You shouldn't statically import a module unless you need it _now_. Otherwise, dynamically import a module that defines a feature, when that feature is needed. Each feature/screen should only statically import what it needs to initialize the critical parts of the feature, and everything else should be dynamic.

In ES2015 this is quite easy. Reduce the number of these:

 import * as foo from '../foo.js';
and use these as much as possible:

 const foo = await import('../foo.js');
Then use a bundler that's dynamic import aware and doesn't bundle them unnecessarily. Boom, less JS on startup.

3
SCdF 4 hours ago 1 reply      
> Ship less JavaScript.

Please do. The fastest code to parse is the code that doesn't exist.

4
geocar 4 hours ago 6 replies      
I am surprised at just how much faster the iPhone is than the next nearest Google device (a laptop). I haven't used an Android device for a long time, but I hear the "Apple is overpriced" so often that I assume someone has been checking this out.
5
yunolisten 4 hours ago 3 replies      
> Ship less JavaScript

posted on a site shipping 456.1KB of packed JavaScript

6
czbond 2 hours ago 0 replies      
My take on the JS performance 1) wait for CPUs of mobile to catch up with current JS flow [not acceptable, but easiest path ] 2) Web Assembly to create native experience or 3) something like Elm as a JS replacement (front end compilation). We know interpreted langs are slow - however JS is slow now where people see it most [Front end] they dont see the slow of back end. For example, if a user had to wait for npm install when they used the server the first time. The JS community has a great habit of adding all the things for significant bloat. [module for leftpad, and lots of go-arounds due to the language being bolted on for its current use rather than designed from the ground up]. Its 6am in Caliso take it with a grain of salt.
7
throwanem 3 hours ago 1 reply      
I love that this article, which is quite good, appears on a site whose every page crashes and boot-loops multiple times in iOS Safari and has to be repeatedly reloaded by the browser, presumably running less of its code each time, in order to correctly render some text with images in it.

Sort of gives added point to the thesis, by providing a marvelous example of something you should never, ever do.

8
txprog 2 hours ago 1 reply      
I'm confused.

One of the idea behind CDN for javascript/css is to leverage caching by reusing the same resources across websites.But then optimization tools said we should bundle everything into one javascript, which delay the loading time, but defeat the initial purpose.

I wonder if the caching could be more intelligent, by recognizing libraires bundled into the "big" javascript that website delivers, and parse only the new content.

9
isoos 3 hours ago 0 replies      
Slightly related: pre-parsing code and loading already initialized application state is available for the Dart VM for a long time now, and the technology yields faster startup times:

https://www.dartlang.org/articles/dart-vm/snapshots

10
mschuster91 4 hours ago 1 reply      
Hmm. Usually browsers cache the raw CSS and JS assets - could that be improved so that browsers cache the compiled CSS/JS? That wouldn't help for the first load, of course - but quite a lot for sites like newspapers which are not SPAs but bundle a metric ton of JS cr.p for each page load.

edit: Chrome actually does that, as mentioned in the article - but what about Chrome Mobile and Firefox/Safari/IE?

11
jijji 4 hours ago 1 reply      
Why not have the js engine, i.e v8, parse and compile one time, and then refer back to this object code each time to eliminate the startup delay... Either an http header cache-control flag or something similar.
12
jerianasmith 3 hours ago 1 reply      
Right,Generally programs store the crude CSS and JS resources - could that be enhanced with the goal that programs reserve the accumulated CSS/JS? That wouldn't help for the principal stack.
13
efxzsh 2 hours ago 0 replies      
10
A GNU-Readline-like library for .NET github.com
35 points by tonerdo  1 hour ago   6 comments top 2
1
tonerdo 29 minutes ago 1 reply      
Library author here. I would love if people took it for a spin and gave me feedback. Also contributions are welcome, I'm currently trying to figure out ways of testing deeper into the library.
2
youdontknowtho 1 hour ago 1 reply      
That's really cool. I was just looking at the ReadLine that the PowerShell team released the other day.
11
Join GitLab's March Issue Bash gitlab.com
19 points by dwaxe  39 minutes ago   7 comments top
1
jasim 27 minutes ago 6 replies      
I'd love to know what the magic recipe that GitLab uses to be so prominently in-your-face at all times on HN.
12
George Washington: A Descendant of Odin? publicdomainreview.org
18 points by samclemens  1 hour ago   1 comment top
1
sandworm101 1 minute ago 0 replies      
Great. I cannot wait for nick cage in the thor-national treasure crossover.

I lived in new england for a while and this mythmaking culture around the founding fathers gets old very quickly. The US just isnt old enough to have mythical kings and gods. That went away when the US broke from the uk.

13
How New York City Gets Its Electricity nytimes.com
72 points by Osiris30  3 hours ago   15 comments top 7
1
jawns 2 hours ago 1 reply      
One of the things you'll probably not notice about this article, although you can be certain a lot of effort went into it, is the simplicity of language and sentence structure.

The article describes a hugely complex network of technologies, and also touches on how public policy shapes their use, and yet I bet that as you're reading, you don't struggle at all with understanding what it's talking about.

The writer, Emily S. Rueb, uses short, straightforward sentences, paragraphs of only a sentence or two, utility words instead of flowery language -- and yet the writing doesn't seem at all unrefined. It just seems natural and to the point. That's a real service to the reader, and a real show of talent by the writer.

I guess, like the incredible infrastructure that goes into supplying power to NYC, you know it's working well when you hardly notice it.

2
msisk6 8 minutes ago 0 replies      
This is one of the better articles I've read about the grid.

I don't work in this region, but I work for another ISO that controls one of the other grids doing a cross between DevOps and SRE on the systems that directly control the grid.

I've worked for startups (including my own) and public cloud providers, but this is the most stressful job I've had. If I fail people will quickly start to die as traffic lights and life-support systems fail.

The article touches on the complexity a bit, but it's huge job controlling a deregulated open-market energy grid. You have to scale for peak demand while providing financial incentives to support a mix of generation with widely varying prices. You have to deal with distribution congestion and natural disasters. Even generation disruptions -- a 2-GW nuke plant can shutdown at any moment. You need to deal with that deficit and quickly get something else online to meet demand.

And it all has to work 24/7 without a second of interruption.

It's an eye-opening experience. I'll never complain about paying my electric bill again. ;)

3
leroy_masochist 37 minutes ago 0 replies      
My first project as an investment banker was refinancing the turbine barges in Brooklyn briefly mentioned at the beginning the article.

The article should have gone into capacity auctions. It's an interesting process; generators participate in Dutch auctions to win contracts to be ready to provide a certain number of MWs onto the grid. The entire winning side of the market gets paid at the market-clearing price when the ISO gets enough MWs allocated. The capacity is auctioned off over three time horizons: a "strip" auction for the six-month winter period and the six-month summer period that happens about a quarter before the period starts; a monthly auction that clears a few months out; and a monthly auction that clears on the eve of the start of the month.

I was initially surprised to learn that clearing bidders are not even paid to generate; they're paid to be ready to generate, which means they have to stay inspection-compliant and periodically fire up the plant to ensure everything's in good working order. If they generate, they get paid for the MWs that go onto the grid.

For an asset like the Brooklyn barges, they might get spun up 20 days a year on a hot year, and actually put meaningful MW's on the grid for 10 of those days. All in July/August when it's hot out and people are blasting AC. The capacity market is highly seasonal.

The NYISO website has lots of data if you want to learn more: http://www.nyiso.com/public/markets_operations/market_data/i...

4
unwind 2 hours ago 2 replies      
Since it says that the biggest threat against the electrical supply system is trees, shouldn't it be possible to design a drone-based automatic transmission line inspection system?

I mean "just" have a drone with appropriate sensors fly the length of the lines, looking for tree branches that are threatening. Perhaps using machine vision, or something simpler like lidar/radar/ultrasonics/whatever. Signal human inspectors when something looks fishy enough, with video attached for quick review/triage.

Optimally designed to allow recharging the drone's batteries somehow from the top of the pylons (aren't there electrical fields that might be used somehow[1]?), to allow extended automated runs.

Obviously covered in rubber or something to make the chance of damaging the infrastructure in case of a crash minimal.

Crazy?

[1] No idea if this is possible but it would be super neat since it might allow it to scale to run on hundreds of miles of power lines without many humans and with good coverage.

5
bradfa 1 hour ago 3 replies      
The article says that 4 operating nuclear plants supply 1/3 of the electricity for NY state. That's impressive!

Sadly, 4 of the NY nuclear plants were commissioned in the 1970s. A 5th plant, in Shoreham, is the newest but is also no longer operating. Nine Mile Point has one newer unit from the 1980s, but all told, most of the nuclear power generation capacity in NY is 40+ years old.

6
SonicSoul 1 hour ago 0 replies      
if you like this, you may find this book interesting:

The Works: Anatomy of a City.

my GF got it for her post grad architecture degree and i found it really well illustrated and interesting! For example it outlines the steam grid in NYC and current vs historic uses

https://www.amazon.com/Works-Anatomy-City-Kate-Ascher/dp/014...

7
cwal37 2 hours ago 0 replies      
If you want an extremely thorough look into the operation of New York State's (including NYC of course) electricity market I would recommend the market monitoring unit's State of the Market Report[1]. The deregulated markets have a lot of quirks, but I think their operation and set of outcomes is pretty interesting.

[1] http://www.nyiso.com/public/webdocs/markets_operations/docum...

14
Considerations on Cost Disease slatestarcodex.com
261 points by apsec112  10 hours ago   107 comments top 23
1
abakker 27 minutes ago 0 replies      
This article does a really nice job of laying out this issue, but avoids an explanation that is difficult to show in data, I think. Many of these issues he addresses - healthcare, housing, infrastructure, and education, are enabled by federally assumed debt.

The core problem, I believe, is that federal debt is like a blank check that nobody is personally accountable to repay. The result is that people in charge of spending simply do not bargain well. There is no immediate incentive to drive cost down by forcing suppliers to deliver at a lower cost, or find the the real market equilibrium where nobody will offer a service.

Loose spending from the government is a different problem than government spending in general. If the government were interested in getting the best deal, it has more bargaining power than most entities in the world. However, they frequently pay MORE for the same services. That alone should be a trigger. I think most people intuitively know this and rail against it when confronted by examples like student loans being spent on predatory for-profit colleges, but, in aggregate, I blame most of the problem here.

The government has a nearly infinite potential for more debt, and as a result they have no incentive to get a good price. Things have spiraled out of control.

2
justcommenting 7 hours ago 3 replies      
The question I'd ask in response to this post is "So where did the money actually go?"

I suspect inequality and wealth transfer explains much more of the observed trends than the author acknowledges at the end. In each of the verticals discussed, there have been strong (albeit sometimes less obvious) trends for consolidation among institutions and organizations. This includes companies, government contractors, and even vendors in ecosystems we tend to think of as decentralized like local schools, where significant consolidation might be occurring over time at the level of food suppliers like Aramark, utility companies, or diesel fuel suppliers for school buses.

These organizations' compensation and capital structures, in turn, likely grew increasingly unequal over time. Stockholders, stakeholders like executives, and intermediaries like insurance companies in those organizations likely extracted more and more capital relative to traditional stakeholders like the college students, physicians, and teachers addressed in the post.

Wealth transfer from traditional stakeholders (college students, physicians, teachers) to organizational stakeholders (execs, stockholders, suppliers in consolidating markets) seems like both a cause and a consequence of the 'cost disease' discussed in the post.

3
tptacek 3 hours ago 4 replies      
Nominal teacher salaries may be reverting back towards the mean salary, but teacher compensation is not. Public school teachers in the US have a unique compensation structure:

1. They're salaried at a middle-class level (in a major metro like Chicago, the median teacher salary is $70k).

2. The get an extraordinary amount of time off. There are large public school districts where teachers have a contractual maximum of 190 work days.

3. Most importantly, many (most?) receive a defined-benefit pension plan. During the period of time this piece documents the teacher salary decline, virtually all competing jobs of every status lost defined-benefit pensions and switched to defined-contribution plans.

Further, the trend over the last 10 years has been towards tonier school districts paying spectacularly high teacher salaries. The perception in those districts is that a dollar spent on the schools (in any way) pays back more than a dollar in increased property values. So my kids at Oak Park River Forest high school have teachers --- not assistant principals, not coaches, not LD/BD specialists, just teachers --- making over $100,000. And bear in mind, if you normalize this back to a 50 week work year, that's closer to $140,000. With a defined-benefit pension.

And we're pikers compared to Buffalo Grove, which has several teachers making over $150,000.

4
jchrisa 41 minutes ago 0 replies      
So everything has been running in successively more nested layers of virtual machine, ever since the Magna Carta. What we need is for someone to start a "rust for civilization" project.

Further thought: could the thing that's happening be that money is getting less real? So it takes more of it to "push the string."

To me it seems that heavy redistribution, with high taxes and a basic income, could help make money more real.

One has to trust the opportunity landscape has shifted due to the Internet, and now capital accumulation is not a requirement for innovation. Otherwise it looks really scary to tamp down on the idea of capital accumulation. Especially to the bankers, who might be out of a job.

5
BrailleHunting 9 hours ago 4 replies      
In the US, education, military and other budgets are seen as sacred cows as proxy signals for the emotional investment in their missions, such that insisting on service value for budget outlay is viewed as "social treason."

The US needs to do more of what Robert Reich ("Inequality for All"), Michael Moore ("Where to Invade Next?"), Bernie Sanders and Plato suggest: copy what works from elsewhere in the world (which others often copied from elsewhere including the US in the past), strengthen unions and get people more engaged in all levels of politics. (The US isn't a special snowflake, policy prescriptions can be tried and customized for commonwealth utility. But first, get money out of politics.)

"The price good men pay for indifference to public affairs is to be ruled by evil men." -Plato

6
struppi 9 hours ago 3 replies      
Ok, maybe I'm getting this wrong, but to me, it's not entirely surprising that those costs have been rising, even when adjusted for inflation.

AFAIR, we spend a lot of money on food, transport, holidays, electronic devices, ... All of which have become cheaper over the last few decades. The consumer price index reflects this. So, if a lot of items have become cheaper, even adjusted for inflation, others must become more expensive, adjusted for inflation.

The article even mentions this effect:

 First, can we dismiss all of this as an illusion? Maybe adjusting for inflation is harder than I think. Inflation is an average, so some things have to have higher-than-average inflation; maybe its education, health care, etc. Or maybe my sources have the wrong statistics.
But: Maybe this does not explain the whole effect, and maybe I am missing something important. I am not an economist, after all.

7
misja111 6 hours ago 2 replies      
I'm probably biased because I'm from Europe but to me it seems that the decreasing cost effectiveness for health care and schooling in the USA is due to the capitalist philosophy that is so strong in USA politics.

For instance in health care, in the USA pharmaceutic companies have much more opportunity to sell or advertise their products directly to consumers. This is because as a consumer in the USA you have more influence in the choice of medicine or medical care you receive; as long as you are willing to pay for it, you can get almost anything. In many European countries it is not like that, it is many times the doctor who decides what is the most appropriate cure for you and even he can not always decide which medicine to use, many times this is regulated by the state or by health insurance companies.

In schooling there is a big competition in the USA between universities to have the best ratings; this competition drives up salaries of top professors to a level that does not reflect the extra benefits that they bring to their students. All of this can happen because universities are pretty much free to ask whatever fee they desire to students. Again in many European countries it is not like that, universities have only limited freedom to determine their tuition fees by themselves.

8
lend000 9 hours ago 2 replies      
Diseconomies of scale is a very real phenomenon, and it makes a pressing argument for decentralizing by giving power back to the states. Was that a foreseen side effect the founders wanted to avoid with the 10th amendment? Probably not, but freedom is inherently a decentralized proposition, and tends to be more efficient in many cases as a result.

https://en.wikipedia.org/wiki/Diseconomies_of_scale

> Cowen assumes his readers already understand that cost disease exists. I dont know if this is true.

Unfortunately, "cost disease" has become politicized, so just like some won't accept global warming, others deny basic realities about economics.

9
rjeli 7 hours ago 0 replies      
I vaguely remember someone saying the #1 doomsday scenario that keeps them up at night is complex systems collapse: that everything just starts to fail and no one knows why. Anyone have a link?
10
martinpw 7 hours ago 1 reply      
For the specific case of public university tuition costs, this analysis from the UC system suggests the increase it is primarily due to reduced government funding, requiring students to make up the difference:

http://www.ppic.org/main/publication_quick.asp?i=1119

It also suggests that increasing administrative costs are not a significant contributor, but that there is some effect from increased student services.

11
John_Innocent 5 hours ago 3 replies      
Isn't it obvious? All of the fields experiencing the cost disease are dominated by government. Either through massive funding, or vast numbers of regulations.

Even within sectors, we see the pattern. For example, the two fields of medicine which have seen the least cost growth are cosmetic [1] and laser eye surgery. Not only have costs risen the least, but the quality of some procedures in these fields has improved tremendously over the last two decades. Both are electives, so there are fewer mandates requiring that they be covered by insurance and fewer redistributive programs to subsidise them.

The source of the complex system dysfunction is the political and social system, which enables special interests to mislead the public in order to implement policies that benefit themselves but have a negative-sum impact on the economy as a whole. One of the 'big lies' that these special interests have succeeded in convincing the public of is that the free market is a misguided ideology that is promulgated by the rich in order to exploit the poor, when in reality it is a basic rule-set necessary for economic development in complex systems (due to properties like the rule-set enabling effective large-scale coordination of economic resources through price signalling, aligning private incentives with the public interest through laws granting and protecting the right to property produced through one's own efforts, or acquired through trade (and inversely, prohibiting acquisition of property through theft, armed robbery and other non-voluntary and predatory means), etc).

[1] http://healthblog.ncpa.org/wp-content/uploads/2013/06/HA1-06...

12
brilee 2 hours ago 0 replies      
My take on it is, even if health costs were 30% of my cushy software salary, I'd pay them, because well, everyone wants to live. And so, prices continue rising to the point where people are willing to pay for them. I'm willing to pay 30% of my salary for healthcare. I'm willing to pay 20% of my salary to get a good education for my kids. I'm willing to pay 20% of my salary to live in a city with good culture. I'm willing to... oh wait, what's that? I ran out of money because 40% of my salary went to federal and state taxes? Oops!
13
earljwagner 2 hours ago 0 replies      
We see similar phenomena in software: "software bloat", "feature creep", the "second system effect" (see Wikipedia for details). Linux provides the same basic functionality as Unix in 1980 but has orders of magnitude more lines of code. Why? It's more complicated because the world is more complicated, and we have higher (and more detailed) expectations for that functionality.

Basically, once you have a lean MVP that works, all social and economic pressures are to add new features with decreasing marginal gains, support standards of interest to fewer and fewer users, handle increasingly obscure edge cases, etc.

14
bradleyjg 3 hours ago 0 replies      
For schools and healthcare I'd like to see total FTE across every employment category in the sector per student and per capita respectively.

I'd guess that even if the teacher and doctor ratios are staying the same or going down the overall ratios are shooting up.

If so, the missing money isn't mostly being siphoned off to a few people but is going to enormous new workforce that didn't exist in the medical and education systems of the 1970s.

15
jasonmorton 1 hour ago 0 replies      
Imagine a distant future where advances in technology makes everyone fantastically wealthy in modern terms. Material goods are as cheap as digital storage feels now. What would spend money on in this utopia? My guess would be extending our time and quality of life (health care) and improving our minds (education).
16
aidenn0 7 hours ago 1 reply      
Elizabeth Warren gave a talk about this about a decade ago (before becoming a senator):

https://www.youtube.com/watch?v=akVL7QY0S8A

For a family of 4 compared 1975 to 2005, all inflation-adjusted:

Median housing went up by 76%, median number of rooms only went from 5.8 to 6.1 (5% increase), and the house is older.

74% increase in median health-care costs for a family of 4 with employer sponsored health-care.

52% increase in spending on cars, but cars are cheaper; many more families have 2+ cars.

Childcare went from the median family of 4 spending $0 to being a significant expense.

Tax liability went up by 21%

17
ekianjo 7 hours ago 0 replies      
good article but it should look at who profits from increased revenues and why instead of focusing only on costs increasing.
18
hamilyon2 5 hours ago 1 reply      
Underpaid/free labour (by women and black) that was very common back then. Much less now. This sosiety shift alone could double every price in economy, while keeping wages stagnant.

I.e. household income rose even if wages did not.

19
rdiddly 1 hour ago 0 replies      
I suspect the hidden factor is population growth. You can find numerous examples of things that become less efficient (more difficult/costly yet also crappier) when more people are out there demanding that service. Or not even using the same service, but simply interfering with each other in ways unrelated - getting in each other's way, affecting each other. Cost is not in a linear relationship with population size, or maybe it's population density. There's a kind of friction that increases. Diminishing returns of scale & complexity, you could call it.
20
venomsnake 8 hours ago 2 replies      
I think I can explain the subway costs - the construction itself is relatively cheap - tunneling and excavating is something mine companies do regularly on a bigger scale.

It is the NIMBY, regulations and not breaking anything. Unlike China you cannot rule by decree - there be subway here - dig.

21
_bpo 9 hours ago 4 replies      
Tangential, but why do people write things like "even after adjusting for inflation" when discussing long timespans (40 years in this case)?

Is there any reason to not adjust for inflation? The "even..." in the sentence suggests that the author is being liberal with potential critics when it would seem only rudimentary to acknowledge inflation when discussing a 40-year timespan. A typical (new) car cost ~$4.5k in 1977 in the US....

22
fnord123 6 hours ago 0 replies      
This is a 3% compounded increase in cost YoY. Not bad seeing at the population about doubled in the same time (2.5% YoY):

http://www.censusscope.org/us/chart_popl.html

23
John_Innocent 3 hours ago 2 replies      
Anyone who identifies the problem and the solution is labeled a free market ideologue and ignored.

This is the solution:

http://www.ncpa.org/pdfs/st349.pdf

It's not a mystery why all of the industries suffering the cost disease are dominated by government.

15
The Mystery of the World's Least American Cactus atlasobscura.com
13 points by Petiver  2 hours ago   2 comments top 2
1
ianai 35 minutes ago 0 replies      
"But today, as crossing borders becomes ever more fraught, its worth remembering that with the right support, even a humble epiphyte can make it pretty far."

This hurts right in the realities.

2
zdean 59 minutes ago 0 replies      
Couldn't a genetic comparison give us an estimate of how long ago the 2 lines split?
16
H-1B visas mainly go to Indian outsourcing firms economist.com
319 points by known  6 hours ago   259 comments top 32
1
loph 3 hours ago 16 replies      
This one sentence says it all:

"The Economist found that between 2012 and 2015 the three biggest Indian outsourcing firmsTCS, Wipro and Infosyssubmitted over 150,000 visa applications for positions that paid a median salary of $69,500. In contrast, Americas five biggest tech firmsApple, Amazon, Facebook, Google and Microsoftsubmitted just 31,000 applications, and proposed to pay their workers a median salary of $117,000."

None of those salaries listed are competitive with what a non-H1B (read citizen or permanent resident) would earn. Indeed.com quotes the average SD salary in Seattle (think Amazon and Microsoft) as 126,000 and San Francisco at 134,000. Companies sponsoring H1B need to be held to the letter of the law -- the salaries must be competitive. The demand for H1B visas would fall if the imported labor was paid fairly.

2
geebee 31 minutes ago 3 replies      
The economist proposes getting rid of the rule that requires H1B workers remain within the company that sponsors them

That sounds reasonable, but why then require that a company sponsor an immigrant in the first place? Why not let that immigrant choose where he or she will work?

In fact, why not let the immigrant choose what to study, where to work, where to live, all in response to market signals?

People have posted various lists for the average H1-B salary at what they consider top companies, like Google. $130k. Is that the salary in mountain view?

You know, I actually think that salary is somewhat low for what a talented and well educated person can earn in the Bay Area. Why force these people to get hired as developers? Why make them study what google says they should study, take interview exams on second year data structures and algorithms the way google says they should? Why on earth should google get to have this power, over anyone?

Let's just have immigration. All immigrants arrive in the US free, free to choose what they will study, where they will work, how they will go about it. They can sell real estate, install drywall, write python code, write novels, paint portraits, or whatever they wish to pursue. Nobody owes them success, but in the US, they should have the freedom to pursue happiness as they define it.

Not as Facebook defines it. If working as a dev in an open office so big it has a horizon line for a CEO who says things like "young people are just smarter" for $152m a year doesn't sound as appealing as the flexibility and stability of working as a dental hygienist for $110 a year (roughly the median salary in SF), then that's the market's answer.

I still maintain this - any immigrant system that allows corporations to decide who gets to come here is flawed. Allowing immigrants to quit once they're here would be an improvement, but it still allows corporations to decide who does and doesn't get to come here.

3
planetjones 5 hours ago 4 replies      
Is this disagreeing with Blake Irvine. See

https://www.google.co.uk/amp/amp.timeinc.net/fortune/2017/02...

This is the man who refers to H1B visas as genius visas. I have worked with many Indian outsourcing companies and while talented people do exist, calling their employees genius is wholly inaccurate (as would be calling most software devs in the Western world genius).

I did laugh when I read irving's original post on LinkedIn and some former employer of GoDaddy expressed just how Mr Irving was using his H1B allocation I.e. to get the same job done for less dollar...

4
throwaway100217 47 minutes ago 1 reply      
I am on an H1-B work authorization.

The company applied for my position with a salary of ~$100K/yr (on the LCA and the H1 application) but they actually paid me ~$240K/yr. This year and the next, it will be well north of $300-$330K/yr.

Why would they do this? Simple - to be able to pay me a prevailing wage and keep me in status in case the shit hits the fan.

If they applied to the government saying they'd pay me $240K/yr, and for some reason they had to give me a pay cut, I'd be out of status or we would have to make a less-confident amendment to my H1 auth. A pay cut amendment should be viewed with skepticism in my opinion and I would avoid it.

It's like underpromising and overdelivering.

My actual salary will never be reported in an H1 database. But it will be on my tax returns.

This isn't a common case but just something to keep in mind.

5
anjc 3 hours ago 5 replies      
>Although it is true that foreign workers at the Indian consultancies receive more visas than higher-skilled workers at better-known firms, a simple solution exists. Congress could raise the number of visas issued. Given that the unemployment rate for college graduates sits at 2.5%, it is fair to say that most native workers displaced by H-1Bs land on their feet.

Absolute scum. Native workers displaced by H1Bs is ok because the fired workers eventually find other work? Vile.

A significant proportion of IT and STEM graduate are unable to get work in their chosen industry, and proceed to waste years of education by going into other areas out of necessity.

I can't believe somebody could shit out the quoted text and have it published.

6
koolba 5 hours ago 5 replies      
What's the argument against using an auction for H1-B visas rather than a lottery? That'd maximize the tax collected from their salaries and ensure the salaries are on par with the going rate for said workers. Arguably it's in the interests of everyone besides companies trying to get cheaper labor via H1-B visas.

The only counterpoint I've ever heard is "It's not fair for company XYZ in low cost of living Podunk, USA because we can't compete at those high salaries against banks / SV / expensive cities". So what? I doubt they can compete against the ability of TCS or Infosys to game the system and get the lions share of the visas either.

7
throwaway251 2 hours ago 2 replies      
Looking at the discussion so far - probably going to get trolled/downvoted but here goes:

From all of the above comments - people are trying to undo only the parts of globalization that they don't like (wage arbitrage is one of them - stop crying!). I truly wonder what would happen if the Indians (and the rest of the world) started treating Americans the same way the America treats them and starts to roll back the impacts of globalization:

1. Stop American businesses from getting favors under trade deals and especially with sales of military equipment

2. Ask each American to provide all their social media account information when entering the country or throw them out

3. Set quotas for American businesses to sell their products/services.

4. Force Google, etc to locate servers and data-centers in China/India directly and give the keys to local governments (if the US government can get access why should other governments not?)

If a trade war did happen:

Specific to India: Their economy is mostly non-export oriented (except the IT services part) - they will probably take longer to raise the quality of living for their population - but it will probably be a better path to take (a trade war would probably help grow domestic businesses faster)

Specific to China: The USA needs access to the Chinese markets rather than the other way. Plus they can always dump all those treasury notes

Perhaps a trade war (rather de-globalization) would be a good idea for the developing world - it would bring better balance to the world and undo globalization as a whole and not parts of it (which is exactly what USA voted for when they elected Trump).

P.S. Please don't give a self-righteous BS response about USA being the land of the free and so on.. I think it's pretty obvious most immigrants are there for the money and quality of living (the kind of quality that comes with money and not society, safety, etc)

8
throwo5 11 minutes ago 0 replies      
I used to work at American Express as full time H1B employee, Arizona location as an Engineer I (10 years exp). They paid me 80K while they paid 110K starting salary for fresh American graduates from Arizona State for Engineer III position. (Engineer I > Engineer III). Also they were promoted from Engineer III to Engineer I within an year, while I did all the hard work with no promotions or salary raise.

H1B visa abuse at its best by an American Company. I am not in US anymore. Left it for good.

9
CodeSheikh 36 minutes ago 0 replies      
Why don't we just stop accepting visa applications from Indian outsourcing firmsTCS, Wipro and Infosys for one year and see how it unfolds? It will resolve the ongoing debate -- at lease prove it one way or another. From a candidate's point of view, I am all in for finding new opportunities in a foreign prosperous land. But gaming the system is not great for the local US economy. Maybe having a new visa category with temp status and easy renewals every six-months. If those candidates are good enough, they can easily find regular H1-B jobs with regular American companies. Some foreigners spend a lot of money and go through severe hardships to get educated at American universities adhering to American socio-economic values. If they are good enough, they get hired by American companies with regular pays. I think it is unfair for them to get categorized under the same blanket rhetoric of "Abusing of H1-B visas".
10
itissid 2 hours ago 0 replies      
As an H1B visa holder, I think the issue of abuse at its core has to do with two things(at least):

1. Too many tech firms require/opt for low cost workers to subsidize their payroll bill. If you raise H1B salaries, firms that have thin margins might automate and outsource. I think to scrap the lottery, and add the market based approach to H1Bs like in the House Bill that was proposed in Jan is a better alternative, it would force firms to be more productive and boost payroll more organically.

2. The program is underfunded, its entirely fees driven and the fees is clearly not enough to prevent abuse we keep hearing about so much.

Remember top end silicon valley's don't really need to care about the salary issue as long as the reform does not dry the talent from coming to the US completely.

11
darkdreams 4 hours ago 2 replies      
Slightly off topic. My understanding is that for every H-1B application that is filed the US government takes a ACWIA fee that is supposed to be used for improving competitiveness of the American worker and providing scholarships.

From https://www.uscis.gov/forms/h-and-l-filing-fees-form-i-129-p...

"SEC. 414 Collection and use of H-1B nonimmigrant fees for scholarships for low-income math, engineering, and computer science students and job training of United States workers".

I am curious whether they could quantify/prove/debunk the skills shortage theory using the scholarships that are given. Does anyone know about this?

12
paulus_magnus2 1 hour ago 0 replies      
(EU citizen point of view) I'd never consider a role in US that pays less than 150k (outside SV) $200k (SV/NY).

H-1B is really bad because the holder has limited bargaining power vs US citizens hence he's forced to accept a lower salary when competing for jobs.

There's also no clean way to allow talent to move around the world.

A fair solution would be to agree on visa free movement of specialists earning above certain threshold ($100, $150k etc), even if it starts with a group of "most favourite nations".

13
bischofs 1 hour ago 0 replies      
Citing the low unemployment rate of STEM graduates to indicate that native workers have nothing to worry about is silly. Basic economics says that wages do not increase until full employment is reached. I may have a job but my wage would be higher if I wasn't competing with 100,000 H1-Bs
14
Consultant32452 20 minutes ago 0 replies      
I wonder how many people against H-1Bs are also opposed to Trump's worker protectionist policy re: NAFTA. Seems that at least philosophically they're aligned.
15
rodionos 5 hours ago 2 replies      
Number of H-1B visas issued for Indian citizens, 1997-2015: https://apps.axibase.com/chartlab/1bc51064

Top H-1B countries: https://apps.axibase.com/chartlab/04040e14

16
nicholas73 1 hour ago 0 replies      
I would go further and say that underpaid H1-B's are a straw man to the real mechanism that depresses American wages. Even if imported workers are paid exactly market, that still means market prices do not go up as there is no one to bid up salaries. That means people are not being compensated for delivery high value, or for developing skill in a difficult or rare area.

There is a lot of hot air between the salary a person would accept versus the value they generate for a company. Having some unemployed people makes it so that the balance always tips towards the low end.

17
forgotAgain 2 hours ago 2 replies      
A constant theme for support of H-1B's is that they supply the US with an irreplaceable resource for starting new technical companies.

Despite this I have not heard reference of any individual who came to this country on a H-1B visa to work for an Indian outsourcing company who later participated in a significant successful startup. Honest question, does anyone know an example of this having occurred?

18
caseysoftware 1 hour ago 0 replies      
> That is not a good argument against them

It's odd that the Economist has this sub-head but then goes on to make the case that Indian outsourcing companies are the biggest consumers and paying a fraction of what the others are paying.. so they are abusing the system.

The rumor (proposal?) is that Trump is going to shift the minimum salary from $60k to $130k which makes it closer to software dev salaries in Seattle, SF, NYC which I think would address this.

19
Technophilis 1 hour ago 0 replies      
The articles gives a good overview of the "H-1B situation". However, if you want to dig deeper here is some data I put together http://h1bpay.com/blog/2017/01/30/h-1b-visa-basics-applicati...

Basically, only Microsoft is among the top 10 sponsors and not of the major sponsors is among the top 10 average salaries.

20
nottorp 4 hours ago 4 replies      
Isn't a H-1B a form of indentured servitude? As in, if you change jobs you lose your visa?

If yes, you don't get the competent ones but the cheap ones who have no choice. The good ones work on their terms for whomever they please.

Edit: combine that with kls's answer about incentives, and you see why this visa system isn't quite working.

21
ausjke 2 hours ago 0 replies      
This has been true for 15 years at least, which is one reason why Wipro/Infosys was getting most IT assignments for US market. When you look into how it worked it is amazing to realize the way the system was abused while no action was taken for decades.
22
mattfrommars 2 hours ago 0 replies      
Well it takes The Economist to point of this fact which I've been telling about it for ages. None of them believed that it was Indian who are favored to get H1B compared to other nationality. Indian take so much pride with Indian worker in these large tech firms and other places instead of realizing the fact they are favored. Why not give other nationality a chance to see what is going on? I've seen it happen with firm like Microsoft with HR manager being Indian and favoring Indian internee and granting him job then Pakistani developer who was without a doubt better performer.

Would love to see some H1B crackdown happening.

23
winter_blue 4 hours ago 1 reply      
This article is repeats the false idea that you cannot switch jobs to other companies. You can switch jobs to other companies that are willing to transfer your H-1B visa. Plenty of tech companies will happily do a visa transfer. Yet, this facetious lie is often repeated. It just goes to show how inaccurate and poorly-researched this article was.
24
hackerboos 3 hours ago 0 replies      
Planet Money republished their podcast on immigration recently: Episode 436: If Economists Controlled The Borders

http://www.npr.org/sections/money/2017/02/08/514152963/episo...

25
harichinnan 19 minutes ago 0 replies      
Many of the people who are against H1B are missing the bigger picture.

1. H1B employers pay wages based on the numbers Department of Labor provides them.

2. Labor department contains a la carte of titles available for the same job to choose from. Employers generally chose a title with lower wages. Programming Analysts and Software Engineers do essentially the same job with upto 50K spread in wages.

3. The limit on H1B works against American Employers who usually pay much higher wages than the Indian consultancies. The American employers have to compete with a flood of applications from India. This forces big companies to subcontract Indian firms.

4. Companies in India randomly select employees to file for visas irrespective of whether they have actual business in US. Many H1B recipients don't actually come to US. Many wait in India for years before their companies arrange for a actual job in India. The lottery forces everyone to game the system.

5. US cannot actually deny Indian consultancies from operating in US. That would be denying market access to India. Indian companies does 60 Billion dollar worth of business in total. That's paltry compared to trade between US and India. US sells everything from Boeing to Starbucks and any restrictions on visas could invite trade war from India too.

6. People who oppose H1Bs actually dream of earning Wall Street salaries in Silicon Valley. That's a pipe dream unless you work in quant engineer jobs in Hedge funds and HFT. You can't build businesses that would pay the average worker 400K in Silicon Valley.

7. Restrictions on H1B would move American programming jobs to Asia much like the manufacturing jobs of rust belt.

8. The world is a much bigger place than America. An Indian worker taking a job doesn't necessarily mean a loss of opportunity for an American worker. It's not a zero sum game. H1B workers take up tech jobs. In most big companies, the ratio of tech to non tech jobs is atleast 1:6. Look at Amazon creating 100K non technical jobs in last few years. A few million technology workers in America make most of the high tech for the rest of the 7 billion human beings on the planet. Trade restrictions are not something you need. There could be a Chinese firewall in every country to protect local jobs and to rob Google and Facebooks of business opportunities. There would be more Baidu's, Youku's, Wechats and Alibabas in every country on the planet.

9. There's actually such a thing as skills shortage at every level. Even at the blue collar end, Many Americans cannot pass tests at 9'th grade level and are functionally illiterate. Tech jobs require a whole lot more skills that the coal miners in Michigan won't be able to take up. https://www.nytimes.com/2017/01/30/education/edlife/factory-...

To summarize, if you are concerned about H1B lowering wages in your market, campaign for worker mobility and higher wages for H1B and the market would work it out. Americans are best at creating free market solutions to problems. H1B lottery system and the restrictions is a socialist solution that best works in restricted economies. Also campaign for a startup visas. Both for entrepreneurs and employees. The Frech Tech visa could be a model http://visa.lafrenchtech.com/ . This would help many of the people languishing in H1Bs to create more companies and more jobs here in US.

26
omouse 1 hour ago 0 replies      
BAM, there is no fucking tech labour shortage! They're just hiring outsourcing firms instead of training. Fucking knew it.
27
perseusprime11 2 hours ago 2 replies      
This thing is the only thing that is keeping the salaries low. I can easily imagine a good software engineer getting paid at least 250-300K if we don't have H1-B visa system.
28
aaron-lebo 5 hours ago 5 replies      
Have there been attempts at pumping H1B money and similar efforts into schools?

I've always wondered if we want minorities to code and we are worried about job loss in the US, why not stop dumping money into hiring foreigners and instead dump it into CS programs at community colleges?

It would take some time before you could build up a domestic work force as talented as foreigners, but would it not solve several issues? Or is it just not practical for other reasons?

29
pinaceae 2 hours ago 1 reply      
yes, and it's filling shit jobs that no American CS grad wants to do.

who wants to do outsourced QA for Oracle? menial, mind numbing clicky work.

who wants to maintain monster codebases built 20 years ago for some internal bullshit system at a Fortune 500?

most of Software work by now is akin to facilities management. hence you give it to motivated foreigners, just like in farming, etc.

30
sergiotapia 2 hours ago 0 replies      
So President Trump was right about this.
31
redsummer 5 hours ago 0 replies      
You could make the Sanders and Trump people happy if there was free college and education for economically underprivileged Americans for STEM / Programming jobs. It doesn't seem likely given the political divide.
32
calvinbhai 4 hours ago 2 replies      
Because citizens of countries other than india and china don't really need h1 visas to work in US.

Many countries have treaty work visas (Canada / Mexico citizens can work on TB visa)

And those who study in US, if their employer starts green card process they can get their EAD before OPT expires.

Only Indians and Chinese have to rely on h1b.

17
UK Teen Hacked 150,000 Printers to Show How the Internet of Things Is Shit vice.com
67 points by Osiris30  1 hour ago   27 comments top 10
1
OJFord 46 minutes ago 1 reply      
This deserves a better, less clickbaity (which I think has the opposite to intended affect with this audience) title.

Particularly towards the end, s/he comes across extremely self-aware, and makes some salient points around the state of school CS education in the UK. It certainly resonated with my experience.

2
milesf 20 minutes ago 0 replies      
The "S" in IoT stands for security :)
3
pen2l 55 minutes ago 2 replies      
I apologize in advance for making a meta-comment.

I hate news organizations using curse words. Don't get me wrong, using curse words is completely fucking fine, but it's weird in the worst way seeing news organizations using these words, they're supposed to carry a voice of impartiality. It's extremely unprofessional, and it seems overly emotional and petulant to use curse words. The Vice is absolutely one of the worse offenders of this.

4
kevinwang 4 minutes ago 0 replies      
Sad to hear that they feel gloomy about their future. Very knowledgeable for high schooler.
5
mentos 21 minutes ago 0 replies      
Could anyone see the FTC creating some sort of unit test that crawls IP addresses in the US and tries to find vulnerabilities like this. Maybe printing out a message for the owner to contact some authority on the 'Internet of Things' ?
6
atesti 35 minutes ago 1 reply      
How is this possible? If random restaurants are able to assign a public IPV4 address to every POS printer they have, do they all have their own /24 net? Why wouldn't they all use NAT?
7
cominous 33 minutes ago 3 replies      
Something that really presses my buttons is this mentality of lots of people to just jump on the complain-train and blame the world that certain technologies for beeing not "as good" as others. And even worse are the people using that to create cheap articles to generate clicks.

"Javasript is sooo broken.. the world is unfair", "Iot is sooo shit...", "Language X is soo bad.. thanks Obama", "Framework Y is soo 2016...".

Thousand people are trying to make a difference in the world and the ones just writing articles about "XY is shit" do mostly nothing. News about bad, bad "IoT", are so low hanging fruits to click-bait. There is almost never a constructive appraoch. Just complain and generate clicks.

Where are the leading ideas to make "IoT" better? Where is the differentiation, that open printers installed by stupid users are not a prove how "shit" IoT is?

You might also say the "internet is shit" because there is major dataleak happens every week.

... just my 2 cents...

8
jwilk 52 minutes ago 1 reply      
Get we get "?utm_source=..." removed from the URL?
9
h4nkoslo 16 minutes ago 0 replies      
Weev's use of every IP-connected printer in the US to publish antisemitic "samizdat" significantly predates this. There's no real exploit in most cases, just things connected to the internet doing what they're told.

https://storify.com/weev/a-small-experiment-in

https://storify.com/weev/why-i-am-getting-a-bowlcut

10
patcheudor 56 minutes ago 2 replies      
Hacking, IMHO is finding something cool about a thing & then taking that something and mixing it up. Make the thing do something never intended. In this particular case a kid found that he could print to printers which were connected to the Internet and then decided to start printing to printers using what amounts to a script. This isn't hacking. It's not novel. People connect printers to the Internet and he abused those connections. This is therefore a prank, not a hack.
18
What the feud between Nabokov and Edmund Wilson says about translation chronicle.com
26 points by lermontov  3 hours ago   5 comments top 2
1
flurie 1 hour ago 1 reply      
I have not read Nabokov's translation of Eugene Onegin, but this piece led me to wonder if he was an inspiration for Stanley Lombardo's translation of the Iliad and the Odyssey, which are stark departures from previous translations that eschew meter in favor of clarity and meaning.

(As a side note, those translations are fantastic, and I highly recommend them.)

2
adricnet 1 hour ago 0 replies      
This is an interesting and thought-provoking read. Thank you for posting it.
19
Go Web Examples gowebexamples.github.io
293 points by adamnemecek  13 hours ago   73 comments top 16
1
StevePerkins 4 hours ago 1 reply      
It seems like there are three classes of "web problems" to solve:

(1) The absolute basic "hello world" stuff at the outset. How to route URL's to their handlers, how to render dynamic templates, how to map data structures to and from JSON, etc.

(2) The problems that are one level up in sophistication, but still common to nearly all web apps. How to handle authentication (username/pass, OAuth/JWT, etc), how to manage session state or the lack thereof, juggling synchronous blocking operations with async background ones, etc.

(3) Really advanced issues that are probably specific to your industry domain or specific application, and aren't necessarily common to others (e.g. how to isolate this medical data for HIPPA compliance).

I don't want to poo-poo this website, it's nice. But in my mind, the world is already flooded with Category #1 examples. At the same time, Category #3 doesn't really lend itself to this sort of thing at all. So what the world really needs is more Category #2 stuff.

Example: What are the patterns available for authentication, including their advantages and drawbacks?

If you do your own authentication with usernames and passwords... then you have to either have to pass them every time and re-auth every request, or else store a session cookie on the client side and find some way share it across all your server-side instances (or use a load balancer with sticky sessions). That's material for 3 or 4 patterns right there!

If you use OAuth with JWT tokens, then you can (maybe) avoid the need to store any session state on the server side. But how do you explicitly "logout" prior to the JWT expiration? Another 3 or 4 patterns down this route!

Category #1 is fun and easy to write about, and a lot af that material comes from newbies who are writing in order to teach themselves. But Category #2 is the brick wall that people always run into in the real world... and because you need a lot of real experience to write about that class of problems, there's a vacuum of available material out there.

2
azinman2 11 hours ago 2 replies      
Fantastic. I've found previous examples hard to get into... either way too complex, not doing enough, or not structured well. Thank you!

Update: 2 requests: http/2, and https (self-signed, with chained cert, and mutual auth)

3
amelius 1 hour ago 0 replies      
> This example will show how to register a route and get the data using just the net/http package.

What is a "route"?

(I don't need an answer, but I think the example should explain it).

4
Shanea93 6 hours ago 1 reply      
This is great, really simple and easy to follow examples.

Would you be able to add some other examples such as interfacing with a database or making remote network requests over http?

5
pkulak 11 hours ago 0 replies      
Love the way that library handles websockets. Perfect example of the way Go tends to work: no magic at all, yet it's also simple, concise and elegant.
6
jhoechtl 9 hours ago 5 replies      
I am currently searching for an example / skeleton in Go where I can upload a file via the browser to the server (drag and drop added bonus). Any examples?
7
bharani_m 11 hours ago 8 replies      
I am interested in trying out Go for my next side project. Can anyone recommend any good books/videos that can help me get started?
8
omginternets 2 hours ago 1 reply      
Anybody have experience with Echo? [0]

[0] https://echo.labstack.com/

9
jchannon 4 hours ago 0 replies      
10
mreithub 9 hours ago 3 replies      
Has anyone of you experience with the fasthttp Go library[0]?

Does it keep its promise in terms of speed? (I assume real world code would spend much more time in actual business logic than their examples/benchmarks so I guess there wouldn't be as much a difference as they claim, but the idea of zero allocations intrigues me)

[0]: https://github.com/valyala/fasthttp

11
petepete 9 hours ago 1 reply      
These look great, thanks. I'm currently learning Go and find this kind of example much easier to follow than documentation.

I'd like to see an example for embedding HTML templates properly. It took me an embarrassing amount of time to get beyond 'header.html' and 'footer.html' being separate, and I'm sure my solution isn't as elegant as it could be.

12
ilozinski 10 hours ago 6 replies      
Gorilla mux is way overrated and overused. I'm not sure why it's the first choice for so many people. There are much better alternatives such as https://github.com/julienschmidt/httprouter or https://github.com/gin-gonic/gin.
13
elmacho39 5 hours ago 1 reply      
As always (not go specific), forms example is missing form validation. Is that too old-fashioned?
14
gempir 10 hours ago 1 reply      
This is awesome. Examples are so important when learning something and I always found it hard to get good Go web examples
15
tlow 12 hours ago 1 reply      
Why put "<li><s>" and "</s></li>" in the html file? Given your style you should clearly abstract this out static HTML.

Edit: update. Odd that people are upvoting and downvoting this comment.

16
adamnemecek 11 hours ago 1 reply      
Actually the link should be to https://gowebexamples.github.io/ I screwed up when posting this lol.
20
A Month of Hello, World magenta.as
30 points by ingve  3 hours ago   14 comments top 8
1
Apfel 2 hours ago 1 reply      
I think this article would be more useful/interesting if it was extended to include language-specific things you found interesting, and specific pieces of expertise you gained in your usual working languages by exploring new ones rather than the rather generalised picture of the entire journey you've included.
2
bad_user 1 hour ago 2 replies      
> I now have experience with 30 additional technologies

He probably hasn't gained anything.

"Hello world" is useless and you're not learning anything with it, not even how to install the compiler or interpreter, because in a year from now the current distribution model might be obsolete already. And regular people can install apps too.

In my experience it takes about a year, at a minimum, to learn a language and absorb the concepts and best practices in its ecosystem. And if a language is so similar to something you already know that it takes less than a year and you don't need it urgently for anything in particular, then that's time wasted.

If in that set there are 10 interesting languages worth learning, then it will take the author at least 10 years.

3
ptero 1 hour ago 0 replies      
A similar approach used to be pretty common in an intro to programming course for a CS major at universities at least 15-20 years ago.

Most homework assignments would be writing a very simple program in 5-6 substantially different languages (e.g., C, Scheme, C++, ML, Pascal). Knowing a number of different languages (and knowing that if needed I can pick a new one pretty easily) was very helpful, at least for me later on.

That said, 30 languages IMO is an overkill, as some are likely to be very similar -- one risks spending too much time on learning the syntax (e.g., commas vs semicolons) instead of getting exposed to fundamental language differences (e.g., closures and objects). My 2c.

4
shekhargulati 2 hours ago 0 replies      
I also ran couple of such series:

2013: 30 technologies in 30 days https://shekhargulati.com/30-technologies-in-30-days/

2016: 52 technologies in 2016 https://github.com/shekhargulati/52-technologies-in-2016

5
Raphmedia 1 hour ago 0 replies      
> My aha moment came around day 16, when I came to the perhaps obvious realization that every Javascript framework is just someones organized, opinionated way of writing code.

About time! Everyone goes on and on and on about Javascript fatigue ... everyone fails to notice that every one of those framework do nothing new except changing how the same concepts are written.

6
diggan 1 hour ago 0 replies      
Probably the Github repository is a better source of this experiment, because it actually contains some learnings from the author: https://github.com/hagata/30daysofHelloWorld
7
amatheus 2 hours ago 0 replies      
I too like learning other programming languages but I use programming praxis problems for that. I have some projects in my bitbucket: https://bitbucket.org/amatheus_personal/
8
dajohnson89 2 hours ago 1 reply      
Check this out: http://www.99-bottles-of-beer.net/abc.html

99 bottles of beer in all the languages.

21
Make Bitcoin Great Again (with Monero's Full Privacy) freedomnode.com
11 points by spaceboy  2 hours ago   1 comment top
1
flaviuspopan 30 minutes ago 0 replies      
TL;DR - Sell BTC on Shapeshift for XMR, Sell XMR on XMR.TO for BTC.

You're basically tumbling the coins through third parties. In reality this has little to do with BTC/XMR but more of a generalized strategy to obfuscate via alt-shifting. You could do the same via Changer, Coinigy, or any other similar service.

23
How Satya Nadella revived Microsoft afr.com
108 points by ghosh  4 hours ago   61 comments top 11
1
awinder 2 minutes ago 0 replies      
I think Microsoft has done a great job of transforming to being very brand-conscious and these stories are both a recognition of that from the press, and a bit of a self-fulfilling prophecy (they talk about how different they are, the press does, then they point to the press and dress up their UX, it's very self-feeding in a way). That said -- I can't say I've experienced great technical changes in the products from microsoft that I do use, at least yet. Windows 10 is still windows underneath, and I've run into some real rough patches on windows 10 on my tower setup. Xbox controllers are still breaking down after several months. Their Cloud offering just blows my mind in a bad way in almost every interaction I've had the displeasure of experiencing.

So I read articles like this and look at the rebranding and really have a hard time deciding whether I want to point to these things as at least having a good direction, and showing that they understand that they need to care about these things. But then I'm dismayed at the execution. Does anyone else have the same thing going on?

2
d--b 2 hours ago 2 replies      
People at Microsoft said the wind of change was blowing long before Nadella took the job. They timed the announcement of Nadella to coincide with deliveries of new products / open platforms. The whole thing was to change the image of Microsoft: New products + new face = new Microsoft.

It's not like Nadella fired every one and started afresh.

3
yaseer 2 hours ago 2 replies      
Although I think Nadella has made some good changes to company culture, there's a lot more variables that influence a company's performance than the CEO, just as there's a lot more variables that influence a country's Economic performance than a ruling party.

The piece does that all-too-common simplification of providing a single cause to explain the fluctuation is MSFT's fortunes, a form of cognitive bias I believe. We like simple stories for complex phenomena.

https://en.wikipedia.org/wiki/Fallacy_of_the_single_cause

4
johnnycarcin 1 hour ago 0 replies      
I can't speak to the product side because up until a year ago I hardly used any MSFT products but I can say that the culture that has been brought by Satya, and likely his reports, is one of the reasons I joined MSFT after being a longtime hater.

When I was approached to come work for MSFT I said "no" right away. The person recruiting me said "just listen to our pitch and then you can say 'no' if you want". After hearing Satya talk about where he wants the company to go and what it'll take to get there I had a 180 degree switch on how I felt about MSFT. I talked with some other old-timers who were there to see if things really had changed and they all told me that it was a slow progression but things were certainly changing for the better.

Now it's totally possible that they all are just good at selling to people but it was enough to get me to join and 95% of the time I'm glad I did.

5
seunosewa 3 hours ago 2 replies      
Somehow, they managed to give Satya the credit for things that, by their own admission, started under his predecessor, such as Office for iPad, the One Microsoft initiative, and the rebound in Microsoft's stock price. They didn't let the facts get in the way of their heartwarming story.
6
mark_l_watson 2 hours ago 1 reply      
I really like the direction Microsoft is going in, especially Surface devices, the very nice Office 355 service, and Azure. The one area where I think they have failed horribly is in the phone market.

The lack of having a solid phone that interacts with their other devices, and has a rich app/developer ecosystem cost them my business recently:

I am in my 60s, and comfortably semi-retired. After decades of being a Linux-as-much-as-possible enthusiast, I now want my interaction with my devices to be as easy and workable as possible. The time I still spend writing and software development should be as efficient as possible, in the deep work sense I want to spend just a few hours a day producing things hopefully useful to society, as effectively as possible.

Recently I spent a month evaluation of staying with Apple or getting a Surface Pro. I stuck with getting a new MacBook and with my iPad Pro because of the availability of the iPhone (I am still on Android, but will switch soon), with nothing comproble from Microsoft.

Whatever it takes, I think Microsoft should get back in the phone business with a winning product.

7
kermittd 3 minutes ago 0 replies      
Is everyone a cynic online?
8
intended 3 hours ago 0 replies      
>Nadella, meanwhile, is keen to stress that the goodwill and positive headlines the company is receiving is only of temporary importance. The main responsibility is ensuring Microsoft remains on the right path in the long-term.

Key point. I'm very happy with Microsoft, and have been since they first announced the surface line and the various other moves they've made to make a single OS.

But They've crossed the threshold of the "holy crap? MSFT did that?" And they are in the "well, this needs to work a whole lot better for me to stick around."

The fact that the CEO is aware of this already is a good sign. Looking forward to the surface event this year.

9
starik36 35 minutes ago 2 replies      
I am not sure how he "revived" Microsoft.

1. They've completely dropped the Phone market under him. As in gave up.

2. They are not in the personal assistant game at all. Alexa & Google have that market all to themselves.

3. The educational market is steadily switching to Chromebooks. The new generation of kids will likely not even know how to use MS Office.

4. On the back end, they've made .NET work on Mac and Linux. This is great for me, since I don't have to worry about Windows Server licenses, but doesn't this eat into their large cash cow?

5. SQL Server - the other cash cow, is great, but free alternatives will start affecting the bottom line.

6. Windows 10 rollout stalled very much short of their stated goal of being on 1 billion devices.

10
anjc 3 hours ago 3 replies      
Ehh. All the credit being given to Nadella are initiatives that Ballmer not only started, but has to push against the will of the board to pursue.

I had major hope for Microsoft in the last few years, and I'm starting to see Nadella's MS slowly unravelling all of the good progress. I just can not believe that Windows Phone adoption got as high as 15% in Europe and then under Nadella they immediately dropped it like it was dirt. The hardware and software was superior in every way to every alternative, and this deprecation has made many of their other successful manoeuvres pointless.

It's mindboggling to me that they're crediting Nadella with Surface and Hololens, in particular.

11
igravious 3 hours ago 2 replies      
Well, if that isn't that a comically spooky infographic:

https://data.afr.com/2017/01jan/microsoft-ceo-hype/index.htm...

24
Show HN: We Freelance, a community where freelancers share stories and resources we-freelance.com
18 points by florentsuc  2 hours ago   3 comments top 2
1
JTxt 46 minutes ago 1 reply      
All I see is "WF" in a box.
2
nicorama 2 hours ago 0 replies      
How much time did it took ? Like commenting a hackernews child in hackernews :)
25
Trump2cash A stock trading bot powered by Trump tweets github.com
288 points by laktak  9 hours ago   108 comments top 27
1
djb_hackernews 2 hours ago 2 replies      
>This bot watches Donald Trump's tweets and waits for him to mention any publicly traded companies.

Ok, simple enough, go on.

> When he does, it uses sentiment analysis to determine whether his opinions are positive or negative toward those companies.

Eh, sentiment analysis isn't perfect but it has come a long way, plus Trump does typically say what he means in simplistic language.

> The bot then automatically executes trades on the relevant stocks according to the expected market reaction.

Ah market psychology. Stop. Do not pass Go. Do not collect $200. This is the tricky bit that a toy project will never get right and turns your slick algorithmic trading project into a monkey and a dart board.

Though this is a sweet example of an API mashup.

2
karangoeluw 4 hours ago 3 replies      
I actually did some basic analysis of this recently - can you really make money in the stock market trading on Trump's tweets. https://medium.com/karan/can-you-really-make-money-when-real...

My answer was maybe but I'd rather not put my money at the mercy of a lunatic.

3
dsacco 4 hours ago 2 replies      
Very cool. Funds have been capitalizing on Trump's tweets since at least December (but probably November) of 2016: https://www.bloomberg.com/view/articles/2016-12-07/flash-cra...

I'd be interested in learning why TradeKing was used , as I haven't used it myself - was it for an especially solid API, or just because TradeKing is easy to get set up and doesn't require a minimum investment? If anyone wants to play around with this and has at least $10k to play with, Interactive Brokers will give you better fees.

OP, you might be interested in Quantopian's Zipline algorithmic trading library, which is also in Python: https://github.com/quantopian/zipline

Also, I feel like there should be a disclaimer that given how accessible this strategy is (both in required skill and resources) and how much attention the Trump tweets phenomenon has gotten, any alpha from this has been almost certainly lost to other firms. This is a cool project, but it's probably not actually an effective strategy anymore.

4
thedrake 3 hours ago 1 reply      
another version could be to capitialize on this:

"The efficient markets hypothesis may be "the best established fact in all of social sciences," but the best established fact in all of financial markets is that, when there is news about a big famous private company going public or being acquired, the shares of a tiny obscure public company with a similar name will shoot up. I don't know what that tells you about the efficient markets hypothesis, but it happened to Nestor, Inc., and to Tweeter Home Entertainment, and to Oculus VisionTech Inc., and now it has happened to SNAP Interactive Inc.:

In what is almost surely a case of mistaken identity, investors sent shares in a little known startup called SNAP Interactive Inc., ticker STVI, surging 164 percent in the four days since Snap Inc. filed for a $3 billion initial public offering. The $69 million SNAP Interactive makes mobile dating apps, while the IPO aspirant is the parent of the popular Snapchat photo-sharing app.These stories are always less impressive when expressed in dollar terms than they are in percentages. In the four trading days since Snap Inc. filed its S-1, SNAP Interactive has traded 19,963 shares, worth less than $200,000, according to Bloomberg data. If you had a cunning plan to buy up SNAP shares and sell them for a quick profit when Snap filed to go public, it might have worked, but not in particularly huge size."

article here: https://www.bloomberg.com/view/articles/2017-02-09/quasi-ind...

5
swalsh 3 hours ago 2 replies      
The fact that Trump has the vocabulary of an 8th grader makes this a lot easier.
6
chis 7 hours ago 1 reply      
Pretty clever and effective message. I'd put money on this being shared by all my friends on Facebook in a few days.

Trump's last couple tweets have backfired and actually increased his targets' stock price though. You might have to flip the algorithm around if this continues.

7
butler14 5 hours ago 0 replies      
An ad agency did this 3 weeks ago

https://www.t-3.com/works/the-trump-and-dump-bot/

8
nodesocket 6 hours ago 4 replies      
Novel project, but not useful. Trump tweeted thanks to Intel on Feb 8th which was a super positive tweet. Since then $INTC is down about 2.6%.
9
martin-adams 5 hours ago 4 replies      
This could make quite a fun experiment to see if it works. Crowdsource an investment of something like $10,000 and let it run for a year. At the end, donate all proceeds to charity.
10
captainmuon 7 hours ago 1 reply      
Brilliant. I've been thinking about something like this for a while.

Why don't you take it to the next level? Provoke Trump on Twitter, associating yourself with a company. Then short that company's stock and wait until he lashes out against them. Profit!

11
mhuffman 3 hours ago 1 reply      
I have been doing the opposite.

Whenever Trump signs on executive order or signals an intention, I presume it will either fail or will end in bad results for the poor and middle class.

Then, working as if that were true, I see if there are any stocks or ETFs that would benefit from that and buy some.

I have been doing pretty good!

12
gliese1337 2 hours ago 1 reply      

 This bot watches Donald Trump's tweets and waits for him to mention any publicly traded companies. When he does, it uses sentiment analysis to determine whether his opinions are positive or negative toward those companies. The bot then automatically executes trades on the relevant stocks according to the expected market reaction.
OK, so what's the expected market reaction? If Trump hates your company, does that make your stock go up, or down?

13
jnaddef 7 hours ago 1 reply      
If many people use this bot, it will make Trump's comments actually effective, which is a pretty bad idea imho.

What about a bot which would do just the opposite?

14
theocean154 2 hours ago 0 replies      
I have one of these as well, but i would've died if it was live trading during the nordstrom incident. They've gone up quite a bit since then.
15
carlmcqueen 3 hours ago 0 replies      
Regardless of this bot being complete, there are better options already mentioned using options instead of short term stock purchases.

Looks like deciding the sentiment of the tweet is still on the to-do list.

def get_sentiment(self, text): """Extracts a sentiment score [-1, 1] from text."""

 # TODO: Determine sentiment targeted at the specific entity.

16
hmate9 6 hours ago 1 reply      
I think Nordstrom stock went up after a negative tweet because it was seen as good publicity for the company. Trump didn't actually "threaten" to retaliate or anything, whereas for companies like Ford or GM he actually said he would impose a heavy "border tax" if they opened factories elsewhere.
17
gurgus 5 hours ago 0 replies      
Cool project!

Makes me want to think of another fun way of crowdsourcing stock tips for fun.

Maybe something like TwitchPlaysWallStreet?

18
chatwinra 3 hours ago 1 reply      
You know at the end of Men in Black it zooms out and our galaxy is just one marble in a galactic game?

I feel like this game is just a small scale version of the one being played by a higher force by making Trump president.

19
AltGr 2 hours ago 0 replies      
I can't state how stupid this is. As if that person's bullshit's power of nuisance wasn't large enough, let's add automated echo chambers with a direct effect on the economy!

Don't be surprised if this crazy economic system can collapse any seconds with ideas like this.

20
makerofthings 7 hours ago 1 reply      
I like this. I think something similar for Theresa May and the GBP:EUR exchange rate might be a good addition.
21
jagermo 7 hours ago 0 replies      
Nordstrom went up after Trumps rant, didn't it? But all in all this looks like a fun thing to do.
22
Gupie 6 hours ago 2 replies      
It would work better is you knew in advance what Trump is going to tweet, not possible of course unless you work for twitter or for Trump?!
23
paulpauper 4 hours ago 0 replies      
the problem is hedge funds are already on his twitter like white on rice. Maybe there is hope in looking for breaking news from less-followed accounts.
24
tmalsburg2 7 hours ago 2 replies      
This is fun. However, does this bot perform better than a random bot? and if yes, how much?
25
AznHisoka 4 hours ago 1 reply      
How many times do we have to go over this?

stocks are a random walk.

26
B1FF_PSUVM 7 hours ago 1 reply      
> mention any publicly traded companies

How often did that happen in the past?

(Not paying attention, specialty of the house ;-)

27
DanBC 7 hours ago 0 replies      
But see this, which says his tweets generally don't make much difference: https://www.ft.com/content/a962c1f8-ee44-11e6-930f-061b01e23...
26
Tesla employee writes of low wages, poor morale; company denies claims arstechnica.com
39 points by sosuke  1 hour ago   26 comments top 6
1
gdulli 1 hour ago 2 replies      
> Speaking to Gizmodo through Twitter Direct Messages, Elon Musk said, Our understanding is that this guy was paid by the UAW to join Tesla and agitate for a union,

I guess he got this tactic from Trump, claiming that any dissent comes from paid protesters. While offering no evidence that it's actually the case.

> Update 10/2/17 9:20am EST: In a statement this morning UAW categorically denied that Moran had ever been paid by their organization.

https://gizmodo.com/elon-musk-responds-to-claims-of-low-pay-...

2
iopuy 54 minutes ago 0 replies      
Wow this article looks like it was written using Gizmodo as the only source. Not only that, it is almost a verbatim copy. Generally Ars has great coverage but this is horrible. Also fun fact the title had to be updated due to being misleading.

 Update: This story went up with two headlines. One of them, "Tesla employee calls for unionization, Musk says thats 'morally outrageous,'" could have been construed as Musk claiming that unionization itself is morally outrageous, which was not the case. We have replaced that headline with the other.

3
rb808 49 minutes ago 2 replies      
> Tesla employee writes of low wages, poor morale; company denies claims

Lol this could be any organization in the world. :)

4
sosuke 1 hour ago 0 replies      
Just a note I really appreciate how ARS documented the title changes at the bottom of the article explaining their reasoning.
5
ashmud 16 minutes ago 0 replies      
How does this compare to Nissan (US), also non-union?
6
simplehuman 1 hour ago 3 replies      
That Tesla pays poor wages is well known. But this is expected since they are more a startup and do not have that much cash to go by. I guess the sad part is they don't hand out much shares either
27
Honest status reporting and AWS service status truth ably.io
105 points by matt_oriordan  3 hours ago   40 comments top 17
1
apeace 2 hours ago 1 reply      
I feel for the OP, I really do. Have been through this many times before.

But it is going too far to call this a "lie" on the part of AWS. If you talk to any AWS customer support representative, or read their docs, they are very clear that the unit of availability for their service is a region, not a zone.

Notice that the status update said "connectivity issues in a single Availability Zone".

If you are deploying in a single AZ, and your app is not tolerant to at least one AZ failure, then you should not be telling your customers/boss that your app is highly available. That's not the fault of AWS, it's how you're using AWS.

With that said, I do think that AWS could do two things: 1) maybe show a yellow warning sign instead of a blue "i", for something that borks an entire AZ, and 2) make it even more clear that each AZ is not high-availability.

2
nhumrich 49 minutes ago 1 reply      
I have worked at amazon, and I can validate this. When I was on an AWS team, posting a "non-green" status to the status page was actually a Manager decision. That means that it is in no way real time, and its possible the status might say "everything is ok" when its really not because a manager doesnt think its a big enough deal. Also there is a status called green-i which is the green icon with a little 'i' information bubble. Almost everytime something was seriously wrong, our status was green-i, because the "level" of warning is ultimately a manager decision. So they avoid yellow and red as much as possible. So the status's aren't very accurate.That being said, if you do see a yellow or red status on AWS, you know things are REALLY bad.
3
kevin_b_er 2 hours ago 1 reply      
Note that if they did have non-green, they might have to credit you due to SLAs. There's likely a strong disincentive internally to have a problem cause non-green status, because it is a metric that will cost amazon money. Amazon has service credits for reduced uptime. As Amazon employees are known for being very very harshly judged on metrics, lying on the status page is thus incentivized.

You are looking at the consequences of amazon's employee culture. Metrics or die means the metrics might need to be fudged to keep your and your team's jobs.

4
hueving 2 hours ago 1 reply      
Aws is massively incentivized to lie on the status pages until the outages become egregious. It's literally a page for marketing to point to when describing how reliable the service is.
5
skywhopper 1 hour ago 0 replies      
I agree that AWS could improve their openness around service problems. That said, what impacts some customers doesn't impact all customers. Ably may be offline when AWS has certain types of trouble, but that doesn't mean everyone with services in that AZ or region are having similar problems.

You also have to keep in mind that AWS status reports aren't just informational. They have a direct impact on how people use their service. As soon as AWS puts up a yellow dot, people start changing DNS and spinning up failovers in other regions or AZs, which has the potential to cascade into a much bigger failure when resources are unnecessarily tied up in other regions for something that maybe isn't as big a crisis as it sounds at first. This has happened before, and is part of the reason why AWS is so circumspect about announcing problems.

Point being that there are ways to architect around AZ and region failure that don't rely on trusting AWS's own reporting. Anyone with a significant investment in or dependency on AWS or any cloud service should not rely on those service's own indicators.

All that said, the fact is that in a system as big as AWS there are minor problems going on all the time. It would be healthier for their customers, their PR image, and the response to bigger incidents for them to reveal more details showing that yeah, most of the time some tiny fraction of the system is broken, offline, in a failure state, or some other unknown issue. 500 errors happen with the API sometime, let's see streaming feeds of fail rates from however many endpoints there are. EC2 instances have hardware failures or need to reboot sometimes, let's see some indication of whether that's happening more or less often than normal. Network segments fail or get overloaded... Revealing some of the less pristine guts of the operation (without revealing sensitive details of course) would go along way to being more honest without the bigger risks of saying EC2 in us-east-1 is down!

6
CoolGuySteve 2 hours ago 0 replies      
I've had this happen to me multiple times. Luckily I use AWS for batch research jobs, so I'm not losing any money when things break.

But when researchers come to me with AWS problems even though I've not changed anything, I usually spend 10 minutes on one of the nodes for some cursory investigation and if it's not obvious what the problem is I tell them to wait a day or two before further investigation. 95% of the time, the problem clears up on its own.

I've learned the hard way that spending half a day tracing SQS messages, dynamo tables, spot bidding, etc is usually a waste of time.

That the AWS console flat out lies like this wastes so much of everyone's time. It's not even hard to fix, AWS could run internal test cases on every subnet group.

7
gommm 1 hour ago 0 replies      
I've long learned that the status page from AWS is useless, it usually only updates 30-40 minutes after the problem starts (by which time, you've already noticed it) and they will always minimize the problem on the status page...

It's frustrating when there's an issue with AWS and my clients tell me that their status page reports that everything is OK.

8
throwaway2016a 41 minutes ago 0 replies      
Related to this:

My favorite is "We are experiencing an elevated level of errors" Which almost always translates to "This service is completely down!"

Although, with that said, they are not lying. 100% failure is "elevated" from "0"

9
web007 1 hour ago 0 replies      
AWS status page is a point of contention within the support teams as well. I've contacted them for problems when the status page says "everything's fine" and they have known outages. There is ALWAYS a lag between them knowing and anything being reported, usually on the order of 15-30 minutes. Most of the time their response is "Yes, there's a problem. I don't know why the status page isn't updated yet, it would really help us (support) out if the page was accurate."

Twitter is usually the best place to watch if you think it's not you, search #aws or #{service_name} and view the "Live" tab to see if others are reporting the same trouble.

Calling attention to these failures via support ticket, via sales rep and even publicly via @ customer support heads has made zero difference. Here's hoping this blog has enough visibility to make a difference.

10
jwildeboer 2 hours ago 3 replies      
But AFAICS they didn't contact AWS support? That would be my very first step, especially when running customer code/services ...
11
benmmurphy 2 hours ago 1 reply      
they really need two dashboards. one for people that want something useful and one for people that want to be lied to.
12
rawat81 54 minutes ago 0 replies      
Damn lies about the number of 9s. I face similar issues with Google Cloud on a daily basis with the APIs and their standard response is to go and buy and buy the next support level before they can look at the problem.

These companies claim to have state of art technologies where as in reality their customers inform them about outages and performance problems.

13
65827 1 hour ago 0 replies      
Seen this a few times myself, weird network outages with AWS refusing to acknowledge
14
franciskim 1 hour ago 2 replies      
Amazon did not lie. You just don't know the difference between Regions and Availability Zones. AWS clearly recommend in just about every paper to use ELB to make your app multi-AZ, and therefore fault-tolerant. That's like the very basics of AWS and you just outed yourself as a complete noob.
15
exratione 1 hour ago 0 replies      
A while back, after some months of frustration with significant levels of S3 API failures in bulk that never made it to the status page, I ended up writing a tool that continually monitors S3 via making random requests, and alerts if too many fail per unit time. The outcome of that experiment is the finding that there are a lot of significant spikes in S3 API failure rates that go entirely unannounced by Amazon, though the situation has improved considerably in the past year.
16
ccvannorman 26 minutes ago 1 reply      
This literally happened to me for the first time this week after years with AWS! My site was totally down for 6 hours but AWS still reports everything as green and no notifications were sent.

Any suggestions for a very simple 3rd party "check" that will constantly monitor my site and alert me (email/text) when it's unresponsive?

17
Dinius 2 hours ago 0 replies      

 TL;DR: Dont trust AWS status reports, theyre lies
Or as some would say, alternative facts.

29
How UPS trucks saved millions of dollars by eliminating left turns ndtv.com
47 points by rootein  2 hours ago   53 comments top 14
1
mstade 2 hours ago 8 replies      
> In the US, and other countries where you drive on the right side of the road, right turns are free, but for a left turn you need to wait for a green light.

I don't know of any other country than the US where turning right on red is allowed, unless there's specific signage or additional traffic lights to allow it. Indeed, the turn-on-red rule in the US seems to work inverse to how I've seen it elsewhere i.e. it's always allowed unless there's specific signage saying it isn't.

I was just in the US, and I love their turn-on-red rules, and I kept thinking that sometimes it's got to be smarter to turn right than go straight ahead, when there's a red light, even if it may be a longer drive. Glad to see someone took the time to figure this out.

2
leereeves 2 hours ago 3 replies      
> eliminating left turns wherever possible after it found that drivers have to sit idly in the trucks while waiting to take the left turn to pass through traffic. So, it created an algorithm that eliminated left turns from drivers routes even if meant a longer journey.

Sounds reasonable.

> In 2005 ... the total distance covered by its 96,000 trucks was reduced by 747,000km. In 2011, ... the company had reduced distance travelled by trucks by 20.4 million miles. A recent report by The Independent says that the total reduction in distance travelled by UPS trucks now stands at 45.8 million miles.

But how does eliminating left turns even it means a longer journey reduce the total distance travelled?

Edit:

The Independent article referenced here explains:

> The efficiency of planning routes with its navigation software this way has even helped the firm cut the number of trucks it uses by 1,100, bringing down the companys total distance travelled by 28.5m miles despite the longer routes.

3
Unbeliever69 2 hours ago 1 reply      
It's funny because occasionally my wife will tell me to take a certain route to a destination because that is what her traffic app told her was the fastest route. I'll disagree on the basis of this experiment; that the "fastest route" has too many left turns and that my route is better because it has fewer. Needless to say, she is never amused. Maybe after this news I'll be vindicated!
4
d--b 48 minutes ago 1 reply      
> In 2005, a year after it announced that it will minimise left turns, the company said that the total distance covered by its 96,000 trucks was reduced by 747,000km, and 190,000 litres of fuel had been saved.

Er... I understand that you may reduce your fuel consumption by not waiting at red lights, but that the total distance driven has been reduced is clearly wrong!

Guys, you have a bug in your spreadsheet!

5
eggie5 16 minutes ago 0 replies      
i talked to my friend, a UPS driver, about this last year when this same story surfaced on HN. He said it's not really true and that they turn left all the time -- but he was aware of some training about it.
6
amelius 2 hours ago 3 replies      
Does typical navigation software assign the same cost to a left turn as a right turn?
7
sctb 36 minutes ago 0 replies      
Discussion from a few years ago: https://news.ycombinator.com/item?id=7538316
8
snrplfth 2 hours ago 0 replies      
This is an impressive result, but not exactly a UPS exclusive. Pretty much any vehicle routing software worth using has offside-turn-minimization options, usually as one of several 'weighting factors'. For example, almost every garbage collection service of any size minimizes its offside turns.
9
shakencrew 1 hour ago 0 replies      
A different article about this was discussed on Hacker News before: https://news.ycombinator.com/item?id=7538316. The cited article also references the Mythbusters video.
10
briankwest 2 hours ago 1 reply      
Yes they may save money doing this, but when you do that follow your driver for your delivery thing they just came out with, It looks like your driver is on crack the travel patterns make no sense. No order. They are all over the place, similar to how Delta does connecting flights.
11
vermontdevil 1 hour ago 0 replies      
Hence the need for more roundabouts.
12
imcoconut 1 hour ago 0 replies      
now that's what I'm talking about. good for ups.
13
Theodores 55 minutes ago 1 reply      
This optimisation must exist at a basic level in all satnav software because journey time must be longer for the route with more left turns, as fed back into the system from previous journeys by other users.

It would be an interesting experiment to see how different consumer grade satnav systems compare on this pure travelling salesman problem.

14
spraak 2 hours ago 0 replies      
This is why I love data science
30
Finland's Largest Trade Union Slams Basic Income as Useless bloomberg.com
59 points by elsewhen  1 hour ago   80 comments top 14
1
nabla9 1 hour ago 4 replies      
This is because unions think they would lose power if basic income is implemented. Unions in Finland have abandoned part time and low income people and only look after people who have full time jobs (who are members in the unions).

Green party in Finland made 560 euro basic income proposal few years ago that was cost neutral. That was verified with microsimulation model that used real world data.

2
stuaxo 1 hour ago 1 reply      
> but the union also suggests that making it easier to refuse unpleasant jobs may create inflationary bottlenecks.

Shitty jobs being valued higher in monetary terms doesn't sound like the worst outcome I can think of.

3
jonwachob91 1 hour ago 3 replies      
It's interesting to understand the primary motivator for UBI around the world. It sounds like the Fins want to use it to take away the minimum wage, while in the US the primary motivator is to take away the rest of the welfare services.
4
digitalronin 12 minutes ago 0 replies      
How utterly pointless is it to say "This experiment will fail" - they seem to have missed the point of what an experiment is.
5
digi_owl 39 minutes ago 6 replies      
In the end it all seems like a no-win situation.

basic income, much like housing debt, will just set a new price floor. This any merchant worth their salt will price things at that point plus a markup.

Thing is that going the other way and freezing the amount of currency in the economy, like say with a gold standard, will do crap all as well. As that just means things will be prices fractions of fractions, and the pay will be likewise.

In the end the only thing that can really solve things is massively, and i mean Star Trek scale massively, crank up supply of goods and services.

And unless we have something like Deuterium (aka the handwaved wonderfuel of every space age sci-fi) and replicators to go with, thats not going to happen as we run into an energy crunch sooner than we like to think.

Effectively humanity can't win.

6
ende 1 hour ago 2 replies      
Union leaders don't like UBI because it empowers individuals, not union leaders.
7
jcon321 39 minutes ago 3 replies      
I don't understand the principle behind UBI. If everyone makes +$X amount more doesn't everything just raise by +$Y?
8
ar0 28 minutes ago 0 replies      
Almost all of the comments here are very dismissive of the trade union's arguments, but I think it is very important to look at the key problem UBI faces IMHO: Proper work is not just a source of money, but also a source of pride and a feeling of purpose / usefulness. And UBI alone cannot replace this.

This is not only a problem with UBI but with all welfare-focused policies, regardless of how well-intentioned and useful they might be. Put yourself into the shoes of someone on the receiving end. The story a blue-collar worker you are automating away is hearing is this: "We are sorry, but in our AI-fuelled, robot-driven world you are useless. We have no need for you anymore because you can't program deep neural networks. You just don't have what it takes to be a useful member of our society. But - don't worry! We recognize this is a problem, so we have introduced UBI / <insert other welfare programs here> to make sure you are financially covered and don't have to worry about your mortgage."

Is this something you want to hear? No. And that's why people are dismissing UBI and vote Trump - because "You are a hard-working, very capable individual whose job was unfairly stolen by an illegal immigrant" is a much, much better story for your self-esteem. And self-esteem is important [1]!

Not all people have a bucket list of things they would want to do in their spare time, like contributing to cool open source programs or writing the next Harry Potter series. Many just want to live a "regular" life and want to feel useful by providing for their families. They benefit from having a job where they feel useful and are accepted and where someone else (their boss) structures their work and tells them what to do. Taking this away and replacing it with UBI cannot be the (whole) answer. [And, no, I don't have the right answer, either.]

Money is not everything.

[1]: https://en.wikipedia.org/wiki/Self-esteem#Importance

9
Rezo 50 minutes ago 0 replies      
"Not only does SAK say that the system may reduce the labor force -- for instance by tempting mothers of small children or those close to retirement to take more time off"

As long as the unemployment rate is > 0, that sounds like a net benefit for UBI. Someone else who isn't at the same life stage can then fill the freed position. You would only think this is a bad thing if you see workforce participation as the end-goal in itself, while all signs point towards massive societal upheaval due to automation without something like UBI.

10
amelius 52 minutes ago 3 replies      
BI is just a negative form of income tax. So can't we just reduce the relevant income taxes slowly until we start running into problems?
11
ilikeatari 32 minutes ago 0 replies      
I cannot recommend enough Kurt Vonnegut's Player Piano as a great thought provoking material on the UBI topic. It made me rethink its potential implications.
12
markhahn 57 minutes ago 0 replies      
Unions want to present themselves as the only possible savior for the downtrodden. But of course! Since when have any two saviors been willing to coexist?

Also a bit strange to hear capitalist-incentive reasoning coming from a union...

13
sharemywin 1 hour ago 2 replies      
I would be much more in favor of a employer of last resort program.
14
snarfy 50 minutes ago 0 replies      
Automation will replace all of those trade union jobs, and then they will find basic income not so useless.
       cached 10 February 2017 17:02:02 GMT