I wanted to introduce our startup SnapEDA to the HN community. We recently completed Y Combinator, and have been quiet about the platform while we've been improving it. With that said, wed love to get feedback from the HN community!
Our goal is to build a canonical library for making circuit boards: one trusted, centralized place to get digital models. These digital models include PCB footprints, schematic symbols, and 3D models. The library exports to a growing set of popular EDA tools: EAGLE, Altium, KiCad, Cadence OrCad/Allegro (Beta), & Mentor PADS (Beta).
The library is free because we believe in making this data widely accessible to enable innovation. The purpose of this new feature, InstaPart, is to give designers an option to "skip the queue" and get a part quickly if it doesn't yet exist in the free library. Once that part is made, it is then made available for the entire community to download for free. Growing the library is a top area of focus, so we hope to eventually render the InstaPart feature obsolete and just have everything available natively. :-)
In terms of standards, all new libraries are being made to IPC, and we also source models by partnering with component manufacturers. To ensure quality, we have an automated verification checker on each part page that provides a pass/fail result on common manufacturing issues that we plan to expand with additional checks.
So, it's your friend...but it's also there to sell you stuff. But just think of it as your friend.
It sounds like they don't quite know what they're selling or how it's going to be useful to people (which they kinda admit, to their credit). I could see getting utility from a VA that also suggests service's for specific needs, but a friendship with "ambient intelligence" behind it figuring out how it's going to chum up its next product placement? If it's really a "true AI" why not sell it on that merit alone?
Maybe it's a bit off topic, but I'd like to put here fiction sources (which I know) that explore this kind of setup:
1) A Cognitive Discontinuity story by Andrej Karpathy (2015): http://karpathy.github.io/2015/11/14/ai/
2) Sleep Dealer movie (2008): https://www.rottentomatoes.com/m/sleep_dealer/
What sticks with me, however, is how the story gave the player freedom while still remaining engaging (exactly the points the article makes). So-called computer RPGs focused on the most boring parts of role-playing---combat---and completely ignored the thing that makes RPGs what they are, the player's role in creating the story! Bards Tale and Wizardry games that are contemporaneous to Quest for Glory are decidedly weaker sauce. Games like Ultima that tried to create an open world mostly ended up filling that world with bland variations on the same theme, and lost any narrative drive through endless side quests. Although we ground like only kids with nothing better to do can, there was no need for excessive grinding in Quest for Glory. You could push the story forward fairly quickly, and every puzzle had multiple solutions.
Totally agreed with 32bitkid that Deus Ex is the closest contemporary game to Quest for Glory, combining strong narrative with player freedom.
> Rule #1 is The Player Must have Fun. Its trivially easy for a game designer to defeat players. We have all the tools and all the power. The trick is to play on the same side as the players, to tell the story together, and to make them the stars.
So many game designers (especially amateur ones) seem to think game design is about screwing over the player at every opportunity. That it's a battle between the designer and the player, with the aim to make things as difficult as possible.
And the point about a lot of Sirerra's designers not actually having much experience playing video games kind of says a lot, doesn't it? Certainly explains how brutal most of their adventure games were.
In fact, just this morning, my little brother asked me if the new dues ex is worth picking up and I described it as "scratching my itch for the old quest for glory series"
It's great to read insights into some of these games, though I sometimes worry about the history of a lot of the influential works in this medium. I'm afraid it'll get lost and unappreciated, and be unplayable. Can there be a "criterion collection" for early video games and interactive fiction. Is something that even matters? Maybe I'm just getting old.
Snapchat can win here based on brand alone. The hardware features are a plus, but they're going to sell a lifestyle. Think GoPro + Versace. Commenters here are caught up in the tech. It's not the tech. Get a few celebrities in these, people will buy them and barely use the recording features. They're cheaper than Ray-Bans and I bet you and half of your friends own a pair of those.
Snapchat can assemble an AR powerhouse from the ground up with brand goodwill. Evan and his team have figured out the best market strategy to do so. Google is not "cool" and could never attempt to pull this off.
I have tremendous respect for Evan Spiegel right now. Bold move. Amazingly positioned. I wish them the best of luck. Dare I say, it has the scent of Jobs to it - the vision, the risk ("we make sunglasses now!") and definitely the "cool-factor." Don't misinterpret - this isn't the iPhone, not yet anyway, but I think they're on to something very big.
Of _course_ they're sunglasses.
Of _course_ it's focused completely on video.
Of _course_ it's marketed as being about sharing your memories as you lived them.
Of _course_ you can only record 10 second videos at a time.
Of _course_ snaps automatically sync to the app.
Of _course_ they're designed to appeal to young fashionable people.
Of _course_ the charge lasts all day
This is one of those things where once you see it it's just obvious this is what it was supposed to be all along.
It's better because it focuses on the one thing that is really easy to do well. It does not try to do everything at once. It doesn't try to give you apps in your glasses and everything under the sun. This is the right approach to products. Do one thing but do that well.
Before you criticize me think back to the original iPhone, it didn't start with an App Store and everything under the sun like the iwatch did. And yet the iPhone is an icon and the watch is no big deal.
Quite impressive, you have to see it in action:
1). The messaging emphasizes it's just "a toy", a low volume experiment. More playful and more humble approach makes it a smaller target for ridicule.
2.) Pricing at $149 also makes it less pretentious and more importantly, puts it in the discretionary income range of what the heck I'll give it a shot.
Snapshot is an online multimedia application.The infrastructure required to move from online to hardware requires significant investment (beyond the $1.8B they recently raised) - that of which I don't believe Snapchat can fund without a serious re-monetiziation strategy beyond Ads. It is only a matter of time before FB makes the move into Snapchat's market more than they already are.This is an unproven market. Google tried it and didn't succeed. A better play - let someone else test the market a bit more and then move in with a solid Ad monetization strategy around the Spectacles. Why Hardware?! Seriously? I believe Evan is overplaying his hands with so much VC capital coming his way.
The ability to have cheaper, stylish, handsfree video recording of my POV has a lot of potential. How-to videos, the "capturing memories" as noted in the article, even just easily recording benign life experiences (police stops, for instance) seamlessly and without hassle is huge.
I do hope there is a tattletale light or something so that the average user can't surreptitiously record things and otherwise easy privacy controls... and I hope it's not long before someone hacks this or they unlock the product to do more than 10 second clips...
If I were GoPro I'd be nervous.
Edit: Actually a second thought- this would be a lot better than body cams in a lot of situations (or certainly a good companion) because it would capture the officer's line of sight.
Just like Google Glass users being called Glassholes, SnapChat glasses will probably be called something like SnapChads, because only white rich guys in pastel shorts and rugby shirts named Chad will use them. The aesthetic just isn't there for wide adoption.
Particularly since I feel it will inspire the next product which is an IR flood light that renders all digital cameras useless, since there are so many people oblivious to the fact by trying to capture the experience for themselves they're detracting from the experience for everyone else.
Letting people who need a digital memento silently get one without intruding on the experience of those of us just there to enjoy and be in the moment is a great compromise.
It's also the shape of nearly all screens in the world. Perhaps I'm not visionary enough, but I don't foresee a circular computer or phone screen really improving the current situation...
Are they able to darken the lense glass to hide the camera a bit? Maybe they could match the black of the camera sensor to the black of the glass a little more. Otherwise it looks a lot like two cameras on your face.
The way they framed this product is _so_ refreshing.
If anything kills GoPro it's something like this.
That said, I worry about implementation. My guess is that it's going to be directly and permanently tied to Snapchat itself. Which significantly reduces the potential usefulness of this product - not everything you record is something you only want to have sent directly to Snapchat. Personally, I want files. Plain, old files. Is that so hard to understand for all those cloud-first companies?
How do they solve the personal privacy issues that arose with Google Glass? Or have they even bothered?
I cannot believe this is still an issue for major publications.
While it's less offensive than Google Glasses, this doesn't look like "normal" glasses.
> (Why make this product, with its attendant risks, and why now? Because its fun, he says with another laugh.)
Sometimes you can look at something and just KNOW that there is not a chance that pile of junk is gonna gain traction.
>The most important principle on HN, though, is to make thoughtful comments. Thoughtful in both senses: civil and substantial.
"Google Glass 2.0" and similar cheap bashing isn't just against the rules, it's boring and petty.
Take it to 4chan, you'll get the attention you're after.
We already had 10 Tbps per fiber pair: http://www.extremetech.com/internet/231074-googles-faster-un...
Go go Nokia Bell, go away zdnet ;)
- aesthetic surgery (in a close race with actresses Emanuelle Beart and Angelina Jolie)
- aliens (or people looking like, claiming to have met, or communicate with aliens)
- chins (a rather popular comedian has based about 12% of all his jokes about silly puns with Bogdanov's brothers and the word 'menton')
- lots of other stufff
- mad scientists
- and, I suppose, cosmologists (although it is probably a challenge to make a cosmologist joke in front of a regular audience.)
For whatever it's worth, next time you have to make a standup routine in France.
This was, obviously now, not true, at least for the "PhD" part of the claim, so at one point they had to try to pass PhDs for real. It took them quite a bit of time and effort, and this effort, as I had been told at the time the "affair" was unraveling by mathematicians who had been involved in this process, was to call famous scientists of their time over the phone asking them questions about the subject of their thesis (after all they were some sort of journalists so at least they knew how to ask questions), and transcribe the answers. Unfortunately they only understood the words of these conversations but not their meanings, which explains why their thesis can look like some genuine science if you look at the sentence level, but doesn't make any sense when you try to find any meaning in it.
Some recent discussion about the affair (in French): http://sciences.blogs.liberation.fr/2015/07/02/les-bogdanov-...
What's doubling truly sad in this story, IMHO, is that:
1) While the are obviously not the scientific geniuses they claim to be, they are even not good science popularisers, they are actually really bad.
2) There are still people, and not only people attracted by pseudoscience or the bizarre aspects of their personality, that take them seriously. I remember, for instance, that that have been interviewed in recent years on France Info (national public french information radio) about the Higgs boson discovery or their recent books.
There's a certain beauty in an entire field losing coherency.
> One of the scientists who approved Igor Bogdanov's thesis, MIT's Roman Jackiw... was intrigued by the thesis, although it contained many points he did not understand. Jackiw defended the thesis:
> "All these were ideas that could possibly make sense. It showed some originality and some familiarity with the jargon. That's all I ask."
Igor thesis : https://tel.archives-ouvertes.fr/tel-00001503v1/document
They recently had their own parodical song:
This is about their supposedly abnormally long chin (people regularly joke about Bogdanov being aliens).
* Youve got to dig it to dig it, you dig?
* I want to avoid the hecklers.
* Always leave them wanting more.
An obvious concept that can so easily be forgotten at times.
(Also, we changed the URL from http://videlalvaro.github.io/2014/09/a-programmers-role.html to the original source, in keeping with the HN guidelines, which call for original sources. In this case the submitted article does add glosses, but not enough to be more substantive than the original. Historical material is particularly welcome on HN, especially when it hasn't appeared before, and this piece doesn't seem to have.)
"In Jewish mythology, a dybbuk (Yiddish: , from the Hebrew verb daq meaning "adhere" or "cling") is a malicious possessing spirit believed to be the dislocated soul of a dead person."
I know many programmers, that just write the code.
For example: what problem is solved by the developer when he/she coding the login page?
It just like any other profession, only a few people in the group are able to solve problems
Or like the Bitcoin "evolution": CPU -> GPU -> FPGA -> ASIC (this is a very simple, single purpose optimisation, but it can illustrate part of picture)
Only optimising transistor speed, their size or the whole CPU/GPU package has obviously limitations and may be a dead end.
That and the underwhelming offering on Android compared to what Apple has been able to deliver.
Of course work on custom chips has been going on as long as chips were a thing. The article just underlines that this is now pretty much the only way forward.
Ensuring blood thinners are prescribed post coronary bypass surgery or stenting is another one. It should happen every time, but sometimes by human error it may not.
A good startup idea would be to make a hotlist of the top ~1000 of these obvious, preventable errors, combine IT data sources to predict when they might be occuring, and have checking systems in place to prevent them. If it worked even adequately, hospitals would be seen as negligent for not using such a system.
I know of nurses who could barely keep their eyes open while driving to work. And that's because they just finished a shift at a different hospital.
At a time, a friend of mine had to be picking up his wife from work, because he was afraid of what might happen if she drove herself.
So let's fix the sleep deprivation problem.
>>> 30. (von Tiesenhausen's Law of Engineering Design) If you want to have a maximum effect on the design of a new engineering system, learn to draw. Engineers always wind up designing the vehicle to look like the initial artist's concept.
This is common with a lot of figures from the past, of course, from Picasso the wifebeater to Wagner the antisemite, Washington the slave owner, to Gary Glitter the child abuser.
Amazing career. Can't wait to see the movie.
It is difficult to prove a statement like "..he pioneered hard bop.." wrong, but it is definitely a new view on the history of jazz in the post bop era.
That being said. What's actually kind of good (with actual technical specifications) is Bitcoin wiki, even when it's slightly outdated; then official bitcoin website; and sometimes bitcoin stack exchange website (but that can become outdated too).
I don't think Blockchain can be disconnected from Bitcoin, and if you do, it's very general and not that specific.
There is absolutely no reason to pass arguments to ssh-keygen. If it is actually deemed necessary to do so, then that package's installation is inexcusably broken.
But RSA isn't broken, it is well understood, is "boring" (a plus on security, usually), has bigger bit sizes (according to people that know a lot more to me that's a plus point, regardless of EC requiring smaller ones, because of certain attacks), isn't hyped and sponsored by the NSA and isn't considered a bad choice by experts.
Not too many years ago Bruce Schneier was skeptical about EC, because of the NSA pushing for it. Now, I also trust djb and i an sure that ed25519 is a good cipher and there are many projects, like Tor that actually benefit from it, increasing throughput, etc., but for most use cases of SSH that might not be the issue, nor the bottleneck.
So from my naive, inexperienced point of view RSA might seem the more conservative option. And if I was worried about security I'd increase the bit size.
Am I going wrong here?
"So let me spell this out: despite the fact that quantum computers seem to be a long ways off and reasonable quantum-resistant replacement algorithms are nowhere to be seen, NSA decided to make this announcement publicly and not quietly behind the scenes. Weirder still, if you havent yet upgraded to Suite B, you are now being urged not to. In practice, that means some firms will stay with algorithms like RSA rather than transitioning to ECC at all. And RSA is also vulnerable to quantum attacks."
Stick with the battle tested RSA keys, which are susceptible but not as much as ECC crypto. 4097 or even better 8192-bit lengths.
There's no perceptible user benefits to using ed25519 and it's not even supported everywhere. Also you won't have to rotate all of your keys when workable quantum computers start crackin' everything.
Host foo.example.com Keyfile ~/.ssh/my_obsolete_private_keyfile
Did 1083 RSA 2048 signing operations in 1017532us (1064.3 ops/sec) Did 29000 RSA 2048 verify operations in 1016092us (28540.7 ops/sec) Did 1440 RSA 2048 (3 prime, e=3) signing operations in 1016334us (1416.9 ops/sec) Did 50000 RSA 2048 (3 prime, e=3) verify operations in 1014778us (49271.9 ops/sec) Did 152 RSA 4096 signing operations in 1000271us (152.0 ops/sec) Did 8974 RSA 4096 verify operations in 1076287us (8337.9 ops/sec) ... Did 6720 Ed25519 key generation operations in 1029483us (6527.5 ops/sec) Did 6832 Ed25519 signing operations in 1058007us (6457.4 ops/sec) Did 3120 Ed25519 verify operations in 1053982us (2960.2 ops/sec)
(also don't look at these numbers purely as speed, but as CPU time spent)
And I think that was in the context of some DSA or ECDSA weakness, possibly a side channel attack or something similar. I forgot the details :(
What are your thoughts on this? Should we focus more simplicity and robustness of the implementation, rather than just the strength of the algorithm itself?
Thank you, I am sufficiently paranoid enough to change my keys now.
I haven't checked, but I presume this also goes for CentOS, Scientific Linux, and other derivatives.
sources.list (if you're on an older version of debian)deb http://http.debian.net/debian wheezy-backports main
apt-get -t wheezy-backports install --reinstall ssh
ssh-keygen -t ed25519 -f ssh_host_ed25519_key -a 256 < /dev/null
ssh-keygen -t rsa -b 4096 -f ssh_host_rsa_key < /dev/null
(do not password protect server side keys)
ssh-keygen -t ed25519 -a 256 -f yourkey.key -C whateveryouwant
Could someone provide a link with decent explanation why? Is it solely out of fear that it will be cracked soon on quantum computer?
Follow these instructions to update your keys.
many people still using RSA/DSA keys :/some people are doing even worse things.Last week I saw one man who have shared his priv key by email message!
QWERTY people have to grow up!
I like this post - it's good advice overall. Keys are easy to handle and in some ways more secure than certificate management (which relies on extra unnecessary infrastructure).
Um, I'm pretty sure he meant privacy, not "privates." Time for an edit.
After Bowie died I listened to quite a lot of his back catalogue over the next month, I then found my discover weekly had been taken over by 70's prog.
I've heard similar things from people who have let friends or family use their account, or made a 'bad' playlist for a one-off event.
Really I want to 'delete my history' for a couple of weeks - private browsing requires foresight. Maybe it's not possible given how the 'musical taste' data gets hashed? In that case I'd take a full reset!
It completely bypasses the music distribution industry's gatekeeper behavior. This is previously unheard of, because the labels have a lot at stake, wherever there is a natural interface where music can be presented, they quickly move in and start to use money to control the playlists. I'm pretty sure Spotify's charts are heavily influenced by music industry shenanigans at this point.
For me, the best playlists are the ones written by the long-standing institutions, like Hardwax, Berghain and Resident Advisor.
 https://hardwax.com/ http://berghain.de/ https://www.residentadvisor.net/
Spotify weekly went downhill at some point and I wonder if it just feeds back in to itself when I listen to it at work.
That trust has taken a massive hit. You can no longer reply on the same video-id having the same content as before.
YouTube has the most confusing and maddening UXes of any service I've ever used, and it's stuff like this that keeps shortening my lifespan.
It happens on FB when somebody changes the name and goal of a group, hijacking the non active members.
But because the tools to edit aren't available to everyone, they probably don't want to make this become an issue so tried to do it stealthily.
At least they didn't remove a few of the dislikes in the process (although I have no evidence they didn't).
As for the Heroes scheme itself, I don't get it. I don't know what the motivation to put all that work in is, as there is no money in it and the rewards seem to only serve the tool, but not the individual putting the time in.
http://www.popsci.com/europa-or-bust (Sep 2015)
I suppose it has been debunked, but it still seems apropos...http://stanleyrumm.com/?p=12
Port Talbot [in Wales] is a steel town, where everything is covered with gray iron ore dust. Even the beach is completely littered with dust, its just black. The sun was setting, and it was quite beautiful. The contrast was extraordinary, I had this image of a guy sitting there on this dingy beach with a portable radio, tuning in these strange Latin escapist songs like Brazil. The music transported him somehow and made his world less gray.
But the significance of this breach is not the only thing that caught my eye.
These litigants have been entrenched in scorched-earth litigation for years now in which the working M.O. for both sides is to concede nothing and make everything the subject of endless dispute. Big firm litigators will often do this. It is a great way to rack up bills. Clients in these contexts do not oppose it and very often demand it. And so a lot of wasteful lawyering happens just because everyone understands that this is an all-out war.
To me, then, it seems that the big problem here (in addition to the improper disclosures of highly important confidential information in a public court hearing) was the resistance by the lawyers who did this to simply acknowledging that a big problem existed that required them to stipulate to getting the transcript sealed immediately. Had they done so, it seems the information would never have made the headlines. Instead (and I am sure because it had become the pattern in the case), they could not reach this simple agreement with the other lawyers to deal with the problem but had to find grounds to resist and fight over it.
I know that we as outside observers have limited information upon which to make an assessment here and so the only thing we can truly say from our perspective is "who knows". Yet, if the surface facts reflect the reality, then it is scarcely believable that the lawyers could have so lost perspective as to take this issue to the mat, resulting in such damage to a party. Assuming the facts are as they appear on the surface, this would be very serious misconduct and I can see why Judge Alsup is really mad that it happened.
A better title might be:
"Google is trying to get Oracle in trouble for revealing confidential figures"
The Death of "Free" Software . . . or How Google Killed GPLby Annette Hurst (@divaesq)
The developer community may be celebrating today what it perceives as a victory in Oracle v. Google. Google won a verdict that an unauthorized, commercial, competitive, harmful use of software in billions of products is fair use. No copyright expert would have ever predicted such a use would be considered fair. Before celebrating, developers should take a closer look. Not only will creators everywhere suffer from this decision if it remains intact, but the free software movement itself now faces substantial jeopardy.
This wasn't an accidental "slip" by a poorly trained intern. This was a conscious disclosure made by one of Oracle's lead attorneys. She is one of the top IP lawyers in the nation: https://www.orrick.com/People/2/6/2/Annette-Hurst. It is in keeping with the "scorched earth" strategy that has been followed for this case. She knew what she was doing, and she (and her firm) should pay the consequences. If there are no consequences, it will legitimize and reward this strategy.
I don't (or can't, I'm unsure) believe that lawyers of this caliber make mistakes like this. So what was her play by doing this? Did it pay off?
Shouldn't the structure of accountability be in the other direction?
But since the API is "implemented" in code, it seems like for the purpose of copyright consideration that the distinction is simply one of custom.
It's a programming abstraction, to create your own "implementation" of the API you still have to use code that is identical to original.
Alsop's original, overturned, ruling was that as a matter of law API's couldn't be copyrighted because they express an idea that can only be expressed exactly that way, and traditionally this would not be allowed (can't copyright an idea). As I understood it, his concept implied that to get IP protection over an API would require something more like patent protection. (I might be totally wrong on this).
God I hate that woman. When she was a US Attorney for SF, she went around and threatened to seize buildings where medical cannabis dispensaries were located, in full compliance of the local laws. Because she couldn't do any thing to the dispensary directly, she threatened their landlords. This was after Obama had said that DoJ would not interfere with dispensaries which were operating within the state laws.
> If she had had the recipe for Coca-Cola she could have blurted it out in this court right now.
EDIT: I wasn't trying to be snarky or silly, just pointing out an aspect of the story that struck me as funny. Serious request: if that is inappropriate, please let me know rather than just silently downvoting. In that case, I apologise and will delete the post.
Can someone explain me this one??
And if a lawyer did break the law by doing it, I say she belongs on the same high pedestal people put Snowden on.
Actually, what Sartre called existentialism is the Asian way of life, at least in societies influenced by the original Hindu philosophy (Upanishads) and its Buddhist derivatives and finally the Gita.
...I have nothing to apologize for.
Also, the tool aside, this blog post should be held up as the gold standard of what gets posted to hacker news: detailed, technical, interesting.
Thanks for your hard work! Looking forward to taking this for a spin.
It looks like ripgrep gets most of its speedup on ag by:
1. Only supporting DFA-able Rust regexes. I'd love to use a lighter-weight regex library in ag, but users are accustomed to full PCRE support. Switching would cause me to receive a lot of angry emails. Maybe I'll do it anyway. PCRE has some annoying limitations. (For example, it can only search up to 2GB at a time.)
2. Not counting line numbers by default. The blog post addresses this, but I think results without line numbers are far less useful; so much so that I've traded away performance in ag. (Note that even if you tell ag not to print line numbers, it still wastes time counting them. The printing code is the result of me merging a lot of PRs that I really shouldn't have.)
3. Not using mmap(). This is a big one, and I'm not sure what the deal is here. I just added a --nommap option to ag in master. It's a naive implementation, but it benchmarks comparably to the default mmap() behavior. I'm really hoping there's a flag I can pass to mmap() or madvise() that says, "Don't worry about all that synchronization stuff. I just want to read these bytes sequentially. I'm OK with undefined behavior if something else changes the file while I'm reading it."
The author also points out correctness issues with ag. Ag doesn't fully support .gitiginore. It doesn't support unicode. Inverse matching (-v) can be crazy slow. These shortcomings are mostly because I originally wrote ag for myself. If I didn't use certain gitignore rules or non-ASCII encodings, I didn't write the code to support them.
Some expectation management: If you try out ripgrep, don't get your hopes up. Unless you're searching some really big codebases, you won't notice the speed difference. What you will notice, however, are the feature differences. Take a look at https://github.com/BurntSushi/ripgrep/issues to get a taste of what's missing or broken. It will be some time before all those little details are ironed-out.
That said, may the best code searching tool win. :)
I don't think this is correct. glibc has architecture specific hand rolled (or unrolled if you will lol) assembly for x64 memchr. See here: https://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/x86...
Chris Mason of btrfs fame did some proof of concept work for walking and reading trees in on-disk order, showing some pretty spectacular potential gains: https://oss.oracle.com/~mason/acp/
Tooling to do your own testing: https://oss.oracle.com/~mason/seekwatcher/
Id like to try to convince you why you shouldnt use ripgrep. Often, this is far more revealing than reasons why I think you should use ripgrep."
Love that he added this
Tried it out on a 3.5GB JSON file:
# rg rg erzg4 k.json > /dev/null 1.80s user 2.54s system 53% cpu 8.053 total # rg with 4 threads rg -j4 erzg4 k.json > /dev/null 1.76s user 1.29s system 99% cpu 3.059 total # OS X grep grep erzg4 k.json > /dev/null 60.62s user 0.96s system 99% cpu 1:01.75 total # GNU Grep ggrep erzg4 k.json > /dev/null 1.96s user 1.43s system 88% cpu 2.691 total
I think a lot of the residual love for mmap is because it actually did give decent results back when single core machines were the norm. However, once your program becomes multithreaded it imposes a lot of hidden synchronization costs, especially on munmap().
The fastest option might well be to use mmap sometimes but have a collection of single-thread processes instead of a single multi-threaded one so that their VM maps aren't shared. However, this significantly complicates the work-sharing and output-merging stages. If you want to keep all the benefits you'd need a shared-memory area and do manual allocation inside it for all common data which would be a lot of work.
It might also be that mmap is a loss these days even for single-threaded... I don't know.
Side note: when I last looked at this problem (on Solaris, 20ish years ago) one trick I used when mmap'ing was to skip the "madvise(MADV_SEQUENTIAL)" if the file size was below some threshold. If the file was small enough to be completely be prefetched from disk it had no effect and was just a wasted syscall. On larger files it seemed to help, though.
Some discussion over on /r/rust: https://www.reddit.com/r/rust/comments/544hnk/ripgrep_is_fas...
EDIT: The machine I'm on is much less beefy than the benchmark machines, which means that the speed difference is quite noticeable for me.
RUSTFLAGS="-C target-cpu=native" rustup run nightly cargo build --target x86_64-unknown-linux-musl --release --features simd-accel
I'm convinced to give it a try.
Just out of curiosity, what kind of use case makes grep and prospective replacements scream? The most "hardcore" I got with grep was digging through a few gigabytes of ShamePoint logs looking for those correlation IDs, and that apparently was completely I/O-bound, the CPUs on that machine stayed nearly idle.
I find this simple wrapper around grep(1) very fast and useful:
Is it enabled when you specify a directory (rg somestring .) ?
There does not appear be a popular indexed full-text search tool in existence.
Think cross-platform version of Spotlight's mdfind. Could there be something fundamental that makes this approach unsuitable for code search?
Alternatively, something like locate, but realtime and fulltext, instead of filename only.
Warning - Conditional always returns true.
Oh well. Waste of time then.
2. Pcre is good regexp flavor to master. It is have good balance of speed, power and popularity. In addition to Ag, there are accessible libraries in many languages, including python.
I think it would be good if everyone settled on Pcre, rather than each language thinking they will do regexps better.
... $ rg -uu foobar # similar to `grep -r` $ rg -uuu foobar # similar to `grep -a -r`