http://weavesilk.com is a side project of mine for many years. I put out a brand new version of the iOS app this month and am thinking about developing it further but find it very hard to decide what direction to go.
The website is popular, and people love Silk, but for different reasons: some find it relaxing and meditative, others like that it closes the gap between their artistic ability and taste.
It's been used as an inspirational sketching tool for artists (http://bit.ly/1Qlm1kA), to make album art, and has been on exhibit at the Children's Creativity Museum. Some teachers use it to teach kids about symmetry.
I've made something compelling but don't know what to do next, or how to figure it out.
First, what are your thoughts on this market?
Also, some advice: I'm currently building the prototype based on my own experience as I'm my own user. That's the only thing I've been focusing on, I haven't looked for external feedback yet and haven't spent much time looking for people to join me. I figured both of those things will be much easier once I have the prototype ready. Am I on the right track or should I be doing everything at once?
I am the founder of http://www.pnyxter.com - a video (only) based debate and discussion app.
My question is about product-market fit.
A user can create topic and upload a video selfie talking on the topic or respond to other topics via video selfie. The app does not have provision of text comments at all - only video responses.
I've invited several friends and family for private beta, and its been 10 days now and only 1 of my friend dared to create a video.
I also shared the link on FB and linked in and got several views, but no one created a topic. I also did a FB ad for 2 days - no luck here too. Is this a clear indication of product - market misfit? Or is it too early to conclude?
Or should I shift my focus only on professional and amateur debaters of various debate clubs in cities, universities, schools etc.
I do understand the privacy issues of showing video selfie - but I've given provision of a good privacy control (or at least I think its good).
Video based opinions are already being posted by users in Facebook and youtube - but do not have this consolidated grouping of all video discussions on a topic in one place. Youtube tried video response feature and closed it in 2013 due to low engagement - but of course youtube is a very generic video site - not all videos needs video responses/discusisons/debates.
What's the best way to grow your early user base? Ideally we'd like people who are going to use BugReplay at their jobs and open source endeavors, but we want to keep it relatively small until we feel like it's ready for the widest possible audience.
I've built a prototype for myself, but would like to now connect with the right industry people to help guide the product and be trusted early users. In another reply you mentioned finding users to serve as product advisors similarly (https://news.ycombinator.com/item?id=11183572) . I'm having a hard time discovering who the "right" people are and reaching them in a compelling way. If I were further along [read: post private beta] I would reach out to someone like Phin Barnes @ First Round, but I feel at this stage it's too early for him to really give interest.
We've talked to many startup employees and they all have given positive feedback. One concern that comes up is that startups are not OK with changing the cap table for tiny transactions. In that case, we have plans to offer synthetic liquidity solutions via derivative transactions.
We'd like to hire your thoughts on the problem we are solving and our approach so far.
http://www.stroomnews.com is a bootstrapped breaking news and events focused sharing platform, that allows live-streaming and pre-captured video and image sharing through our mobile app and website. We are also working on an enterprise solution for the news industry that ties into our platform. Our B2C app was released this past Fall, but has shown little traction and we have had discussions with a major news media company about our enterprise product. We believe that signing up users for our enterprise solution will also help grow our B2C platform.
We also have an idea in the area of video compression (both founders have experience in this industry) that we believe could be huge in the streaming video industry. Our initial tests have shown very promising results, but we have not had much time to work on this due to focusing on our platform and enterprise product. The success of this tech does not only provide a large advantage for our business, but opens us up to many other industries and opportunities.
As our ability to bootstrap dwindles (due to amount of savings), do we continue along our path and try to get revenue as soon as we can by working on our enterprise product or do we spend time trying to raise money so we can focus on our tech, which may not produce anything product worthy for a much longer time (or ever as it is still in the research stage)?
Thanks for your time!
I have developed a high-speed image encoder that runs on off-the-shelf graphics cards:
I am working on my marketing strategy: need to decide whether to focus on selling to end-users, or licensing the software to other businesses. Second option requires more $$, and a sales team, but seems to have more potential for growing the business.
Any advice would be greatly appreciated!
I'm a co-founder at https://www.missiveapp.com, a collaborative email client (Slack meets Gmail). We've launched our open beta last January and are actively recruiting beta users.
We're a fully bootstrapped team of 4 working from Quebec city, Canada. We were able to bootstrap Missive with the $ we rake in from another project we launched three years ago called ConferenceBadge.com .
My question is, do you think it's a mistake to run two businesses in parallel?
Right now 85% of our time is invested in Missive even though it's bringing $0 in revenue.
Our philosophy is that if we were to look for funding, we would have to invest at least 15% of our time on fundraising and investor relationships.
We also believe that looking for investment before market fit is a recipe for disaster (need not to forget that we are not from/in the valley).
 What are chat conversations doing in an email client? Here are few examples of cool possibilities they enable: https://www.youtube.com/watch?v=VcRQhGfT620
I'm working on a few different projects all of which I think could become viable companies but having a hard time deciding which to focus on. I've already seen interest from relevant parties in each of the separate projects.
I think the hardest one but also the one with the most growth potential would be a project I'm working on to provide management tools (similar to the stuff a ceo might use) to high level government officials. However, with the way that government contracts are handled I wonder if this is even a reasonable industry to target.
Secondly I have two different projects that focus on College Students, where I would be selling solutions to the Colleges themselves. The first project is an art application that allows users to upload art in any medium and be seen by other students. This would allow them to easily build fan bases by taking advantage of pre-existing school connections.
The other idea is similar but focuses on user generated events. It tackles the question of how does one find interesting things to do, in a new area, when you don't know anyone. And has certain measures in place to help alleviate the awkwardness of trying to join pre-existing groups.
Do you think that marketplaces for services is too mature of a market? I'm working on a site called bid2mow.com to help new lawn care companies find work. It seems like everyone is focused on come to my app/website and I'll get you a price versus an ebay model. I know task rabbit had that model and went away from it. Ebay may not be amazon but it's no business to sneeze at either.
My question for you is: what challenges are there around marketing for security-focused products? Of course, trust is a key factor, but are there other things I should consider? I'm thinking of Twitter ads, but I'm sure there are better options between that and cold emailing. Thanks again!
I am one of the Co-founders of Disco Melee (https://beta.discomelee.com/hub). We are a live streaming social network designed around the needs of gamers. You could think of us like if Twitch and Facebook had a baby who liked to party and was eyeing up Reddit for a future fling.
The gamers that find us tend to rave about our overall vision, low latency and other base features like IM system, streamer storefronts, and free donate/sub buttons for all. The problem however is that (except for our hardcore believers) they don't seem stick around very long due to the pull of network effects from the established players. What strategies could you recommend for overcoming network effects to the point where we can start generating our own? Thanks!
My startup is developing technology for natural language understanding (in contrast to NLP). We believe we can approach human-level understanding for standard texts (email, blogs, online articles) in 2.5-4 years.
We are currently self-funded and can comfortably do so for about a year--by the end of which, we believe we can develop a fairly advanced demo that surpasses existing techs in some, but not all, areas.
What kind of demos do you think would impress best-in-class recruits/investors to join our team? The current options we have in mind are:
1) Solving a subset of Winograd schema (commonsense reasoning challenge) with a general approach (i.e., easily extensible with additional investment in knowledge acquisition/data sources). No systems known to public can currently solve all of them (or even a subset generally).
2) A conversational agent capable of conversing with humans at the level of a 4- or 5-year-old (without resorting to typical chatbot tricks).
3) Surpassing the state-of-the-art systems in a couple of standard AI conference tasks or on datasets released by leading companies (Google, Facebook, etc).
4) Other tasks we have not thought of...
Because of resource and time limitations, we likely need to focus the initial efforts on one, or at most two, of the options above. (A mature system should be able to do all those and beyond, but this is only for about one year from now.)
A couple extra questions if you have the time:
- Given the startup's long-term timescale before monetization and its technical nature, what sorts of investors should we focus on talking with?
- Are there chances of IP leaking and causing problems with patenting our tech later on? What should we do to prevent that?
Sincerely appreciate your time to answer these questions.
-- Ken Noppadon
I'm in the idea stages of creating a website, and I have no previous experience with startups or other business; I'm only a senior in high school. The idea is that there are people who would like to own the same game on multiple platforms, and so the website would offer a discount on a game that you already own, for another platform. So, for example, if you have the Xbox One version of Rocket League, the website would offer a discount on the Steam version.
My question is: how should I gauge interest in such an idea? I don't have any real budget to speak of, and I don't know where I should go looking for people to ask.
I created https://codebunk.com. Its an Online Interviewing Tool for developers. Its the best tool out there. It provides code execution in 23 languages, collaborative editor, REPL shells, AV Text chat, Teams, Question banks and a lot more.
Its cash flow positive and has some of the coolest clients.
However, the rate at which it acquires new customers is pretty low (~3/month). I have exhausted (or nearly exhausted) avenues of generating buzz (PH, HN, Reddit, some tech publications). As a developer without any help, how do I promote CodeBunk further? What's the best way to reach my audience (Hiring Mangers, CTOs)?
I am in the midst of creating a new platform as a service product while working at a large tech company. I am not quite willing to reveal the platform to the world yet, but it is a new take on backend-as-a-service platforms that I think will be very intriguing to developers.
My question is not specific to my product and instead pertains to the situation of trying to develop a startup while working full time at a large company. I have done 0 work on the product on my employers time and am not concerned with that aspect. I am more interested in your thoughts on when would be the ideal time to leave my current post to work full time on the startup? My biggest concern is leaving the financial stability of my current job when I have no capital lined up to support a startup. I have already launched a closed beta of the platform and am getting feedback from a small set of users and plan to open the beta up to the world in the coming months. In your experience have you ever seen VC's or Angel's be willing to make a deal with a startup founded by a full time employee of a different company. If I were able to secure funding, I would be absolutely be willing to move on and work full time on the startup but making that leap without funding would be a difficult decision. Any words of wisdom you can offer here would be greatly appreciated!
Feedback Hotline (feedbackhotline.com) is the easiest way for businesses to collect feedback. We provide the hotline for free. We intend to make money through data.
We think we need to prove three things to succeed:
1. Businesses/organizations will join2. People will send a lot of feedback3. We can monetize this feedback
We are currently optimizing for 1, focusing on small businesses. We believe for i, everything before i needs to be true before i can be true. We want to speak about this framework for optimization.
We're developing a publishing network. Individuals and groups can start magazines. Users can read on a timeline and interact like in a social network. What's new? Now anyone can get publishing infrastructure as good as a well funded online magazine. That means lots of unique publications in verticals that well-funded magazines & newspapers cannot do.
What's the best strategy to lure in people to use the service?How do we make money?
We're a link shortener called Credhot and our business model is to syndicate (potentially sponsored) content on an interstitial page. We also rev-share with our users based on the number of visits. The biggest hurdle to this strategy has been building an interstitial people want to share with their friends. Here is an example of our latest attempt at that, leading to "coinmarketcap.com": https://crd.ht/H7TLMpn.
Our volume is low enough that we haven't tested syndicating sponsored content on that page. Right now we're mostly focused on "building a product that users love" but at some point we will need to strike a deal with a publisher. When should those conversations start happening? We're currently bootstrapping but will want to raise capital in the next few months as we're growing (we've doubled since the last HN office hours ~34 days ago). Should we wait until after securing funding?
I've developed a custom toolchain while writing an electronic book. The toolchain automates the conversion of markdown and media into a scrollable "app-novel". Initially I'd hoped to earn income from the book itself, but the naiveness of that idea is quickly becoming apparent.
As a pivot, I am developing a public interface for the toolchain, with the idea of permitting others to write books in the same style. Unfortunately it is not ready to be demonstrated. Nonetheless, this feels like a untapped industry to me, and I wonder what your opinions or suggestions might be.
WHAT WE DO: www.wiredhere.com
We integrate every social activity (university created or student created) happening within an university into a mobile app. The students can attend and provide feedback through the app; we than take the analytics that's created and provide it to universities so they can assess and compare themselves to other universities.
Do you think it's more optimal approach this as a SAAS for the university since we provide them a brand new web platform to make event creation easier and so they can reach students in a faster way?OR be a non-saas and introduce this to the students first and let the universities catch on afterwards, and than work with the university so they can use our web platform and mobile app?
Also, what is your opinion on our concept?
WordBrewery (http://wordbrewery.com) teaches languages by scraping real sentences from news sites around world, then processing each sentence with an algorithm that estimates (on the basis of word frequency and other variables) how likely the sentence is to be useful to learners at different levels.
We are now a member of 1776 and participating in Microsoft BizSpark, so we are on the right track. But I am new to the startup world, and I am funding the website entirely by myself at this point using money from my day-job paycheck. What is the best way to pursue seed funding at this early stage while we are developing core features? Do I need to organize it as an LLC or corporation to get funding?
I am working on a marketplace through which publishers and journalists can sell the ability to be quoted and linked to in an article to the highest bidder. A realtor, for example, might be willing to pay to be quoted and linked to in an article about how a city's real estate market has been accelerating.
My question is how to get started with marketing it. While the growth hacking crowd would say to simply spam a bunch of reporters to get inventory, I have found in other ventures that people absolutely hate receiving any form of unsolicited email with any kind of pitch.
We have been working on http://growthzilla.com - a data driven growth solution for salons for a little more than a year now. We have paying customers since launch and have zero cancellations. Our customers love the product for 3 reasons (1) ease of use (2) effectiveness in driving growth and (3) customer service
The problem: we are growing slower than we would want. Is there anything you'd like to suggest?
I'm building games and tools for language learning. While talking to users, I asked them what language learning tools they wished existed when they were studying in the past. A lot of them talked about a tool that lets them talk to native speakers of their target language online.
I know I should build something users want, but there are plenty of tools that let you do this (including a YC company, cambly) and I don't want to build another one. So what do I do with their answer to my question?
Problem: in the midst of running everything, I do all the coding slowly by myself, and the urgent coding todo list has exploded while some of our sites age. What should I do?
I'm cofounder of a start up within the beauty space. We have small but active group of users that love creating and consuming a unique type of content only available on our platform. The revenue stream is to eventually sell tangible products so we would like to make use of all this content to convert purchases.
Is there a way we can test out how well the conversion rate could work without shipping tangible products? We don't yet have capital to stockpile.
http://nestify.io founder here. We're improving PHP CMS hosting (WordPress, Drupal) with better scaling, on-page optimizations, better security. We have paying customers and ~100 die-hard users that will be really sad / lose revenues if we shut down.
Should we focus on building our brand while scaling or switch to whitelabel and API services and partner with hosting providers?
As for the shop itself, the photographs need improvement I set up a small lighted space on my kitchen counter and am working on this. Shop: http://zebbles.etsy.com
I'm the founder of WebArcs (http://webarcs.com) an RSS aggregate for discovering and subscribing to websites. I'm just starting out and I want to see this be the way people surf the web in the future.
I was wondering which demographics I ought to target too too help build a strong user base?
The resistance to this is most code is made for $ and the people with $ tend to prefer fungibility. JS devs are easier and cheaper to replace than Purescript ones. From the lens of developers as a list of buzzwords you can to grep this makes sense.
Frameworks get trendy but new languages have a hard time even though the learning curve may be no different.
I just hope somehow better languages than JS will win.
Has it not? A more appropriate question would be "When will it step down?"
> it takes 40% more development time to finish a project with AngularJS than without using it.
And by "without using it", does it mean you write in vanilla JS? I believe you jumped into conclusions way too early. Working with any technology will take long. It's not the actual writing of code that will take long, it's the learning (if you're a total noob), setting up (if you don't have scaffolding tools), and debugging (if you don't have tools). The actual work is just a fraction of what you're actually doing.
> How was your experience with React or any other
If you mean React alone, then it's like Angular with directives... alone. You'll have to debate on your router, your data flow library, your server-communication layer, your build tooling, your process. It's all the same thing under the guise of a different syntax.
> Should I invest more time to keep me updated with one of them?
Get to know all the libraries, but never try to use them all. Choose one and get things done.
> I still think, front-end should just present the information, not the whole application logic.
It's like saying JS should have stayed on the browser, but wait! There's Node.js (server), Espruino (hardware), FirefoxOS (OS), PhoneGap (mobile), NW.js (desktop). If everyone was thinking the way you do, these would have never been invented.
I think you are absolutely right! Progressive enhancement is a foreign concept to some developers. They've forgotten that hyperlinks are the engine of application state.
- I don't know.
I work at a small startup with a roughly 10-person eng team.
When we write docs we focus mainly on architecture and processes. The architecture docs often emerge from a "tech spec" that was written for the development of a feature that required a new service, or substantial changes to a new one. We keep everything in Github, in a README or other markdown files.
We also write API docs for HTTP endpoints. These are written with client developers and their concerns in mind. When doing this for a Rails app we use rspec_API_documentation, which is nice, but it can be annoying to have testing and documentation so tightly couples. We've talked about changing how we do this, but we always have more pressing things to do.
We never write docs for classes or modules within an app/service.
I'm actually pretty proud of the search that I put together for this setup too, it's all done in the browser and the indexes are built at compile time which is then downloaded in full for a search, which sounds silly, but it works surprisingly well .
So we have a doc folder in the repo that is like staring into the maw of Cthulhu and takes up 90% of our build time on the CI server sucking that down mass of garbage for the checkout.
Saner systems have been proposed, but rejected because the powers that be are too averse to change...
- Easy editing (namely markdown files in folders)- Runs on "cheap" hosting/everywhere (built with PHP)- Supports multiple languages (so you can create docs in english, german, etc.)- Can have editable try-on-your-own demos embedded into the documentation- SEO friendly (clean URLs and navigation structure)- Themeable (themes are separated and run with the Twig templating engine)- Works on mobiles out of the box- Supports Plugins/Modules for custom content/behaviour- Formats reference pages for objects/classes/APIs in a nice way- Supports easy embedding of disqus for user feedback- Other stuff I forgot right now
The system powers the knowledge base of my recent app "InSite" for web developers: https://www.insite-feedback.com/en/help
Another instance of docEngine runs for my pet html5 game engine: http://wearekiss.com/gamekitThis one uses the default theme, has most pages in two languages and again incorporates a couple of live demos.
I host a little documentation about the engine itself here, but its not complete right now: http://wearekiss.com/docEngineYou can also find the github link to the project in the footer of every hosted documentation.
Have fun with it - I give it away for free. Critics and comments welcome!Everything I have linked was built by myself.
Trying to get people onto Sphinx , and use it for some non-sanctioned documentation with good success, but unlikely to make it official.
I really think version control is important: what changed, who changed it, provisional changes through branches, and removing the bottleneck of "I updated the docs, please everyone check before release and send me your comments". It should be patches, and only patches.
Besides the README.md to get started, the app defaults to a private portal with a component playground (for React), internal docs (for answering "how do I"), and tools for completely removing the need for doc pages at all.
I believe that documentation has to be part of the workflow, so component documentation should be visible while working on the component, tools for workflow should have introductions and helpful hints rather than being just forms and buttons, etc.
So far, this is proving fruitful.
(Side note: wikis are where docs go to die.)
The important thing about docs is to keep in mind the audience. This is important because it lets you estimate their mental model and omit things that are redundant: for example, if it's internal documentation for a codebase, there is little need to explicitly list out Doxygen or JSDoc style information, because they have access to the damned source code. External audiences may need certain terms clarified, or some things explained more carefully because they can't just read the source.
I'd say that the biggest thing missing in the documentation efforts I've seen fail is the lack of explanation for the overarching vision/cohesive architecture of the work. This is sometimes because there isn't a single vision, or because the person who has the vision gets distracted snarfing on details that are not helpful without a preexisting schema to hang them on when learning. So, always always always have a high-level document that describes the general engineering problem the project solves, the main business constraints on that solution, and a rough sketch of how the problem is solved.
Ideally, the loss of the codebase should be less of a setback than the loss of the doc.
I will say that, as your documentation improves, you will hate your project more and more--this is the nature of the beast as you drag yourself through the broken shards of your teams engineering.
So whenever a new staffer comes along, I get asked to give them wiki access... but I'm the only one here that uses my edits (only ops staffer). Sure, have some wiki access, for all the good it will do you!
I really don't recommend our model :)
Anyway, this is an important point: documentation is not free. It takes time. Even shitty documentation takes time. If you want good documentation, you need to budget time away from other tasks. When I used to work in support, the field repair engineers would budget 30% of their hours for doing paperwork - not documentation specifically, but it clearly shows that 'writing stuff' is not something that springs as a natural/free parallel to other activity.
I believe that literate style of code writing has many benefits in any language.
Basically mix markdown with the codebase and export the documentation from the same file.
For a very well executed and interactive example check out
Which make it easy to create html, pdf, epub, latex formats, etc.
I like to create a user guide, developer guide, and ops guide for each large project.
1) Write to all of your target audience. For example if your product is targeted at both technical and non-technical people, then write the documentation in such a way that non-technical folks can understand it. Don't just write for the technical people.
2) If possible, write documentation around several 'how do I do XYZ task'? My experience has been that people tend to turn to documentation when they want to execute a specific task and they tend to search for those phrases
3) As much as is possible, include examples. This tends to remove ambiguities.
The rest of the systems are documented ad-hoc. Some readme files here and there, a large block of comments inside of confusing files, the occasional style guide, etc.
We also have an onboarding guide for new devs (just a PDF) which walks them through our systems, our tools, etc. Nothing fancy, about 10 pages.
* MS doc(x) on a network folder with an excel spreadsheet to keep track of docs (and a lot of ugly macros).
* MS doc(x) in a badly organized subversion repository (side note here, docs comments and revision mode are heavily used in those contexts, which is really annoying)
* rst + sphinx documentation in a repository to generate various outputs (html, odt, pdf...) depending on the client.
In some cases we also use Mako (a python template engin) before sphinx to instantiate the documentation for a specific platform (ex: Windows, RedHat, Debian...), with just a few "if" conditions (sphinx could do it in theory, but it's quite buggy and limited).
I've also put in place a continuous build system (just an ugly shell script) rebuilding the sphinx html version every commit (it's our "badly implemented readthedocs.org", but it's good enough for our needs).
In other cases we use specification tools like PowerAMC or Eclipse/EMF/CDO based solutions, the specification tool in that case works on a model, and can generate various outputs (docx, pdf, rtf, html...).
At home, for my personal projects, I use rst + sphinx + readthedocs, or if the documentation is simple, just a simple README.md at the root of my repository.
As a personal opinion, I like to keep the documentation close to the code, but not too close.
For example, I find it really annoying when the sole documentation is doxygene (or equivalent), it's necessary to document each public methods/attributes individually, but it's not sufficient, you need to have a "bigger picture documentation" on how stuff works together (software and system architecture) in most cases.
On the other side, keeping the documentation away from the code (in a wiki or worst) doesn't work that well either, it's nearly a guaranty that documentation will be out of date soon, if it's not already the case.
I found having a doc directory in the source code repository a nice middle ground.
I found wikis annoying in most cases, rarely up to date, badly organized and difficult to version coherently and properly (ex: having version of the doc matching the software version).
We also have higher level documentation, which is meant to serve as a sort of conceptual overview of the framework, as well as to show what the framework comes with out of the box. This section is written mostly in kramdown, which gets parsed by jekyll before it's turned into HTML.
We generate the bulk of those manuals based on our object model, which is liberally sprinkled with (text only) descriptions. We've created a simple XML-based authoring framework which allows us to create pretty tidy documentation. Including images, tables, code examples etc.
We convert that XML to Apache FOP. At the end of the process, we're left with a bunch of tidy PDF manuals in a variety of languages.
This is the most important step. If you cannot remember it from a blank slate, then no one can. Keep doing that until you understand the code at first glance. Then your code will be easy for anyone to maintain.
Ideally, I'd love to find a mechanism that:
- provides the OO principles in documents; Encapsulations, Abstraction, Polymorphism, Inheritance . - Accessible & maintainable by non-techies. - Allows scripting (I toyed with PlantUML, but it was a bit rigid).
I have a CVS repository of PDF and Word docs.
The business side uses docx format, so using markdown and generating docx is not really feasible. I have run into issues of people changing the filename and it creating a new entry in the version control. I have a idea I plan to implement to fix this.
What I would really like is some linux system that would make it easy to pull the text out of docx and make it searchable. I would want something that could run on the command line that does not have a ton of dependencies.
As for code, auto generated docs from jsdoc etc. headers are fine but I never use them honestly. I find unit tests to be the ultimate documentation in terms of code level docs.
This seems like something that is a really good idea, but is hard to find any projects for it.
Beautiful documents, but it takes a decent chunk of time to create. We do extract some docs via XML to generate code, somewhat backwards from how most engineers merge docs and code.
I'm trying to gather a community of git supporters to push for git.
However, after three months I still haven't gotten a computer suitable for my job.
So that means allocating placements to be used either for adsense / auction-style ad placements.
* I will look for a regression test suite: how extensive is it? If you don't see any tests, that's a big warning sign.
* If you don't see any documentation, another warning sign: the behavior is not specified and could change.
* Who uses it? If the code is reasonably widely used, that de-risks it for you quite a bit, particularly if the existing uses resemble your intended use. They trod through the code paths before you and uncovered the bugs.
* Lastly: if need be, can I just maintain this myself? How easily forkable is the thing? This point can overcome some other issues. If some code is 95% of the way there, and is organized in a good way that I can take it the last 5%, I might just do it.
I also look at how friendly the project is to pull requests and outside development. If a bug is important to me, I will spend the time to code a fix, but if there's no hope for getting any changes made, that represents a large risk to me. It's not a bad thing if the team in charge pushes back for higher quality code, code style, or solutions practical for wider audiences.
If there's continuous integration in place with automated testing and static code analysis that is fantastic. It's not a deal breaker if it's not, but having it in place is a good sign.
Depending on the project, I may have a look at the source itself. I certainly don't look the source of every application I use.
I have found that online recommendations in developer communities are not always good. More than once I've tried out projects based on people evangelizing them only to find out that they are pretty far from acceptable.
There are differences in how I judge things that will be customer facing or that I'm going to have to support operationally. I'm a lot more critical at that point, but my top 3 points are:
1. Can I put it in a container. 2. Are the developers committed to long term support.3. Will they give me a t-shirt.
I would say that "project is open 6 months" is a pretty poor metric, because 6 months is a lot longer than the handful of days that your custom code will have existed when you add it instead.
Reasons to segregate:
1) The single biggest one is that it firewalls the liability of the businesses from each other. Whether this is important or not for you depends on what the businesses are doing: if it's Regular Internet Stuff then your E&O policy is probably good enough in terms of risk mitigation, but if 1+ of your products are in highly regulated spaces (hello HIPAA, finance, etc) then putting them in their own LLC isn't a crazy solution.
2) If you're religious about doing not just the paper ownership but the business accounts separately for each business, that makes eventually selling or otherwise disposing of them much, much easier. Otherwise you're looking at weeks of work and/or very fun professional services bills when you decide to do the division later.
3) If you have co-founders or investors, or the prospect of getting co-founders or investors, separate legal entities are going to be pretty much required. You don't want them to accidentally get ownership of your side projects; they don't want to own your side projects (ownership is a risk; they know the risks they're signing up for and don't want additional sources of uncontrolled unknown risk).
4) A minor factor, but there is non-zero social friction involved in "We've been talking about my trading name of $FOO but remember that the invoice/contract/etc will be from $BAR, LLC."
Reasons to not segregate:
1) It's a lot of extra work.
2) There's a running cost to keeping an LLC open, both the yearly fees and the operational complexity of maintaining separate books, accounts at various providers, and (if you're doing things in a complicated fashion) keeping up appearances with regards to the LLCs being formally separate from each other.
As an ex-consultant with some accidental knowledge of the payments space: I would be doing double-plus firewalling between any payments startup and anything I'd lose sleep about losing, and I would be happily writing a sizable check right about now to a lawyer rather than taking HN's advice about my compliance obligations and potential sources of risk.
Note that is the 3rd edition. I had read an earlier edition. He lists the differences between 2nd and 3rd.
ISBN-13: 978-0321125217ISBN-10: 0321125215
I needed an easy place to dump everything. As work uses Trello and my startup uses Basecamp.
I have Trello organized with the following boards (from left to right on the screen):
"Thing to Do" - This is really my inbox. While I tend to make the cards in the list they need to go, if I just don't know I place it here. About once a month I go through it to make sure nothing has fallen off my radar that shouldn't or more importantly see if I can just delete it because it doesn't matter.
"Priority Tasks" - These are bigger tasks that I know are things that need to get done as workflow permits.
"Doing" - What I'm currently working on. Usually 5-10 items depending on dependencies.
"Dated Boards" - At the start of each week I create a new board with the title of being just the date. All tasks that I complete that week go onto that board. For really long tasks I may copy a card and keep it in "Doing" but put a copy in that weeks board.
Trello also has card labels, think colored flags to identify things quickly. I always have the following labels:
Red - Critical, e.g. an emergency task that takes priority over all things.Orange - Urgent, can wait, but not long.Yellow - Time sensitive, I don't always use the date feature on Trello cards, so I use the yellow label for things that need to be done by a set dateBlue - Big tasks or Big wins. I want to be able to find in past week when I had a big win.Green - Interdepartmental dependency, either someone else needs this from me or I need something from someone else.
At my current job I can see back nearly three years what I did week to week. I can quickly search to see when something was done, or browse it. But more importantly it's easy to make a Trello card and once it's in Trello I can easily organize my time and tasks.
Then I realized that the process of breaking down workload and personal life into small tasks does not fit well in a to-do list app context.
What worked for me was using something like BusyCal, which is a souped-up calendar. It was more aligned with the 'breaking down' process because I used the 'banner' feature to help me define my weekly goals then I proceed with tasks assigned with specific chunks of time.
I then track the planned vs actual time spent using letsfreckle.com, which is a time tracking tool.
For the more higher-level stuff (ie. monthly goals, yearly goals), I just keep them in Trello.
As for habits, I have a weekly review to check where I am vs monthly goals and I also use that review to allocate tasks into my calendar.
I use Google Keep for simple todo lists like grocery or things I have to do for short duration.
For bigger long term projects I like Trello
Title: Freelance Software Developer
Years Exp: ~12
Been doing freelance development for last 5+ years. Enjoyed the flexibility and the freedom of being an indy dev. Started off with aspirations of building my own SaaS on the side and bootstrapping with consulting, but got sucked in by nice hourly rates and cushy lifestyle. Cant complain too much though! Getting the itch to do something new now, though. Trying to decide between
1. Trying another startup as a founder or very near to it - before freelancing I worked in a few startups, one was successful (as an employee).
2. Re-joining corporate america to see if I can make a big impact with a relatively high and predictable income + benefits. Eyeing health care IT as one potential area of impact where theres seemingly lots of opportunity and change right now due to health care reform (and its a distinct area of interest beyond coding).
Next Move: Building another software startup. Go big. Apply the lessons learned from my previous company, while trying something in a new space. Get back to coding every day after being a owner/manager for years. Find other awesome people to work with, as in my experience this is the single biggest factor in happiness at work.
Years SE Experience: 7
Next Move: Use my extensive experience in conversion optimization and web performance optimization to increase profits for clients. This cash will fund the next phase of my open source projects.
Title: ML postdoc, formerly software engineer
Years Exp: >5, depends how you count
Next move: Finding a job in industry (tech or finance in NYC or Bay area), solving hard problems using ML, so I can wrestle with the world instead of with reviewers, and be part of a team again.
My father was a software engineer at DEC in the 70's, so I learned to program at 5 on a kit computer. I've been a hobbyist programmer all my life, but I got pulled into teaching. I've been teaching 6-12 grade math and science for 20 years.
Over the last four years I've started to build a second career in the programming world. I have a couple open source projects that have shown promise, and I recently published an introductory Python book that's doing pretty well.
Next Move: I'm excited about the possibilities, all of which are pretty appealing.
- Write more.
- Pick up development on two main open projects:
- introtopython.org | An open introduction to Python based on jupyter notebooks. Anyone familiar with notebooks can contribute a project.
- opencompetencies.org | An open platform for building education standards.
- Switch to teaching CS full time instead of math and science.
- Pick up more freelancing work.
Title: Software Engineer
Years SE Experience: 2-3 years professionally
Next Move: I'm not too sure. I am keeping an eye on the job market to see if anything comes along that I feel would be a great move. I'd love to start a company but I don't think I'll do it yet.
I've been programming since I was 12, so it's fun helping my friends through our CS classes.
My next move is to find a job... anywhere.
Title: Lead Consultant (Mammoth Data) / Founder/CEO (Fogbeam labs)
Years SE Experience: ~20
Next Move: (do X so that I can Y)
Continue building Fogbeam Labs so that I can eventually make that my full-time thing, hire employees, and build a real company. As part of that, we just started working on a new product that I'm really excited about. I don't want to say too much about it just yet, but it's going to be a gnarly Machine Learning / AI project.
Title: Lead Engineer
Years SE Experience: ~12
Next move: Stop coding within a year. The gameplan is to lay solid technical foundations for our MVPs, then assemble a team of devs who are better than me, and get out of their way. I'm not going to retire by any means, I'd just like to move to a management/executive role.
* Don't use queues. Use logs, such as Apache Kafka for example. It is unlikely to lose any data, and in case of some failure, the log with transactions is still there for some time. Also Kafka guarantees order of messages, which might be important (or not).
* Understand what is the nature of data and what are the queries that are made later. This is crucial for properly modeling the storage system.
* Be careful with the noSQL cool-aid. If mature databases, such as postgreSQL can't handle the load, choose some NoSQL, but be careful. I would suggest HBase, but your mileage may vary.
* NoSQL DBs typically limits queries that you might issue, so the modelling part is very important.
* Don't index data that you don't need to query later.
* If your schema is relational, consider de-normalization steps. Sometimes it is better to replicate some data, than to keep relational schema and make huge joins across tables.
* Don't use MongoDB
I hope it helps!
Most of the big data tools out there will work with data in this format -- BigQuery, Redshift, EMR. EMR can do batch processing against this data directly from s3 -- but may not be suitable for anything other than batch processing. BigQuery and/or Redshift are more targeted towards analytics workloads, but you could use them to saw the data into another system that you use for OLAP -- MySQL or Postgres probably.
BigQuery has a nice interface and it's a better hosted service than Redshift IMO. If you like that product, you can do streaming inserts in parallel to your gcs/s3 uploading process for more real-time access to the data. The web interface is not bad for casual exploration of terabytes of raw data. And the price isn't terrible either.
I've done some consulting in this space -- feel free to reach out if you'd like some free advice.
My advice is to step away from AWS (because of price as you noted). Bare metal servers are the best startup friend for large data in regards to performance and storage. This way you avoid virtualized CPU or distributed file systems that are more of a bottleneck than advantage.
Look for GorillaServers at https://www.gorillaservers.com/
You get 40Tb storage with 8~16 cores per server, along with 30Tb of bandwidth included for roughly 200 USD/month.
This should remove the IOPS limitation and provide enough working space to transform the data. Hope this helps.
* Use a Queue. RabbitMQ is quite good. Instead of writing to files, generate data/tasks on the queue and have them consumed by more than one client. The clients should handle inserting the data to the database. You can control the pipe by the number of clients you have consuming tasks, and/or by rate limiting them. Break those queue consuming clients to small pieces. Its ok to queue item B on the queue while processing item A.
* If you data is more fluid and changing all the time, and/or if it comes in JSON serializable format, consider switching to postgresql ^9.4, and use the JSONB columns to store this data. You can index/query those columns and performance wise its on par (or surpasses) MongoDB.
* Avoid AWS at this stage. like commented by someone here - bare metal is a better friend to you. You'll also know exactly how much you're paying each month. no surprises. I can't recommend Softlayer enough.
* Don't over complicate things. If you can think of a simple solution to something - its preferable than the complicated solution you might have had before.
* If you're going the queue route suggested above, you can pre-process the data while you get it in. If its going to be placed into buckets, do it then, if its normalised - do it then. The tasks on the queue should be atomic and idempotent. You can use something like memcached if you need your clients to communicate between eachother (like checking if a queue item is not already processed by another consumer and thus is locked).
Have you looked at Google at all? Cloud Bigtable runs the whole of Google's Advertising Business and could scale per your requirements.
In my previous job we processed 100s of millions of row updates daily on a table with much contention and ~200G size and used a single PostgreSQL server with (now somewhat obsoleted by modern PCIe SSDs) TMS RamSAN storage, i.e. Fibre-Channel based Flash. We had some performance bottlenecks due to many indexes, triggers etc. but overall, live query performance was very good.
Realistically, this is what I would do (I work on something very similar but not really in adtech space):
1. Load data in text form (assuming it sits in S3) inside hadoop (EMR/Spark)
2. Generate reports you need based on your data and cache them in mysql RDS.
3. Serve the pre-generated reports to your user. You can get creative here and generate bucketed reports where user will fill its more "interactive". This approach will take you a long way and when you have time/money/people, maybe you can try getting fancier and better.
Getting fancy: If you truly want near-real time querying capabilities I would looks at apache kylin or linkedin pinot. But I would stay away from those for now.
Bigtable: As someone pointed out, bigtable is good solution (although I haven't used it) but since you are on AWS ecosystem, I would stick there.
(Hope that might be helpful! A bunch of us hang out on IRC at #cassandra if you're curious.)
If your data are event data, e.g. User activity, clicks, etc, these are non-volatile data which should preserve as-is and you want to enrich them later on for analysis.
You can store these flat files in S3 and use EMR (Hive, Spark) to process them and store it in Redshift. If your files are character delimited files, you can easily create a table definition with Hive/Spark and query it as if it is a RDBMS. You can process your files in EMR using spot instances and it can be as cheap as less than a dollar per hour.
And pay a little to read this book: http://www.amazon.com/Designing-Data-Intensive-Applications-...
And this one: http://www.amazon.com/Big-Data-Principles-practices-scalable...
Nathan Marz brought Apache Storm to the world, and Martin Kleppmann is pretty well known for his work on Kafka.
Both are very good books on building scalable data processing systems.
This architecture will get you to massive, massive scale and is pretty resilient to spikes in traffic because of the Kafka buffer. I would avoid Mongo / mysql like the plague in this case, a lot of designs focus on the real time aspect for a lot of data like this, but if you take a hard look at what you really need, its batch map reduce on a massive scale and a dependable schedule with linear growth metrics. With an architecture like this deployed to AWS EMR (or even kinesis / s3 / EMR) you could grow for years. Forget about the trendy systems, and go for the dependable tool chains for big data.
We looked at just about every open source and commercial platform that we might use as a backend, and decided that none were appropriate, for scale, maturity, or fairness / scheduling. So we ended up building, from scratch, something that looks a bit like Google's Dremel / BigQuery, but runs on our own bare metal infrastructure. And then we put postgres on top of that using Foreign Data Wrappers so we could write standard SQL queries against it.
Some blog posts about the nuts and bolts you might find interesting:
If we were starting today, we might consider Apache Drill, although I haven't looked at the maturity and stability of that project recently.
If you just want reports (and are okay getting them in the matter of minutes), then you can continue storing them in flat files and using apache HIVE/PIG-equivalent software (or whatever equivalent is hot right now, im out of date on this class of software).
If you want a really good out-of-box solution for storage + data processing, google cloud products might be a really good bet.
DO NOT write to txt files and read them again. This is unnecessary disk IO and you will run into a lot of problems later on. Instead, have an agent which writes into Kafka (like everyone mentioned), preferably using protobuff.
Then have an aggregator which does the data extraction and analysis and puts them in some sort of storage. You can browse this thread to look for and decide what sort of storage is suitable for you.
The bottleneck is usually not I/O, but computing aggregates over data that continuously gets updated. This is quite CPU intensive even for smaller data sizes.
You might want to consider PostgreSQL, with Citus to shard tables and parallelise queries across many PostgreSQL servers. There's another big advertising platform that I helped move from MySQL to PostgreSQL+Citus recently and they're pretty happy with it. They ingest several TB of data per day and a dashboard runs group-by queries, with 99.5% of queries taking under 1 second. The data are also rolled up into daily aggregates inside the database.
There are inherent limitations to any distributed database. That's why there are so many. In Citus, not every SQL query works on distributed tables, but since every server is PostgreSQL 9.5, you do have a lot of possibilities.
Looking at your username, are you based in the Netherlands by any chance? :)
- How CloudFlare uses Citus: https://blog.cloudflare.com/scaling-out-postgresql-for-cloud...
- Overview of Citus: https://citus-conferences.s3.amazonaws.com/pgconf.ru-2016/Ci...
- Documentation: https://www.citusdata.com/documentation/citusdb-documentatio...
To save IOPS on the early part of the process, consider using fast encryption (lz4 or snappy) to compress the records before writing to the file system. This might cut your IOPS in half.
If you need to generate rich multi-dimension reports I recommend you create ETL pipeline into star-like sharded database (ala OLAP).
Dimensions normalization sometime dramatically reduce data volume, most of dimensions even can fit into RAM.
Actually 200Gb per day not so much in terms of throughput, you can manage it pretty well on PostgreSQL cluster (with help of pg_proxy). I think mySQL will also work OK.
Dedicated hardware will be cheaper then AWS RDS.
* Logs are written to S3 (either ELB does this automatically, or you put them there)
* S3 can put a message into an SQS queue when a log file is added
* A "worker" (written in language of your choice running on EC2 or Lambda) pops the message off the queue, downloads the log, and "reduces" it into grouped counts. In this case a large log file would be "reduced" to lines where each line is [date, referrer domain, action, count] (e.g. [['2016-02-24', 'news.ycombinator.com', 'impression', 500], ['2016-02-24', 'news.ycombinator.com', 'conversion', 20], ...]
* The reduction can either be persisted in a db that can handle further analysis or you reduce further first.
In a shell (modified for speed and ease of use) get, insert, update data in a simple way, without all the fat from other mainstream (Java) solutions.
Tip: focus on how to backup and restore first, the rest will be easy!
AWS also has Kinesis, which is deliberately intended to be a sort of event drain. Under the hood it uses S3 and they publish an API and an agent that handles all the retry / checkpoint logic for you.
Does the business really need exactly this? What is their actual goal? Are they aware of the effort and resources required to get this report?
If it's modelled in SQL, it's probably relational and normalized so you'll be joining together tables. This balloons the complexity of querying the data pretty fast. Denormalizing data simplifies the problem so see if you can get it into a K/V instead of or relational database. Not saying relational isn't a fine solution - even if you keep it in mysql, denormalizing will benefit the complexity of querying it.
Once you determine if you can denormalize, you can look at sharding the data so instead of having the data in one place, you have it in many places and the key of the record determines where to store and retrieve the data. Now you have the ability to scale your data horizontally across instances to divide the problem's complexity by n where n is the number of nodes.
Unfortunately the network is not reliable so you suddenly have to worry about CAP theorem and what happens when nodes become unavailable so you'll start looking at replication and consistency across nodes and need to figure out with your problem domain what you can tolerate. Eg bank accounts have different consistency requirements than social media data where a stale read isn't a big deal.
Aphyr's Call Me Maybe series has reviews of many datastores in pathological conditions so read about your choice there before you go all in (assuming you do want to look at different stores.) Dynamo style DB's like riak are what I think of immediately but read around - this guy is a wizard. https://aphyr.com/tags/Jepsen
AWS has a notorious network so really think about those failure scenarios. Yes it's hard and the network isn't reliable. Dynamo DBs are cool though and fit the big problems you're looking at if you want to load and query it.
If you want to work with the data, the Apache Spark is worth looking at. You mention mapreduce for instance. Spark is quick.
It's sort of hard because there isn't a lot of information about the problem domain so I can only shoot in the dark. If you have strong consistency needs or need to worry more about concurrent state across data that's a different problem than processing one record at a time without a need for consistent view of the data as a whole. The latter you can just process the data via workers.
But think Sharding to divide the problem across nodes, Denormalization eg via Key/Value lookup for simple runtime complexity. But start where you are - look at your database and make sure it's very well tuned for the queries you're making.
Do you even need to load it into a db? You could distribute the load across clusters of workers if you have some source that you can stream the data from. Then you don't have to load and then query the data. Depends heavily on your domain problem. Good luck. I can email you to discuss if you want - I just don't want to post my email here. Data isn't so much where I hand out as much as processing lots of things concurrently in distributed systems is so others may have better ideas who have gone through similar items.
There are some cool papers like the Amazon Dynamo paper and I read the Google Spanner paper the other day (more globally oriented and around locking and consistency items). You can see how some of the big companies are formalizing thinking by reading some of the papers in that space. Then there are implementations you can actually use but you need to understand them a bit first I think.
Short answer: I think you're looking in the wrong direction, this problem isn't solved by a database but a full data processing system like Hadoop, Spark, Flink (my pick), or Google Cloud's dataflow. I don't know what kind of stack you guys are using (imo the solution to this problem is best made leveraging java) but I would say that you could benefit a lot from either using the hadoop ecosystem or using google cloud's ecosystem. Since you say that you are not experienced with that volume of data, I recommend you go with google cloud's ecosystem specifically look at google dataflow which supports autoscaling.
Long answer: To answer your question more directly, you have a bunch of data arriving that needs to be processed and stored every X minutes and needs to be available to be interactively analyzed or processed later in a report. This is a common task and is exactly why the hadoop ecosystem is so big right now.
The 'easy' way to solve this problem is by using google dataflow which is a stream processing abstraction over the google cloud that will let you set your X minute window (or more complex windowing) and automatically scale your compute servers (and pay only for what you use, not what you reserve). For interactive queries they offer google bigquery, a robust SQL based column database that lets you query your data in seconds and only charges you based on the columns you queried (if your data set is 1TB but the columns used in your query are only some short strings they might only charge you for querying 5GB). As a replacement for your mysql problems they also offer managed mysql instances and their own google bigtable which has many other useful features. Did I mention these services are integrated into an interactive ipython notebook style interface called Datalab and fully integrated with your dataflow code?
This is all might get a little expensive though (in terms of your cloud bill), the other solution is to do some harder work involving the hadoop ecosystem. The problem of processing data every X minutes is called windowing in stream processing. Your problems are solved by using Apache Flink, a relatively easy and fast stream processing system that makes it easy to set up clusters as you scale your data processing. Flink will help you with your report generation and make it easy to handle processing this streaming data in a fast, robust, and fault tolerant (that's a lot of buzz words) fashion.
Please take a look at the flink programming guide or the data-artisans training sessions on this topic. Note that the problem of doing SQL queries using flink is not solved (yet) this feature is planned to be released this year. However, flink will solve all your data processing problems in terms of the cross table reports and preprocessing for storage in a relational database or distributed filesystem.
For storing this data and making it available you need to use something fast but just as robust as mysql, the 'correct' solution at this time if you are not using all the columns of your table is using a columnar solution. From googles cloud you have bigquery, from the open source ecosystem you have drill, kudu, parquet, impala and many many more. You can also try using postgres or rethinkdb for a full relational solution or HDFS/QFS + ignite + flink from the hadoop ecosystem.
For the problem of interactively working with your data, try using Apache Zeppelin (free, scala required I think) or Databricks (paid but with lots of features, spark only i think). Or take the results of your query from flink or similar and interactively analyze those using jupyter/ipython(the solution I use).
The short answer is, dust off your old java textbooks. If you don't have a java dev on your team and aren't planning on hiring one, the google dataflow solution is way easier and cheaper in terms of engineering. If you help I do need an internship ;)
If you want to look at all the possible solutions from the hadoop ecosystem look at:https://hadoopecosystemtable.github.io/
For google cloud ecosystem it's all there on their website.
Oops, it seems I left out ingestion, I would use kafka or spring reactor.
P.S The flink mailing list is very friendly, try asking this question there.
Even non-meta questions can be rather lazy...I mean a couple of throw away sentences that don't provide much context suggest that it's probably not that important.
For example: https://news.ycombinator.com/item?id=11160872
While I don't think of "Ask HN" as StackOverflow, there's something to the response "What <code> have you tried?" and the idea that a two sentence question doesn't necessarily deserve a long detailed comprehensive answer.
so I guess what I"m saying is that I'm not particularly surprised by its speed, because a lot of the stories submitted to deserve to decline quickly
It's not exactly that but https://news.ycombinator.com/ask exists.
Edit: Too bad you can't use asterisks in HN comments...
They got me on to monitoring EVERYTHING with statsd. Great stuff.
It actually tests your API over HTTP, so you can tell it the url (localhost or production) of any api you wanna test. Use it in conjunction with any modern test framework and you'll likely get the request-times/performance metrics echo'd as part of the runner's output.
It lets you generate end-to-end tests from a spec file for APIs that are themselves generated from the same file. Same for client libraries and API docs, generate all the things :)
Why don't these meet expectations?
For Ruby/Python/PHP/node you can use https://gemnasium.com/
I thought about creating something similar for Perl once. I'm sure there's still space in the market.
It's an incredibly sparse list though, given GitHub's popularity. Compare this Atlassian's marketplace
which seems to include everybody, both big and small.
The most valuable thing Github has is the enormous portfolio of FOSS projects and all the free eyeballs that come with it. Yes, it's not paid, but because its public offerings are so popular, most developers are familiar with it and that gives it a trusted inlet to basically every software organization in existence. After all, in a non-dysfunctional organization, the guy who decides what products to buy to support their developers should give far more weight to what those developers want, than what is said by a vendor salesman whose job it is to convince them to buy a particular product.
Which means it's a mistake to neglect their FOSS users -- it seems self-evident that the biggest, best, most cost-effective way for Github to sell its paid features is word-of-mouth from its free users. It's very hard for a competitor to replicate their enormous network effect, which is also what makes it so effective. Their underwhelming response to the "dear github" letter suggests that their upper management is blind to this. I think there's a serious possibility that, in the next five to ten years, their position will be as marginalized as, say, Sourceforge is today -- long ago it was the "gold standard" in hosting services, but today it's only used by barely-maintained projects who can't even scrape together the resources needed to change their code hosting provider.
- Cofounders should have a roughly even equity split, with the min being no more than 10% less than the max (unless there is a significant factor like one founder investing 100k+ into the business at conception). This prevents resentment and keeps the conversations equal.
- You will have to get used to being in the press, it is necessary to be successful. If this makes you uncomfortable, then have the non-tech founder be the face of the company. But your name and company will be in the press.
- The non-tech stuff is his side of the business. 99% of the time it is best to let him handle this. You need to divide responsibility and then let go.
- If your company relates to the plight of the homeless, this seems fine even if it is a bit early. Otherwise this seems like a distraction / gimic to gain publicity in a "look at what crazy startup founders have to do" kinda way.
- Stop caring what other people think. If / when you're successful you'll get even more people who think you suck and talk shit about you, your product, etc.
You are simply not emotionally mature enough to be working at a startup.
The first thing that stood out to me is the judgmentality over such trivial matters.
Saying things like "i associate homelessness with lack of skills and i want to keep my identity small. i'm known for being well read" really reminds me of when I was really arrogant in high school and felt the need to maintain an internal (and wholly undeserved) sense of moral/intellectual superiority over my more popular (not unsurprisingly) classmates.
And then worrying about how your friends might perceive you because you're associated with him. These are not the signs of legitimate concern. These are the signs of insecurity. Similar to your fear that your parents would kick you out of the house if they read this post. This is not the sign of someone emotionally mature enough to be working full or part time at a startup.
Now, you could be totally right about there being too much focus on media attention. But a socially mature adult (and business partner) would be comfortable bringing those concerns directly to their cofounder, not to an internet forum where they can let their ruminations run wild.
The fact that you've also accepted an arrangement that is an absolutely terrible deal financially indicates a certain sense of naivete about employment, how startups work, and finances. This is not how unpaid internships work. The only reason to ever take an unpaid internship is if 1) it's at a highly prestigious company, or 2) it's in an industry where there are hundreds of overqualified applicants for every position.
Either get a proper paid tech internship so you can actually learn what it feels like to have a real job at a real company with real mentorship, surrounded by people smarter than you. Or take classes and study hard so you can get into a good school.
As an equity partner in this start-up you now have a fiduciary interest in the Branding/Marketing/Public Image of the company.
You didn't mention your market/client space. Does the press attention only serve your older partner's ego or does it help convey your company's story to your market?
Before confronting your cofounder-- first seek to understand. Get the specific details of his media plans. Then consult with 1-2 branding experts, find one on Hourly Nerd > https://hourlynerd.com/your-matches/computer-software/grow-b...
A good chat with these guys should be the perfect sounding board whether or not the 'rags to riches' stunt is a sound idea.
Laughable equity, co-founder you can't respect, already built-up resentment - can only get worse.
Fellow longtime Gentoo and recent Alpine user here. I haven't encountered undue conflict burden from configuration file updates. Some projects do churn whitespace etc, in configuration defaults files, which is unfortunate but not specific to any distro.
If an application supports a conf.d style override, I use that, containing only settings which differ from default.
Is there something inherent about Alpine packaging that handles local config differently?
Stop hanging out with this loser, build up your technical skills so you have a track record of projects built from scratch that you can show people, and then partner with someone who'll actually value you like a partner.
3% though? Not in bounds. If you're the technical guy there on day one and you don't receive a market salary or reasonable facsimile thereof every two weeks, you're a technical co-founder whether you want to be or not. Your deal is exploitative. You will likely not successfully negotiate a non-exploitative deal. (Presumption should be 33%, not 30%, and very definitely not 10%. No difference between the cofounders is meaningful as T approaches 5 years from now; if there is a meaningful difference, that person probably shouldn't be a co-founder.)
Everyone in this conversation is a businessman. They've underbid for your services. I strongly advise turning in your two week's notice and washing your hands of this. Your equity, whether or not you've actually been issued it, is equity in a company run by operators who are not dealing fairly with their technical co-founder and who are either comfortable with lying or clueless. No fact which I've just recited would cause me to think "This equity is worth more than typical startup equity", which is worth nothing.
>> He tells me I'm too "9 to 5" security focused
This is a common line. It is just that: a line. It gets cynically deployed against engineers on a regular basis by people who are not willing or able to pay market wages. You should interpret this line as nothing other than "I am unwilling or unable to pay market wages" and act accordingly.
>> He keeps telling me that I need to trust him and that he'll reward people down the road based on their performance.
99% of the people told variants of this line will be screwed by the people offering this term. As a direct consequence, entrepreneurs who are presently cash-poor but who want to incentivize employees/partners/etc do not say this; they say variants of "Here's a third of the company" or "I will commit to consequential cash payments in writing contingent on us achieving milestones together."
"I will pay you a number amenable to me, at a time of my choosing, if and only if I feel like paying you" is not an offer.
I will close with the observation that now is among the best times in the history of ever to be an engineer capable of shipping projects. You have better options. Exploring them for two weeks ROFLstomps the value of continuing to work for this company.
Update: yesterday, CEO tells me he will offer the guy who started 2 weeks ago the same equity I have, because he's done "such a good job". He has been kicking it hard, it's true, and he does really well, but he's fresh, and the CEO's perspective is so skewed based on what's bright and shiny in front of him at the moment that he totally disregards the contribution of people working for him 7 months for free who got him to where he's at now.
The sad part is I think he knows this damn well, and his strategy well may be to operate a company on a string of trusting people who he will continue to cycle in as the disenchanted ones before them leave.
I heard, and believed, that for years. I got nothing.
It's human nature to trust people, until proven otherwise.
Your CEO might even have magnetic charm on par with Richard Branson or Steve Jobs. Their gift for words hit the right buttons. Who wouldn't want to go into partnership with those guys? Time reveals if they actually make good and deliver.
We should all think in terms of highest & best use of our time. Relative to proving your worth-- imagine what this position might have yielded had you been paid at contractor rates. You've invested your time in the CEO. Has he proven his worth back to you?
Good luck, I hope everything works out.
Here's to hoping what ever decision you make works out.
Rather than 64 bit and ECC RAM, you could have high redundancy on the module level. AFAIK Google do not use server grade systems, just lots of them in a failure tolerant configuration.
An important part of the goal is to get an as "closed computational environment" as possible, where risk for BIOS/firmware infection by hacker is minimal.
So just CPU, ECC RAM, ethernet, and microSD (or USB) to boot off.
Online resources, there are many. Too many sometimes. Grab anything from codecadeamy, codeschool, or Tree House. But remember that's not the only thing you need to know.
If you check my profile, we do remote programming courses where people work together with a real teacher for 6 weeks. We offer scholarships (100% free).
Remember:* Focus on learning programming. What's the scope of a variable? what's immutability? etc.* Practice a lot. Code as much as you can.* Look up for a group to work together.
If you get stuck, ask questions on Stack Overflow or Reddit. Good luck
3. Read about programming in general, because the principles apply to any programming language. I'd recommend Code Complete by Steve McConnell. It's a bit daunting but full of great stuff to make you think about your approach to programming.
These books help as well.
Stord.io is a key/value store. This is often modelled as a hashmap or a dictionary in programming languages.
Under the hood, stord.io is powered by Redis, with a thin python application wrapper, based on Flask.Stord.io doesnt assume anything about your data, make whatever nested schema you want!
Full disclosure: if this wasn't already clear, it's my project. I would LOVE feedback/feature suggestions.
I think it would be safe to assume there are also collision problems in unauthenticated ones as well...
What are the key/value durability requirements? (OK to drop values now and then, or does it need to keep them until the end of time?). Need backups? Do values expire, or do you have to expire them manually? Since you can't enumerate or search, how do you delete things? Allowed sizes of keys and values, between bytes and terabytes? How far should it scale? Shared namespace, or namespace per user? Do you need a latency guarantee? How low? Are you gonna use it for something important and need an SLA on the availability of the service as a whole?
A couple of "nearby" points in the solution space:
Amazon S3 is a KV store where the keys look like filenames and the values look like files. High durability, good scaling, pretty high latency. You could also obviously paper a KV store on top of ElastiCache or DynamoDB, which are going to have different properties.
Going low-level and implementing your own in say, golang would probably be the most fun though :p
Hard to say if we could use a SAAS KV store at work without a lot more technical detail on the solution. I'm having a hard time thinking of an app where you'd want a KV store, but not need a database or NoSQL store which you could use instead.
A DIY sensor needs to either be some kind of test strip or a needle. Both are a lot easier to just get through insurance than to mimic.
I'm sure you could perform the chemical reactions yourself, but you'll probably find whatever you make will need to be replaced frequently. Meanwhile, the software, modified Android phones, etc. last a long longer, so that's where is the action.
The challenge for the homebrewer is to build something traceable. When you measure blood sugar today the same sample should yield the same number - not only today but also ten years from now. Yes, Theranos is fighting with traceability, too.