Ask HN: What's a side project you built to make money that hasn't?

Long story short: I pioneered Deep Fakes 16+ years ago, but no body believed what I was demonstrating was possible, or they wanted to pursue porn, which I refused for multiple reasons.

More of the story: Back in 2001 I was a very burned out video game developer. Back in '93 I was a OS developer on the 3D0 and then the original PSX. By '01 I'd lived through the EA Spouse era, wanting out of games, I transitioned to VFX. While being a hybrid developer / digital artist, I did some actor replacement work that was generalizable - meaning with some preparation, anyone could be inserted into a prepared video clip, with feature film quality.

At this time, about 2002, I realized that short form video, such as conventional TV commercials and pop music videos, treated with consumer faces and actor replacement, become a type of Personalized Advertising. Around 2002, digital cable and streaming video services were just starting to take hold, which meant individual consumers receiving digital video streams receive unique-to-them delivered streams - streams that could have Personalized Advertising in place of ordinary ads. In some cases, this would be very desirable, such as film trailers, or really any desirable wish fulfilling product or service.

So I got serious. I started an EMBA program while continuing at the VFX studio, making the actor replacement project my masters thesis. During this I recruited a team of other VFX people, 2 of which have Academy Awards for their work. By '06 I graduated my MBA program, 2nd in my class, and left working at the VFX studio to work on "the company" full time.

By '08 we had awarded global patents for Personalized Media: a combination actor replacement and product placement process, designed for global scale. We also had an operating VFX pipeline. But, this was 2008: we were in the midst of the global financial meltdown. From '08 through '11 I met with everybody and anybody, presenting and pitching to film studios, recording labels and major advertising firms. I was often met with disbelief the technology was possible or scalable or economically viable. If given the attention, I could explain and demonstrate, but that rarely happened. It was not uncommon for the pitch to receive a rude response, like the idea was crackpot.

Three times I managed to put together an investment pool, but each time one of the investors would realize what the tech could do with porn, and they'd fixate on that. They'd sway the other investors, and no matter how I explained the difficulty of controlling a system that let's anyone put anyone into porn, they insisted that become the direction of the company. I refused, and three times the investment pool disbanded.

After '11 I pivoted to a 3D geometry generator for digital artists and games. The twitter site for that is still at https://twitter.com/3DAvatarStore/media?lang=en. That managed to squeak out an meager existence until '15. Never really generating traction, as people simply expected everything free, and these were professional products.

Simply to eat I'd been doing MBA and developer consulting. Over time, that became the majority of what I was doing, and the freaking global patents were $40K a year to maintain. In '15 I sold the patents, and took a job with one of the company's technology partners I'd licensed the 3D Reconstruction tech. The entire effort covered 13 years, I learned a lot, and destroyed my life savings. Can't say I would not do it all over again.

Ack, still a bit too recent and bit too raw but here goes.

Last side project which ended up becoming a bit of a rollercoaster was Brring Conference, a conference calling app where you could schedule conference calls ahead of time and it'd send out calendar invites and then automatically dial everyone in when the call time arrived.

https://www.brring.com

At the time of inception I noticed that the UK had terrible conference calling providers. They'd bill ~£50 for a one hour international conference call with 10 people. I did my numbers and realized I could provide the same call for 1/10th the price using outbound dialing and least cost routing via Twilio/Plivo etc.

I thought I was off to the races but I made the cardinal mistake of making every startup mistake in the book.

It took a few months to build a basic prototype and then using lots of my own money (and some begged, borrowed and raised) it slowly grew to become quite the monster in my life.

I wasn't a complete idiot though, I ran the idea by whom I figured my target market would be, wrote up a business plan, projected costing/break even and looked up how much my competitors were making via their company filings - the market (at least in my eyes) was there.

By the time I launched it was too little too late - our big competitors in the UK (and US for that matter) had completely revamped their products in response to Microsoft Teams. As an example of how competitive things became Google Ad spend for the term 'conference calling' literally went up 10x when I launched in response to competition between Slack, Microsoft and our own local competitors.

We obtained 100+ users and two dozen paid within the first month but the pace didn't really pickup much beyond there so after several months of adding features and running ad campaigns instead of parking it I decided to pivot the whole thing and turn it into a lead generation widget - like Intercom or Drift but supporting phone and WebRTC calling via the browser to a sales agent.

Things did better this time, I signed up paid customers ahead of time to figure out what features to launch with and ensure there was a market there once our product arrived - I was pinging maybe 30+ business owners a day asking if they'd be interested and I managed to get hoteliers, estate agents, cruise ship companies and travel agents interested in using my product.

We built Brring Live Dialer in a few months and things were growing better. Then COVID hit and our customers just evaporated. I decided I'd had enough of this side project which had become my life and slowly wound things down and started interviewing.

I have to say the experience of the above was instrumental in leveling up my resume. I came from a background of XMPP and telephony - where our software was deployed on-prem. I'd allowed myself to become deskilled so building an app from scratch using the latest web technologies of the time meant I had zero issues interviewing and landing a job.

Still smarts though, making so many ruddy mistakes!

Pulselyre: A touch-screen synthesizer app for live music production - https://www.pulselyre.com

Spent a few years experimenting with touchscreen UI and audio synthesis in my free time on and off, and eventually settled on an engine and interface design. The idea is to create 'instruments' in the editor (2nd screenshot) by connecting basic modules to create and transform audio based on user X/Y touch coordinates. Then, stack a bunch of instruments on screen and use your fingers to create and loop sequences of notes live on stage. The demo video ( https://youtu.be/Qk85IrgXRj0 ) shows how that part works.

Once COVID hit, I was sent home to do nothing with full pay, so I had free time and decided to power through and create a functional MVP and website. My plan was to get some beta users, polish it up, add missing features, and then start charging. I showed it around on some music production forums and to friends who dabble in it, and got terrible feedback: 1. Too hard to use; 2. not compatible with existing plugin systems; 3. terrible demo video (I'm not a musician nor an expert on creating cool synth sounds); 4. "touch screens are *"; 5. nobody wants to do live production; 6. it sounds terrible

Of those responses, #2 was the killer. Without compatibility with existing synth plugin frameworks, like VST, it was pretty much dead in the water. No producer was going to use a synth app that didn't support their favorite plugins, and no plugin developers were going to switch to an application with no users (I built my own custom plugin framework for it). I looked into adding support, but VST uses such a completely different architecture from the synth engine that I created that they couldn't really work together in any sensible way. One suggestion I got was to remove my synth engine entirely and just make the app a MIDI front-end for other synths, which is something I might do in the future, though the limitations of the MIDI protocol would mean removing a lot of functionality. (I have a new WFH job now, so no more unlimited time.) I'm also thinking of open-sourcing the project entirely, but I need to clean up some things in the codebase first.

I did decide to split out the UI manager that I created for the app into a separate open-source Windows UI framework, which I still work on and use for other small utility apps that I create: https://tinyurl.com/upbeatui

If I were to do it again, I'd have started with support for existing VST plugins or MIDI and worked from there. I also wouldn't have wasted so much time figuring out auth stuff. The only costs were the domain and VPS hosting.

So what's a side project [...] that hasn't actually earned you money?

There was a time when I dreamed that my "side project" - Scrawl-canvas[1][2] - would bring me fame and fortune, or maybe a few sponsorships, or a job ...

Why do you think it hasn't been as successful as you thought it would be?

There's a number of very well established Javascript libraries for the HTML5 canvas element (Konva, Fabric, EaselJS, Two, Three, Pixi, Processing/P5 ... and many others) and, seriously, the world didn't want to know about yet another one.

What would you do differently if you did it again?

Actually, nothing! The primary goal of my work when I started it was to have a project on GitHub which I could use to help leverage me into the world of professional web development. My library helped me land my first full-time gig, so in that sense it achieved its goals 100%

How much time/money did you spend building it?

18 months full time before I managed to get a job. Since then, maybe 2-3 days a month (if that) on maintenance and feature development. The upside of having an unpopular JS library is that nobody bothers you with questions about how to do stuff.

For the past 18 months I've spent a lot more time on the project - approaching full-time in some months - rewriting it from scratch, giving it a new focus, etc. The work has helped me come to terms with all the new Javascript shiny, and the ever-evolving Web API standards

What kind of iterations / improvements did you make to try and salvage it?

I'm not in the business of "salvage" - nowadays I work on the library partly to keep my coding skills sharp (next on the to-do list is learning Rust/WebAssembly to see if I can make the library run a bit faster), but mainly because it's creative and fun[3] and after this year we all need a bit more fun in our lives!

[1] Scrawl-canvas GitHub - https://github.com/KaliedaRik/Scrawl-canvas

[2] Scrawl-canvas home page - https://scrawl-v8.rikweb.org.uk/

[3] My creative coding collection on Codepen - https://codepen.io/collection/DmgxKv

I began releasing my own Android apps in 2011, and the first several had little or no success.

One piece of knowledge I learned from someone is that advertising paid decently for apps with moderate success. Knowing this steered me in a good direction.

My first app with very small success handled Access databases locally on Android. As I had the first app which could do that, I released it in four days with very few features, but blurbs saying to contact me if they wanted more features. The lesson here is what people wanted. In my mind, I was thinking of how to handle tough technical challenges and new features with domain specific functionality. But what people want was simple. A recently opened files menu. To expand database browsing to database searching. To then expand searching to allow case insensitive searches. To again expand searching to allow wildcards. This was all fairly simple, the features I thought people wanted were much more complex. I would not have learned this lesson if the market had been more mature, if people had another option to use.

I did other niche apps without lessons learned other than that niche markets are small.

Then I had my success (for me a consistent $2000+ in revenue a month without much maintenance work needed on my end meant success). The difference between my success and the previous apps I did was I aimed for a broad mass market, not a niche one, and there were some significant competitors. This is a startup lesson that is heard often - don't aim for a niche market just to avoid competition. There were two main differentiators for me - my app was a book reader, but I did a lot of work to make sure it was easy to browse, search and download tens of thousands of Project Gutenberg books (which I hosted for speed). The other differentiator is I focused on EFIGS languages, not just English, so I was #1 result for libro and libros (books in Spanish) for a long time, and also did well in France, Germany, Italy etc. Project Gutenberg having done the legwork of a supply of many foreign language books helped.

So the main lesson was to aim for the mass market for the Android form factor, and try to find a way to differentiate from competition. People appreciated my niche Access database app, but it took some weeks for me to make $20 advertising revenue from it. Many more people wanted to read Alice in Wonderland and other books.