#317 – A Decoupled Episode

Chris Gammell: This is The Amp Hour Podcast, recorded September 28th, 2016. Episode 317, a decoupling conversation. Welcome to an experimental episode of the Amp Hour. This one we're going to be doing with a call and response kind of thing. Almost, but not quite an homage to the debates that are happening, at least in the States, and the absolute garbage mess that those are. No, this is actually because I, Chris Gammell, I'm in Belgrade right now, and the time difference is always really difficult between Europe and Australia. As we've talked about in the past, that's why we're usually limited on guests that we have on from Europe, because having me and Dave and a guest from Europe, we're spanning three time zones, almost splitting the world into thirds. And so that makes things really, really difficult. So, now that I'm in that third, we thought we would try an experiment here where we talk about things in the news, things that we would normally do on any show anyways, and then basically respond to one another kind of remotely. It's kind of like a, almost like a voicemail box, for lack of a better word. And we're going to, well, we don't actually know which stories we're going to be talking about, or stories on the subreddit, or things that we would have talked about anyways. We're just going to kind of jump in and talk about different things, and then we're sending each other files, and then we're responding to them. And the interesting thing there is that there's no prompting. There is editing, of course, but, you know, we're not going to cut anything out. It's just going to be, hopefully, some brief question and answers, not even questions, some brief call and response, kind of, about things that are happening. And you'll get to hear what it sounds like when we're not talking to each other directly. If you've listened to the show in the past, you know that me and Dave, we feed each other's energy. You know, maybe we'll get upset, or maybe we're discussing a topic. But it's really different when you're just, this is more like writing letters to one another. And I think that's actually kind of interesting. I have no idea how this is going to sound. I mean, I'll know from editing, but we're going to put it out anyways. And it's very likely that this will not be an hour. So if you are expecting an hour, I apologize. And because I'm the one traveling and I forgot to set up a better recording time or situation, I suppose this is on me. But that means if it's awesome, that's also on me. All right, so let's launch into this. We're going to just, well, we're going to copy stuff together or push stuff together and see how it goes. Hope you enjoy. Let us know. We always love feedback. Feedback at theampire.com if you have email, but even better, is in the comment section. Let's try this out.

Dave Jones: Now, I really like this story because it's a good example of how FPGAs can be inserted into designs when you just don't know what you're going to use it for down the track. But let's put one in anyway because it's going to be a good thing to do. It's going to be really handy in the future. I will link to the article down below. But Microsoft have been doing servers for their Azure services. I don't know what Azure services is. I don't know, some AI thing or something, is it? Anyway, Microsoft have their own servers for doing this sort of stuff. And for the last six years, it's called Project Catapult. I guess that's pretty apt. It's an effort to accelerate cloud computing through network FPGAs. And what they're doing is when they built these servers, they went, well, okay, we've got our processor. Instead of just hooking the processor in the server directly onto the, you know, directly onto the internet, we'll put an FPGA in between. Right? So all the traffic is funneled from the processor through the FPGA and then onto the network, you know, the cloud, the internet, the interwebs, whatever it is. So all the traffic was funneled through there. And up until this time, I believe that they haven't actually really done anything with it, with the FPGA. It's just funneling the data through. But now that they're saying they're experimenting with actually re-updating the firmware, the, you know, the configuration of these FPGAs, hence their name, Fuel Programmable Gate Arrays, that's a good thing about you can do this. So they can just remotely update the configuration of these FPGAs. So instead of just passing the traffic through without actually doing anything, they've got all that logic in there that they can actually do hardware acceleration and tasks to take it away from the processor. And, you know, they're saying you can get speed increases depending on the task, of course, anywhere from 10 to 1,000 times. But it's usually in the low, you know, 10s or something like that, says one of the head architects at Microsoft working on this sort of thing. But that's a massive improvement. So you're using your existing server architecture and then just to be able to suddenly increase the ability to process stuff by a couple of orders of magnitude, you know, an order of magnitude or more. That's, it's just amazing. And the power of this thing. But as noted in the article, which we'll link in down below, they said you have to be really careful, though, because with great power comes great responsibility, of course. And if you don't get the configuration, if you don't get that logic right and you push out that change to your 100,000 servers or whatever it is, and there's some little niggly thing wrong with it, wah, wah, wah, wah. Yeah, the whole house is just going to come crashing down. And yeah, it's a real concern. But if you do get it right, then you can speed up stuff and do processing inside these FPGAs instead of on the processor. And it can be order of magnitude or more or several orders of magnitude faster. So terrific. I love this. It's a classic case of thinking ahead, whacking an FPGA because these servers are very expensive. So cost isn't, you know, probably not a major factor. So they put them in and they can probably do some real funky processing and AI type stuff that's really, you know, parallel tasks and things like that, that FPGAs are really suited for instead of sequential processes, which is what your average Intel processor, I presume, they're using in these servers. And they're Altera FPGAs now, Intel Altera FPGAs. But yeah, I thought that's really cool. That was a real smart move for them to do that six years ago.

Chris Gammell: Yeah, that's a good point at the end there about Altera like six years ago. So if people don't remember the timing, Intel bought Altera just in 2015. So maybe this was like their tryout as well where they're like, oh, well, maybe this is actually what spurred them on to go on and buy Altera and trying it out. So I remember I used to use Altera FPGAs for some stuff and they had some interesting, I remember they used to push this thing where they would do the C to H, what it was called, where you'd be writing stuff in C on like one of the soft core processors in there. And then you could like click a bit of code or highlight a bit of code and basically it would try and generate the hardware for you where you basically create a subroutine. The way to think about it is like if you think about an add or a subtract or just an arithmetic operation that happens in a processor anyways. Right. Basically, if you if you make that chunk small enough and if you're doing this bit of processing often enough. Right. So say say you're adding two and you're adding whatever number input number to another input number and then you're multiplying the whole thing by another number. Right. So it's like three operations there. Basically, if you're doing that often enough, you can start to make that into a direct hardware block and and really speed that up because now you're not chunking it through the processor each time you're actually creating custom hardware that that is dedicated just to that. Right. So now you're using multiply accumulators or you're you know, whatever your operation is, you're you're basically going and you're you're creating that as hard logic. Right. With dedicated gates to it instead of pushing it same through the same path over and over again. Right. Because that's really what a processor is doing. It's using the generic hardware and then it's writing. You know, you're basically using the generic hardware to do a variety of tasks, but that's usually much less efficient, like Dave mentioned. And so this is interesting. I'm not sure what it would actually end up looking like for network traffic. I mean, I'm sure that there's, you know, maybe some kind of port masking or something like that where there's there's got to be some kind of net benefit to doing this. But I'm just not privy to what that actually looks like. The interesting thing is that this is probably and I think the reason that Intel is interested in this kind of stuff anyways, is that, you know, if we start really butting up against Moore's law over time. Right. A lot of times the answer was just, oh, we'll just make it. We'll just wait for it to get faster. Right. You wait 18 months and you get the new set of chips in and Moore's law has taken care of you. If we start running out of that, then, you know, the thing that's going to get us more efficiencies is spending more time optimizing code or in this case, optimizing code and turning it into dedicated hardware. And this is definitely one of those paths forward. Like Dave said, the cost is definitely higher, but but that's not that, you know, for servers, at least, that's not a big deal. You might not be doing this on your little inner Internet connected widget or whatever, but especially because dedicated and flexible logic like or flexible logic net dedicated is definitely expensive in terms of transistors, but also power. But either way, if you're, you know, if you're in a situation like this where you're plugged in, you know, you can and you have the time to go and write that customized logic, it could really have some benefits. And especially for something that's high stakes like traffic. The next one for me is kind of a softball and it might end up that we both end up talking about it. But well, each of these will end up talking about, but we might end up recording these before we know. So anyways, it's near and dear to my heart because I actually have one of these things and I talked about on the show before. So the narrative clip, which I was super excited about when I first got it. Yeah, it's they just they filed for dissolution. And which means that the Internet connected device that I was excited about no longer has Internet effectively. And I don't think I talked about it since it was on the show. So but obviously, this is another yet another cautionary tale, both as a consumer. And I should mention, I did get this thing for free. So I'm out a lot less dollars than anyone who actually bought one. Anyways, as a reminder, the narrative clip is this little it's like probably, you know, 10 centimeters across the square rather. It sounds about right. Maybe a little bit less. Anyways, so maybe eight centimeter by eight centimeter by like two centimeters. And it's basically it's a little clip that you wear on your shirt. And it's got a wide angle fisheye lens on it. And it just snaps pictures at whatever frequency you set. And so every 10 seconds or 60 seconds or whatever. And then you kind of scroll back through photos over time and see where you were. And the idea is you're just kind of wearing it there. I like this idea because it was the only part of Google Glass that I liked of like, not life logging, but kind of just reminders of where I was throughout the day. And that's nice for meetings and stuff like that. As a practical note, it took terrible pictures. I mean, it was like, if you think about where your shirt collar is, most of the time, it's not pointed straight ahead. You know, the thing that was nice about Google Glass is that you were always wherever you were looking. That's what you were taking a picture of. But anyways, this is so basically they announced that this is shutting down. They announced that they will release some kind of software to jailbreak the photos that are on a narrative that and they can download also anything that's up in the cloud. But I think we've talked about this in the past where it's like at a certain point, any system where you don't control the entire chain is going to have a weak point, especially if the company goes away. Now, this is a very low is a very, very low stakes game. Right. It's just photos is poorly taken photos. But it's interesting because, you know, that we've talked about the same thing with firmware loading onto devices before. Right. Would have like if you lose if you lose control of the firmware uploading capabilities, then your device is just dead in the water. And now there's a bunch of devices that are out in the world where, you know, I can't go in. I think it's an STM chip on board, but I don't think I can go in and jailbreak this thing and use it for anything else. It's just it's just a, you know, a piece of plastic now, effectively, that's going to sit at my desk, even though it was already in that realm. So I wonder, excuse me, is is what what is the output of this? What is the practical implications of doing something like this? You know, do we do we demand that services, you know, give access at some level, like an API access level? The thing that I wonder about, I really wonder about that, because in this case, it was a Bluetooth connected device. So it went through the phone and the phone had to bounce photos up to the cloud. And you may remember as well, there was some kind of GPS tagging capability. But that might have even been for the phone. I don't remember the conversation there. But yeah, it's it's, you know, basically just another one bites the dust. This is another another cautionary tale of Internet connected hardware. And, you know, if if you were following along the story, they had a lot of delivery issues as well. So that and maybe maybe the hardware schedules, again, killed them like like anything else, like my my manufacturing woes. And the first thing that I talked about. So, Dave, what do you think? What should we do? What should we do about products like this? I mean, like, do we do we boycott at the beginning? Do we do we just say nothing Internet connected? What what do you think?

Dave Jones: And another one's gone. Another one's gone. Another one bites the dust. Hey, hey, we've got to include that clip from Queen. Come on, Chris, do it. Screw copyright. Another one's gone. This is not the first one we've seen. It won't be the last one. I talked about this on the amp hour before. I can't even remember the name of it now. I had a Internet of Things connected thermometer in my lab where it logged the temperature of the lab and everything. And it posted on a website. And everyone could look at the watch, the temperature of my lab. It was really cool. You could see the air conditioner turn on and off. It's in five minute cycles and and things like that. But I this the service that the data was being put through, whatever term you want to use for those sort of things. The data server, I that was bought out by somebody else. So it bit the dust that way in terms of software. And then I'm not sure if the company who made the thing just abandoned it. And so it became useless. So the company might still be around. I can't remember their name making new hardware. But they're they're almost disposable now. They just don't care about the long term. And we've talked about this before again and again and again. And it's just going to keep happening. And really, there's nothing you can do about it. You just need to accept this risk that an Internet of Things device you buy now in 10 years time. I think we put bets like five years time. You're X amount of X percentage likely for it to not be working anymore. 10 years is almost like 90 percent guaranteed. 15 years down the track. I don't know. It's just like it's 99 percent guaranteed not to be working in 15, 20 years time. And it's just it's just ridiculous. And if you devote your entire house to this thing, there's people who've wired. I think we had this on the show, actually. People wired up their house with the Internet of Things or Internet controlled anything. And then the company went bust or the server. They changed the server and nothing works anymore. And you're screwed. Well, it's your own damn fault for jumping on this bandwagon, especially people like us, hardware engineers who know and understand the risk, who can appreciate the risk in these. And if you can't see the risk in this, then, well, you're not a very good engineer. So it's different for the average consumer. The average consumer buys it and thinks, you know, just wouldn't even give it a second thought, wouldn't cross their mind. And that's, you know, fine. Do they need to be protected in some way by some laws, legislation or something like that? No, no. It's the marketplace is going to sort it out. People need to learn this the hard way. You can't legislate this. If a company goes bust, it goes bust. You know, if a data server changes direction or goes down, you can't force a company to stay on a certain path in a certain direction just to keep users happy. It's just inherent in the risk of the thing. You know, there are some parallels here in terms of the right to, maybe not the right to repair bill, but other bills, I think, that have gone in the US that, you know, and in fact, there's even one here in Australia where companies, large companies must keep spare parts for X number of years, you know, and things like that. And I'm not quite sure of the law behind that, but I believe there is something in place for something like that, especially for our white goods and things like that here in Australia, I think like washing machines and things like that. There's an expectation that you'll be able to buy parts down the track, and it's a similar sort of thing that you'll be able to maintain your product. In this case, they've done the right thing, and they've got a tool that you can download it, but it's just like nobody gives a toss because the thing they bought doesn't work anymore, and you can't legislate your way out of this problem. It's, yeah, buyer beware. People need to, I hate to say it, but the average consumer probably needs to be, you know, hit in the gut with this thing that, oh, you know, this is a real problem. Um, maybe there needs to be education so that they understand the risk, but your average consumer doesn't care, but your hardware person should, and they shouldn't complain about it. You guessed it, there's another low-cost pick-and-place machine on, yes, you guessed it again, Kickstarter. This one's called the Chipsetter 1, a desktop pick-and-place machine, and it looks pretty funky, and they've got 73 backers already, which is really quite surprising. I'll explain why in a minute. Uh, they've raised 54,000 Canadian. Um, hi to our Canadian listeners. Um, and they want 140,000 gold, so whether, and they've got 28 days left, uh, whether or not they'll get there, I don't know, but it's like in the order of, uh, 4,600 Canadian they're sold out, so five grand Canadian, something like that. I think it'll, you know, it goes up to six and a half grand Canadian, something like that. So, I don't know, what's that, five grand US or something. So, it's a pretty pricey pick-and-place machine, and this is one of the things. If you're doing a Kickstarter like this, getting people to risk, like, five grand, that's a huge amount of money to put down, and hope, cross your fingers, hold your tongue at the right angle, that they'll actually deliver the thing. So, I'm actually surprised they've managed to get as many backers. Um, well, I think most of those backers haven't actually bought the machine. They've just bought, like, a, just bought them a coffee or something like that, you know, for five bucks, uh, because they just want to support the idea. I don't believe it's open source hardware. I can't, can I type in open? Nope, there's no word open there at all. So, it's not open source hardware. Um, and it's, it looks like they've got reasonable people behind this, uh, sort of thing. It looks really good, and yes, the key, of course, to a pick-and-place machine, the viability of it, is having the feeders, and it looks like they've got their own design custom feeders. They actually look quite reasonable, and there's a ton of them. I don't know how many it supports, but, like, 20, 30 feeders or something? 20-odd feeders, at least? Um, so, it does look really good, and it can support trays, and it's got vision, and, uh, everything else. You know, it, it looks like a reasonable pick-and-place machine. It doesn't, from the videos, it doesn't look particularly quick. So, I don't, they've got lots of, um, data on how many, uh, components per hour and stuff like that it can, uh, place. I'm not sure if that's entirely accurate. I don't know. Anyway, uh, 600 parts, 300 components per hour. Oh, sorry, that's by hand. The chipset are one, 1,250 components per hour. Um, yeah, so, yeah, we'll see. Anyway, it didn't look particularly quick in the videos. But, hey, it does actually look pretty professional. It looks like a, like a 3D printer. All the, um, like, box type of thing. All the feeders are all inside. It looks all very neat and tidy. I don't know where the waste goes from, from all your, uh, feeder tapes. But, anyway, it does look really funky. And, so, it looks like they got their act together. So, I, you know, and because it's on Kickstarter, they do have a real prototype. But whether or not, uh, I think they might be 3D printed and stuff like that. So, if they've 3D printed their feeders and things like that, well, that's not going to work, uh, you know, long term. They'll have to get custom molds and stuff made and things like that. Anyway, um, it's another pick and place machine. And, meh, I don't know. They claim the software is the big thing with it. It's just one touch setup. And, I, I'm calling BS on that. There's no way you can do a one touch setup. It's just, no. Um, anyway, they're talking up that sort of side of things. So, it'll be interesting to see, interesting to follow. But, I, I don't know if they'll make their, uh, target of, uh, $140,000. $1,000, because that's a lot of money to plonk down on. I hope that they might actually deliver. But, hey, good luck to them. Check it out. It's the Chipset-O-1.

Chris Gammell: I think your eye caught the same thing that I did with those, uh, feeders. For sure, that is, that is by far the most important and interesting piece. And, uh, man, uh, like that, you talk to people who are doing, uh, you know, we've had people on the show before. But, like, uh, I remember when I talked to, when we talked to Jonathan from PCBNG, and I've talked, well, we've talked to other assembly people before. You know, the feeders are often the, the, the expensive part, right? So, it's, like, at least $500 a feeder. And that's something that you want the most of. Because you, you want to, if you could, you would put every part in your library as an assembler into a feeder. Because then you're just ready to go. Your setup time drops dramatically. So, if they really are making an innovative, an innovative new, uh, way to get feeders. And, like you said, if the, the wastage and everything else is, is on point there with, um, you know, making sure you can pull the tape through and handle all that stuff. You know, that is often a big source of, uh, strife in assembly anyways. Is just kind of how all this stuff happens. But if all that stuff is working out well, that could be, that could definitely be a key point of innovation here. Um, it is, there is a bunch. So, first off, people should definitely go look. They, they've done a really nice job with, uh, animated GIFs all the way through. Kind of just showing the, the pics and the stuff like that. But when you see the up close, you can definitely see a lot of the stuff. A lot of the stuff that they've designed here, like the, the holders and things like that. A lot of that is 3D printed now. Uh, I, like Dave said, I don't know if they're going to switch over to injection molding. But either way, they're definitely, um, innovate, or not innovating. They're definitely iterating, rather, uh, by creating 3D printed versions of all this stuff. In terms of the money, I don't really know. Uh, so I, I double-checked Dave's, uh, math there. It's actually 10 people have backed the main machine so far. And that is, you know, that's, uh, understandable. That's, that's not many people. But these are big machines. I think, actually, the, the interesting thing is, comparing it to another recent, uh, Kickstarter thing, which is the Waterjet. That was the Wazer, I think. So, let me see if I can find this. The Wazer was announced a while ago. But I'm, but the interesting thing is that this, oh, yeah. So, it had a $100,000 goal. It's already 1.2 million. And so, just saying, oh, it's too expensive is not quite there because, uh, obviously, this is a pretty similar price. It's $4,000 for the cheapest, or maybe $3,600 for the cheapest one. And, uh, I would argue that a pick-and-place is probably more useful. I think it's just less accessible, right? So, a way, the Wazer is basically talking about, like, uh, you know, it's a laser cutter with a, instead of a waterjet, right? But, uh, and people just don't have as much desire for, uh, the pick-and-place thing. Or maybe it's just about accessibility, right? People say, oh, I definitely can cut, or sorry, I definitely can, uh, do assembly myself if I really need to. Whereas, you know, cutting out ceramic or metal, uh, by yourself in some kind of pattern shape, it's less accessible to people. So, it is interesting as, like, as looking at these two things together, why one is so much, so much more, um, so much better funded than the other. I think it's just because not as many people think they need a pick-and-place. And I, I tend to agree with that. I mean, like, I, I, I could have, if I was given the choice between the two, if someone said, here's $4,000, you can have a waterjet cutter or a pick-and-place machine, I would probably pick a waterjet cutter as well. So, uh, yeah, that's, this is definitely an interesting one. Uh, I hope, I hope this pick-and-place does well because it could, it could be, make it more accessible. It's definitely somewhere between the Neoden TM245s and, you know, somewhere like a low-end, uh, a low-end professional pick-and-place. So, that's nice, but we'll see, we'll see how it all works out.

Dave Jones: Someone's in big trouble. The entire state of South Australia, here in Australia, obviously, and yes, it is down south, is without power. And it has been for the last day and a half or something like that, the entire state. And if you don't know how big South Australia is, it's a million, over a million square kilometres. So, that's bigger than Texas at 700,000 odd kilometres, um, square kilometres. So, it's a, the entire state is without power because apparently, um, they still don't know exactly the reason why, but we can touch on it. Um, they do get their, um, power fed from interstate. And the grid just, uh, shut off. They're having a massive storm there at the moment. It's like one in a 50-year storm. Uh, really terrible storm. And that took down a couple of major power lines or something like that, uh, possibly coming into the state. And there's some automated breakers which tripped and everything else, which is fair enough, you know. But why they didn't have enough capacity internal to keep it going, but everyone in the entire state is without power. So, it's chaos and mayhem and zombies in the street and, you know, people stuck in lifts and all sorts of stuff. So, um, like, how does this happen? Um, Sydney has a very, where I am, has a very robust power system. Uh, I ask this on the forum. Can anyone remember the entire, uh, Sydney Basin ever going down? And the answer is no, not in the last 50 years. And even then, it's tenuous back then. It's like little pockets go out here and then. So, you might have 10,000 or 20,000 people, you know, without power for a day or something like that. But it's very robust. It rarely ever goes down. Personally, I've never experienced more than a half-day, uh, power outage. And then, well, if you go to the next suburb, you know, they've got power. So, it's fine, you know. But they have not been able to fix this thing yet. Uh, not quite up with the news this morning. But I believe as of earlier this morning, it's still like two days or something. And they have not been able to restore it. So, it's not like the, you know, the, they can just reroute it somewhere else. But, uh, anyway, and now there's a lot of talk about because South Australia has made a real big push towards renewable energy recently, which is fantastic. But now some people are saying they went too far and compromised the robustness of their grid. And hence, this is a result. And apparently there's some news article about the, uh, government's begging one of the coal-fired power plants, which they, uh, forced to shut down, begging them to come back online. And, you know, um, and people were warning about this apparently just like six weeks ago or something like that, that the grid is not, you know, um, going all renewable could be a, uh, problem. And, well, it's not if you architect it in the correct way. So, yeah, somebody hasn't really, some systems engineer hasn't done their job or maybe they got, maybe they did and they warned. And the engineer in the corner is going, hey, I think there's a problem to, you know, the powers that be. And they go, shut up. You know, this is all about politics and renewable energy and everything else. And, but, but, but I, the grids are really, you know, in, really compromised. And it could easily go down and take the whole state out. Shut up. Really? No, I, you don't know what you're talking about. Anyway, so someone's in trouble. So there's going to be a lot of fallout from this. What's the worst power outage that you've had personally or in your country? Australia's got a really, generally a very robust power system. As I said, especially here in Australia, when they built the snowy mountains scheme, uh, way back, I'm going to have to do a site visit on that, but yeah, uh, which handles peak loads really well and stuff like that. That's a hydro scheme. So anyway, let us know down below. Somebody's going to get their ass kicked for this one.

Chris Gammell: Yeah. I saw these news stories about South Australia. I think lightning is actually a, uh, is an acceptable response, uh, for why your power goes out. You actually, if you don't know, one of the largest, uh, power outages in the States was due, it was back in mid 2000s. We may have mentioned on the show before, but it was actually due to a squirrel. And I was just reading online. There's, there's, uh, there's actually a well-known phenomenon where squirrels take out, uh, they take out large swaths of, of stuff. You know, they blow a transformer. I think, I think your assertion about, you know, things should be more robust. I think, I think these things are houses of cars more often than not. Right. So, uh, the one that I'm talking about, at least by me was, uh, so it looks like it was at least, it was New York, Pennsylvania, Ohio, Michigan, and a good chunk of Canada and stuff further either further East rather. Uh, it's called the Northeast Blackout of 2003 and, uh, yeah, it, it, uh, let's see, it was caused by a software bug in the alarm system of the control, blah, blah, blah. It was something with a squirrel though, too. I definitely remember squirrels being involved there. Um, and, and so if you, and there's another, uh, I was just Googling squirrel power outage. Uh, apparently there was another one in 1968 caused by the great squirrel migration. Um, and so I'll, I'll link both these things in. I just don't think, I mean, like, I know that, like you said, someone might lose their shirt over, uh, whatever this one is because that it, it is an entire region. And especially in Australia, I'm sure that the, uh, I'm sure it's starting to warm up there enough that it's, it's really affecting things. And especially because everything's so, uh, electricity based these days, but I don't know. I think, I think if we can, if it holds up under most lightning storms and then one takes it out, a 50 year storm takes it out. That's better than saying a squirrel killed our power. All right. My next topic is, uh, well, you'll know it when you hear it. It's, it's, I'm a little out of, uh, my element element, but I think it's actually interesting because also because I already talked about the, uh, narrative thing as a connected device. Uh, this is from the files of nobody is surprised at all department. Uh, this is, uh, about IOT devices starting to participate in botnet attacks. Now I feel a little bit weird cause I, I don't really consider the devices they're talking about IOT, but really they are. So anyways, there was about 150,000 IOT devices reported to be sending packets like crazy at this hosting provider. And they said they got up to like a gigabit per second of data that they were throwing at this provider where you, you know, your server is just going to fall over under that kind of load. Uh, but a lot of that came from CCTV devices and DVRs, right? Both of which are, uh, you know, high bandwidth, high enough to send, you know, nice video streams and stuff like that. But basically they were using that as throwing packets, you know, basically like, like, uh, you know, a bunch of water guns. If you had enough water guns shooting off a mountain, uh, off a cliff top, you know, it's eventually it's going to be enough water that it, it could probably destroy something. Uh, anyways, um, yeah. So, you know, they say IOT, I, I, like I said, I think that's a little bit, when I, when I think IOT, I usually think about like seven. And, you know, sensor motes and stuff like that, or, you know, wifi connected stuff. Some of these might probably, are probably wired as well. And so I don't think, you know, I don't think that, uh, you know, a particle photon is going to be, you know, there's not going to be a botnet attack from that anytime soon. But what it does is it just reinforces the same thing we always hear about, which is, uh, security issues in these devices. You know, basically they're default, they left all default passwords. There's not anything, there's really no security protocols at all. And, you know, you read about this other stuff too, right? So I've, I've interviewed Sammy on the show before, and he was talking about a lot of the vulnerabilities in hardware. And it's just not, it's not that challenging because the people are not yet paying for it. And so what I wonder though, is, you know, is there, I mean, there at least is an industry for it for software. And I know it's rising as an industry for hardware, but I think what it really comes down to is, is there some kind of regulation that should come around it? And I mean regulation, but more like, uh, cause I think there are people that are trying to do regulation. It's more about, um, is there general practices? Because I'm sure that some of our listeners are working on connected things. Um, right. Uh, I talked to Omer Kilic, uh, when I was in London, uh, about their connected thing. Right. And, and, you know, that if, if you get a sufficiently large number of devices in the field and they all have, um, the same password, not saying that theirs did. I'm just saying that if, if someone didn't think about that stuff, then it's like, well, what do you, what do you do then? And so how do you, I mean, I guess you have to like hunt down a security firm and, and think about that. I guess the real question is when, when do you really start doing that? Right. Because it doesn't seem like this is all at the hardware level. It seems like it's more at the upper, the upper levels. And it's usually, it seems like it's for devices that are, are using higher, higher level software anyways. Right. So full operating system, Linux based devices. However, you know, we keep seeing more and more devices that it's like, oh, well, Linux is basically free, you know, or it's low powered. So, you know, it's low power enough that you can have Linux on it, right? Use a BeagleBone Black or something like that. And then if you start not locking down your system properly, uh, well, you're basically putting a bunch of traffic, a bunch of, you know, garbage traffic generators on a network and you can really start to mess stuff up.

Dave Jones: One major question here is, well, who's actually responsible for this for, you know, taking care of security and stuff like this? Is it the DVR manufacturer? They sell a DVR and yeah, it hooks onto the internet. But, and, but where does their responsibility, um, start and end? Should, is it, and is the solution as simple as, uh, simply forcing the user, designing the product so it forces the user to enter a password during the setup? Oh, you can't, you take it out of the box and you can't use the thing. It does ruin the out of the box experience. But, you know, that's the price that the consumer might have to pay to ensure that, you know, this problem doesn't exist. But, does the consumer care if their, um, if their DVR has been taken over, their internet of things light bulb has been taken over and it's sending out packets to the internet and, you know, trashing some server on the other side of the planet? They don't care unless, they don't know nor care unless their bill comes in from their ISP and, oh, where did all this data, you know, um, come from or something like that? But, well, cause we have data caps here in Australia. So, go figure. Anyway, so that's something to ask. Where does, like, does the, the consumer's probably not even going to care? And, you know, once again, can you, as I talked about before, can you solve this with legislation and things like that? I, I don't think so. Um, the market just has to sort itself out. And, this one might be a hard one to sort out because, like, the manufacturer, nothing comes back to them, really, um, apart from, you know, some media exposure and going, oh, this, you know, Cisco router or something has got some flaw in it and it might make the headlines. But, apart from that, um, you know, if it doesn't, if the name doesn't get dragged through the media mud, then they're off the hook. The consumer's off the hook. They don't care. They don't even notice that their Internet of Things light bulb is a zombie and is attacking something. Um, and nor should they, I guess. So, you know, manufacturers probably could step up to the plate and, as I said, um, just change the user experience. When you plug in your light bulb, you've got to set up your own password and explain to the user why this is important. You know, security's important. You don't want it to be hacked. You know, maybe you can put the fear of hacking into the consumer, perhaps. I don't know whether or not that's going to do any good. But, yeah, I don't, this one's a tough one to solve, I suspect. Um, you know, there's security and there's a lack of, you know, there's security as in trying to beat security. You know, you don't use a high enough hash encryption or something like that. And that's a manufacturer thing. They've got to choose a good encryption and stuff like that. And the other one is, is just the simple password vulnerability. Um, you know, your standard router. I just bought one the other day for the new office that I'm in here. And, um, yeah, you know, password, you know, username, admin, password, admin. And most people will not change that. So, and I didn't use the install disk, but I'm pretty sure the install disk won't force you to change it. Or whatever, maybe it does. Maybe they could do that. But, hey, of course I changed it. You know, it's some 20 character long thing now. No one's ever going to guess it. So unless there's some hardware, you know, um, you know, some workaround hack for that router, I'm going to be pretty safe. But, yeah, this ain't an easy question, I'm afraid. Well, what do you know? Who would have predicted this one? A company that we've talked about quite a few times, Upverter. They're the online EDA company where you can, you know, your schematic and PCB CAD software is all online. It's all web-based and everything's stored on the cloud and operates from the cloud. And, you know, you don't have any dedicated software on your machine for doing your EDA stuff. And, wah, wah, wah, wah. What do you know? They have, um, had issues. Uh, who is it? Workbench throwaway on the, uh, Reddit. That's his username. His or her username. Um, on Reddit has, uh, complained that, uh, they can't export schematics, uh, for over 18 days now. And there was been no response from Upverter or their, uh, support forum. Other than that, they're saying, and I quote, we're working on interesting projects. Um, and I'll quote again, uh, I'm not sure which is worse. My schematics and layout being trapped, lost inside Upverter. Or Dave being right about the whole cloud model not being suitable for EDA. Yeah, well, herein lies the problem. When you've got your everything online, operating online like this, you are 100% reliant on, A, your internet connection, and B, their server being up and having no issues and stuff like that. And it works at the massive scale for things like, uh, you know, Gmail. Like, I'm not against these sort of things. I've been using Gmail for, I don't know, seven, eight years now or something like that. I made the switch to completely online email. I've had very few, uh, issues. And I do the online, uh, do a lot of, uh, documents online as well. Using the Google Docs online thing and stuff like that. But yeah, CAD from these, I, I hate to say it, but these smaller companies, they don't have the same resources and everything else to make it necessarily as robust. Um, although the good thing is with the small companies, they can often move quicker. Now, I don't know why they didn't move quicker on this, but not being able to export your stuff for 18 days is just ridiculous. Um, so yeah, you're, you're trapped. You can't export anything. You can't do any work. Imagine that. So if you had your, a local, uh, software working locally on machine, you don't need any internet connection. It just works locally online. And the only reason you might need the internet is to, uh, go to DigiKey and look up parts and search for data sheets and things like that. Um, so yeah, therein lies the dangers. Let that be a warning to you of these online CAD systems. Uh, can we say I told you so? Maybe. Hmm. Anyway, it did have a happy ending. Um, after a couple of weeks, they, uh, they cleared whatever processing backlog they had and, and they, um, and workbench throwaway can finally download files. Yay. Upverter. Well done. And that won't be the last time we hear about stuff like this. So I don't know what went wrong there. It'll be interesting to find out.

Chris Gammell: I think that's a good point about the, the resources and the companies there. Cause I think what we're seeing with Upverter and other companies like it, like, I think that they've been focused. The reason that this person who was having trouble getting exported was actually just cause it was an individual. Right. And I know that they've focused more on companies and if you see their concierge service, this is a cool idea. That's all kind of focused towards companies and that's really where the money's going to be. Right. So that's, it seems like it's a, you know, more of a business model thing as well for them where the companies are going to be able to pay big licensing fees. That's a lot of CAD software anyways. Right. So that I, I can understand how that happens, but it is unfortunate that it took that long. One of the things that I had pointed out with Upverter is that their, their strength for a while there was the fact that they were working on so many different exporters, importers, stuff like that. But yeah, it is a, that is a little scary when your, your designs get like completely locked in there and you're like, uh, guys, uh, uh, yeah. And you know, I, we have talked about this to death probably, but you know, I've, I've written about that too. I wrote about, I actually tried out Upverter on a plane just to see what that looks like. And like you said, Dave, it's, you know, it's part partially you need internet anyways to get to data sheets and stuff like that. But at the same time, you know, when you're, when your boss is screaming at you to get something done, if your internet's down or your, your design's locked up because of an outside company, uh, that's, that's pretty scary. So some people might know that I've been working on the Hackaday super conference. That's something I do for my work. Uh, and interesting piece of that is that I've been working on the, um, getting the badge manufactured. Right. And that's a big part of a lot of conferences is, you know, they want to have blinky badges or interactive badges. People that have gone to DEF CON before know that they add all these puzzles in and those are often, uh, so that Joe Grand did some of the early electronic ones. And, uh, then I forget his name. Lost boy, I think is the one who took it over, but there it's, it's often sponsored by propeller and they make just tons and tons and tons of them. Um, and, uh, so I actually did the super conference badge last year too, but that was really simple. It was just a, um, it was just a blank PCB with some prototyping areas and then people kind of hack stuff onto it. I actually rather like that. I really liked that idea of, you know, just complete blank slate, but it was also because we ran out of time. This year we actually planned ahead and, uh, Voya, who we've had on the show here before, also when I've been in Serbia, um, he designed the badge. And if you saw, uh, there was a, I guess I could have posted a link to the, the Hackaday post about, about the badge, but basically it's a pick based design and I've got it in front of me right now. Um, and stuff like that. It's, you know, it's a pick based design and accelerometer and it's got, uh, IR send and receive type of stuff and a bunch of LEDs that can be used as a screen. And, uh, yeah, so interesting, interesting badge. Um, you know, I've seen some of the other ones that have been done for other conferences. We talked about the one that was based on the hack RF, I think for CCC or some kind of other hacker conference, but basically hardware as a badge for a conference is a very common thing these days. And this thing that I'm working on is, is no, uh, exception. I've been working on the manufacturing side of it though, is actually getting this thing made. And the thing that I wanted to talk about and kind of ask Dave about, um, or I would have asked Dave about if we, if we would have been talking has been about the sourcing issues, um, for regional stuff. Um, so it's been really interesting because Voya is here in, uh, Eastern Europe and he's been sourcing, using a lot of, a lot of, uh, uh, local distributors, stuff that he knows and, you know, people that he trusts really because he uses manufacturing, but we're not manufacturing here. We're manufacturing over in the States. And so basically for a, for a while there, I was basically really struggling because I was thinking that we were going to have to get these parts somewhere else. I didn't, I didn't know if I could export or not. I had a couple interesting online experiences where the, the sites that I was trying to buy from said that I couldn't export them or couldn't move them overseas or anything like that. And so I started to get really worried and especially thinking about, I mean, this is, there are export, some export restrictions on components because of lawsuits or weird stuff like that. Um, but then I later found out that it was basically, they just didn't want to sell that stock outside of the region because they wanted to make sure they kept it here. Anyway, um, you know, when you're starting to do a global supply chain, I was actually kind of not that this is a very small microcosm of the whole thing, but I was thinking about it where people design something and then they take it over to China. It's kind of the same, except I was going in reverse. You know, I had a bunch of parts that were available here and design are ready for this market. And then there really weren't any replacements that existed in the States. Um, you know, in terms of stock at the standard, you know, even arrow, avnet, anyone like that, they were available, but you'd had to do standard lead time, six to eight weeks and, um, definitely nothing, nothing in stock. And, uh, so it's been, it's been a bit of a struggle actually ended up just, just got it resolved the other day and, uh, that felt good. But, uh, I just wonder about that. Like if it's almost dangerous to, um, you know, to work outside your region in terms of, uh, not where you are, but actually where you're manufacturing, right? So if you are, say you're going to China for manufacturing, it's almost, it's almost like you have to go there because if you spec in parts that are only available from like Mauser, they have to ship from Dallas to China, right? Or if they're only available from DigiKey, they have to ship from North Dakota to North Dakota or Minnesota, wherever they are, the Thief River Falls to China, right? And same thing here where a lot of these parts were only available in Eastern Europe and at least in stock. And, uh, you know, they're great parts, but they were only available here and, and, uh, you know, and there's cost differences and shipping and taxes and import duties and all that other stuff. And so, uh, it's been an interesting challenge. I'm probably gonna end up writing about it at some point here, but, uh, I, I guess the question is, is do you have to, uh, buy parts from a distributor locally? I mean, basically this is also another thing that's interesting about this is this is at quantities. This is about, I think we're doing about 300 boards and this is a quantities where you can still get away with distribution. I think, I think that's another thing that I'm, I'm realizing is that, you know, a couple of things on here, it was basically getting towards, oh, I really, I should have, I should have been negotiating at the vendor level and I should have been ordering parts at the, you know, the eight week mark. And, you know, basically it was a, it was a brief reminder or a brief foray back into the manufacturing world where, you know, real timelines. And, um, so if you don't plan for that, you can get, you can get in trouble. So again, the question I guess for Dave is, do you have to buy parts, um, locally, right? I mean, Dave's done manufacturing in Australia and, um, you know, how does that end up affecting it? I think with Digikeets, uh, a lot of people do that and they, they end up shipping it wherever, but, um, just wondering about experiences there.

Dave Jones: The basic answer is, uh, no, you don't have to buy locally and you probably, I hate to say it. You probably shouldn't. Um, because, well, especially here in Australia, we just don't have the stock in place. I mean, we have RS components here. We have Farnell here, but very, very little stock locally. And I just, and they're generally more expensive, um, than your Digikeys and your Mousers. So I'm going to design my product almost always, even in that, like, you know, a thousand volume, I'm still going to design it around Digikey and Mouser part availability. And if I think it's a key, uh, critical part, I will, uh, try and maybe source a reel somewhere else from one of the, uh, online, they're usually online American, uh, American distributors and, um, like onlinecomponents.com or something they've caught. I get a, you know, a bit of stock, uh, from places like there, but generally, no, I'm going to design in the Farnell, I'm going to design in the, uh, Mouser and the Digikey because A, they've, they have the most massive stock available anywhere in the world. Like, and, and the scale of parts that they have is just, is just phenomenal. And you can trust them. If you know, you can see exactly how many parts they got in stock. You can, uh, get alternative parts and you can do all sorts of stuff. You know exactly what you're going to pay. And because we, you've used it before, you know, it's going to get through customs, no problems at all. In the case of Australia, anything under a thousand dollars, just sales through customs. You don't pay any duty. There's no holdups, anything like that. Anything over a thousand will automatically, uh, a thousand Australian dollars, including shipping will automatically get flagged by customs. So there might be a couple of days delay there until they contact you. And then you pay your import, uh, duty and, uh, GST and things like that. But no, I'll just order multiple things under a thousand dollars and get them here. Even if I'm manufacturing in China, I will get Mouser and DigiKey to ship the stuff here so that I, I can double check that it's the right stuff and, you know, got the right number and then package it all up. And then I, once I'm right with the boards and everything else, I'll get everything shipped here, the blank boards, the boxes, the whatever, and then ship it off to my manufacturer, either here in Australia or overseas. I don't deal with, uh, local, uh, component supplies, the arrows and the avnets. I, I, I gave that up years ago now, um, like a decade ago that, you know, with online e-commerce and, and Mouser and DigiKey now so phenomenally good. Um, I just gave up, you know, I used to use Farnell and RS a lot more when they had a lot more local stock, but now they just, uh, don't. So I'm going to go for the lowest risk trusted solution every time, even if I have to pay a bit more, it's worth every cent. Uh, like even if thou, you're talking about 300 volume or something like that, nah, it's nothing. I'm doing, you know, thousand volume plus, uh, and I'm still ordering from the likes of DigiKey and Mouser for most stuff. The only component I source locally is the box, uh, which is actually manufactured here in Australia, believe it or not. The, uh, little black, uh, Jiffy box that I use, I'm quite proud that that's actually, uh, manufactured here. And, uh, yeah, I buy that, uh, locally there. I don't, I can only recall one little, uh, stock issue with that, uh, once, but they just churned out more and, you know, I can get as many as I want. So no, I'm, you, when you get from these dealers, they're, A, they're a pain in the ass to work with, uh, sometimes and they don't have stuff in stock and they claim they'll be able to get it. Oh yeah, I have that for you in a week. And no, they, no, you can't trust them. Trusting people is a pain in the ass and I don't like it. I like seeing that it's all online. There it is. I can get my thousand parts and get a real, buy it now. Boom. I know it's going to make it through customs because it's under that and Bob's your uncle. So yeah, easy. I just stick with the big daddies, even if it costs me more.

Chris Gammell: So here's another one related to that whole regional part thing that I mentioned earlier. Uh, this is actually, I was trolling nice chips because if you remember, uh, when I talked to the macro fab guys, we were talking about how do you find about new parts and nice chips is one of the subreddits where we had discussed. I hadn't been on there in a while. So if you can't tell, I'm trying to find links that I don't think Dave has talked about. So we have a lot of back and forth here. Um, anyways, uh, the, on the nice chips thing, there was actually a link to 41 j.com and I can't quite figure out who it is. Uh, it's someone that works with Akiba, uh, who is from freak labs. Who's been on the show a couple of times as well. I assume this person is, uh, on, uh, the, the hacker farm. I apologize. But anyways, the other thing that I loved it. Oh, here we go. Nava, Nava Whiteford. Sorry. Uh, anyways, I love that this, uh, blog subtext is if I get 3000. Visitors a day. My wife says I can buy an electron microscope and you want to help me buy a SEM, don't you? So we can definitely help with that at least. Anyways, uh, this is all about the CL0116 solar lantern controller. I see, I had actually never heard of this before. And if you click over to the data sheet, it is most definitely this chip link semiconductor. This is like a, this is like a Shenzhen semiconductor number five. Uh, it's the kind of chip where it's super cheap. Uh, and, but it's made in such quantities because this is a solar LED lamp controller, right? This is used in those little garden lamps that, you know, they have the tiny solar cell on the top. And then underneath there's usually just a single LED and then a battery and it's just storing power for the day. And, uh, and then basically it's outputting, um, that once, you know, it senses that it's dark out, then it also, it just, uh, turns on it, it switches over from battery power and turns on the LED at night. Uh, so anyways, this is a really interesting little part, uh, TO94 part. Uh, but basically it's just a switcher. It's like a, it's a boost controller effectively. And, uh, someone in the comments on Reddit was talking about how you can basically use these as a jewel thief because it's doing the same thing. It's sipping power from a solar cell and it's trying to basically boost that up to something usable, both for, uh, charging a battery and then also, uh, possibly lighting, lighting an LED directly, um, on, on the actual blog post, this is about a, a little, um, uh, a flower with LEDs on it. And, uh, so I, what I wonder from you, Dave is, uh, do you know about any other ways to get this stuff? I was thinking maybe like AliExpress or if you're trolling some of those places, like you can kind of find this stuff, but I have been thinking about a lot of these, um, you know, there, there are like, so I mentioned the regional parts, but like so many in China are just inaccessible to me, like unless you're in the, uh, you know, you're on Chinese data sheet sites or if you're, you know, if you know what you're looking for specifically, or if you're trolling the markets, then maybe you could find it. But it seems like so many of these things are just kind of out of my realm anyways, that I don't even see a lot of these. And there was some interesting stuff like this. This is probably really low cost and, uh, you know, it's, it's doing something interesting. So I was just wondering if you hear about that, maybe forum or something else.

Dave Jones: This one's really interesting because it really is, as you said, quite difficult. To find out and know about, uh, this sort of stuff because we're, we're always, uh, trawling through DigiKey and Mausa when we go designs and we'll like click around and hyperlink every, oh, look at this. Oh, what's the, what's this category of chips and things like that. And there is no real repository of what's available in the Shenzhen markets. You know, what are, you know, these farting novelty toys, what chip is using, the solar power light, what is that one using? And there's countless, I'm sure there's countless, uh, custom ASIC chips out there doing really cool stuff. That could be very useful for your design, but there is a problem you need to be aware of is that it may not be available tomorrow. So if you're designing a commercial product that might have, you know, a five year, uh, product lifespan or something like that, you've got to be real careful buying these chips, but hey, they might be cheap as chips so that you might go, hey, I'm going to buy a 10 year supply of these puppies because they're, you know, 10 cents a pop. And I'd highly recommend you do that. If you find a chip you like and you think you might sell a thousand widgets, buy 50,000 parts or something like that. You know, you don't know where it's going to lead. Don't just, oh, I'm going to make a thousand, so I'll buy 2,000 of them. No, buy 10,000, 50,000, something like that. Just include it, absorb that into your cost of your product. I know I've gotten completely sidetracked here, but it's an important issue. These chips are great until you can't get them anymore. And that was always the issue with Maxim, for example, was the famous example. They had these fantastic chips before the internet. I had my bookshelf full of Maxim data books. They'd release like every six months. It was three months at one stage. They were releasing so many chips, like 600 chips a year. And they would release, it was so many that they had to release update data sheet, like six monthly updated data books, like a thousand page data books with the latest data sheets for the, you know, the hundred chips they've just, the 200 chips they've just designed. And they'd have them in categories, and that was so wonderful. And it was the greatest thing since sliced bread, Maxim, with all their custom ICs. And then, you know, you design it in, you get all excited, and then you can't bloody buy the thing. That'll be a 40-week lead time, sir. Thank you very much. And it's like, oh, great. There goes my whole product idea down the toilet, because nobody else makes a compatible chip. You'd need, you know, a dozen chips to engineer a solution that, you know, they just made available so easily. So there's, this is great. And I'd love to see some sort of repository, you know, the Reddit list you talked about, I think I visited once, haven't been in a long time, you know, Nice Chips. Someone should have like, you know, nicechips.com or something like that, you know, or weirdchips.com. And then just have a repository and link in data sheets. Data sheets are often hard to get, and things like that. So that'd be, you know, that'd be really terrific if someone can take that by the horns. But yeah, there's probably not a lot, not going to be a lot of love there in return. You have to do it for, just because you want to get it done. But that'd be great if there was something like that available. But yeah, just beware of these things. Great for hobby projects, things like that. And for, you know, dozens of and things like that. But no, if you're in the serious end of town, designing commercial products, really be careful doing this. Unless you are designing a novelty farting toy, you know, that's going to be manufactured in China and sold through the local $2 shop, then that's fine. Yeah, they generally are designed around one big batch thing. They'll make this chip and then they'll sell it. These lanterns could be different. These have a much longer lifespan. So they might have designed this five years ago and they're still flogging off that chip. So, but yeah, that's a bit more generic. But just be careful of the real specific ones. You could come a gutzer.

Chris Gammell: So my last one here, we're only doing five each for people that are listening at home. But my last one here is actually taking advantage. It's probably not going to work out like this, but it's taking advantage of the fact that we're not talking to each other because I can harp on about revision control. And me and Dave, you don't really see eye to eye to that. Now, Dave is going to get the last word on this. So, and that's really how it goes. Whoever starts the conversation doesn't get the last word, which is kind of interesting. But anyways, I actually asked about, on Twitter, if people had worked on a team that doesn't use revision control for hardware and whether they had or why or why they don't. And it was interesting. You know, I actually love, this is one of the things I love Twitter for. It's just kind of getting a lot of feedback from, you know, experiences and then conversations about different things. And yeah, it seems like a lot of the software people that are doing hardware, of course, are using it. The ones who are in teams that don't have revision control are basically stuck, right? I mean, if you think about, especially if you are a junior engineer and you want to use revision control, but you aren't in charge, right? It's like you can only harp on it so much before you just get knocked down. And they're like, shut up, junior. And so that's tough, too. And then there's, you know, various levels of what they're using. It's not like everyone's using, you know, SVN or Git or anything like that. Some people are doing it just with physical backups, right? They write, they zip up the file, they add a date to it, and then they, you know, they, that was one of them. Jonathan actually was called in before. And he said they have a text file with the details, right? That is revision control. It's just not in a control management software type of thing or whatever they call it. And so, yeah, I mean, like that is tracking as long as, you know, that's only as good. Really, any revision system is only as good as if you're using it or not. And we've talked about on the show as well that, you know, if you're just using Dropbox and you're saving stuff in there, it's sometimes better than nothing. But, you know, being able to actually see why you're making changes and having that kind of, that effectively that text file is like the traveling document why you made changes. Now, the reason I brought it up as well is because we were talking to Robert last week, and I was thinking about, I was actually talking today to someone about multi, you know, six plus layer designs and how those are six months and stuff like that. And you talked about, not you, Dave, and talked about working on these designs for like six months sometimes. And I think that's actually really an interesting piece, especially because you must have been tracking that somehow, right? I mean, there's, some of it is just you have to get stuff, you know, point A connected to point B, and that is really difficult. And, but then, you know, once you get to a certain point and, you know, an engineer comes in and says, oh, no, no, we need this chip over here because of RF or whatever the change is, you know, then that you have to kind of track that over time and say, you know, you either have to justify it to your boss or justify it to yourself when you're documenting later. So I guess my question was, how were you documenting that stuff? You know, revision control is one form of documentation, but obviously you were doing that anyway. So maybe you were just keeping a lab notebook or anything else like that. I would almost put forward that a lab notebook is another version of revision control, depending on, you know, how you track where you are in the layout or anything like that. But, you know, obviously I, I'm a big fan of Git. I use KiCad. So that has the text files instead of like Altium has binary files. So that is actually really hard to compare one to another. But in reality, you're never really comparing even with a text-based file format. You're not really comparing it there. So anyways, I guess my question for you would be, how, how do you actually document? This question is morphed as I started talking as it often does when we talk to each other. But how do you, how did you actually document stuff over time when you're doing those big layouts? Because I, like I said, I was talking to someone about that today, about the six month plus layouts. So I'd love to hear more about what you were, how you were actually documenting that over time and how you dealt with that, because that's just a monumental task anyways.

Dave Jones: I think you might be a bit confused with my position on this revision control thing. I'm totally for revision control and have used it in the past at my former company, Altium. We used to use Subversion, the Tortoise plug-in. And I used to love that thing. It actually worked really well. I loved it. But yeah, companies before, because Altium was a software company, right? So obviously that was the, when you go into a software company, that was the first software company I worked at. Even though I was a hardware engineer at a software company, we just were sort of forced to use that. And I really got to like it. And that was my first real experience with a content management, control management system, CMS, or whatever it's called. I don't know. With this revision control, I've never used Git for that sort of thing, because that's what Altium used, the Subversion system. But at previous, because I've worked at hardware companies previous to that, they didn't really have, you know, a Git or a Subversion type system. So they really didn't have, well, no, actually they did. Way back, before the internet, okay, before like 1995, my first full-time PCB layout job was in 94, I think, at GC Marconi. Or was it 95? Anyway, basically before the internet, before I even knew there was such a thing as version control, it was all, the hardware design part of things was actually done on paper. So it would have these big A3 graph papers in these big binders, and we'd have shells full of these binders. And we had effectively a revision control system there. Each project had its own unique code, and then each part of the project had its own subcode, and then you'd have revision, you know, A, B, C, D, or whatever it was. And you would keep these in these folders. So you could go to the folder, and you could physically flip through all these A3 graph papers, where you'd actually, you know, if you drew a circuit, you'd draw it on the A3 graph paper. If you had a data sheet that you wanted to include and mark up, you'd actually photocopy the data sheet on the photocopier, stick it on there with glue, and then mark it up, and you cut and pasted all this stuff onto physical paper. And that was a revision control system in itself. And that worked fairly well, and you could copy it. You could put the whole, you know, some secretary would often copy, duplicate the entire sets, and move it across facilities or something like that. And that was, revision control was very handy. You'd go back. As far as PCB design goes, as I said, first full-time job back in 95, before the internet, before version control, you just, the company didn't really have anything for that. You were, it was just up to you as the professional doing this to just get your job done. And yeah, as you were talking about, I would do the subdirectories, you know, with the different dates or whatever, or I'd call them in my own internal versions, so to speak, you know, before I got to release. And like, I was literally working on a four- to six-month layout board, you know, huge one. And you would do that internally, because you felt you had to, because you had to go back if you, you know, screwed something up. And I just did it that way, using subdirected, and it worked just fine. It got me out of trouble a few times. And if I didn't, hadn't have done that, I would have been in a spot of bother. It would have taken me a lot longer. I would have had to start again from scratch or something. Or, you know, it would have been much more difficult. But it saved my bacon quite a few times. But there's nothing wrong with the subdirectories and, you know, based system, as long as you're disciplined enough to do that. And I would do that often several times a day. If I'm making huge progress, if I'm working on a large board, and I make a strategic decision to route this direction or place parts over there, okay, I know this is a big chain, so I'm going to back this up first. And, of course, that would go on to the back of the internal intranet. So that was all backed up and things like that often. But, yeah, it was just my own system. And, yes, I would document that either on paper form or in a file form, or it's just pretty obvious. Like, you know, internal revision one. It was all in the file name and the subdirectory names and things like that. And that just worked fine. There's nothing inherently wrong with that. Yeah, there are better tools these days for things like that. But I just wasn't aware of them back then before the internet. And the company I worked for, you typically take on the culture of the company that you move into. And, as you said, it is, you know, shut up, youngster. It is difficult to, you know, enact change within a company, especially major, you know, changing the way people do things. That's really difficult. Engineers don't like change.

Chris Gammell: Well, that is a good point to end on. I wonder if engineers don't like change or really listeners to The Empire don't like change. That is the last of our segment, our decoupled conversation. Yeah, so that was different. I think the interesting thing is that when people say they don't like that me and Dave argue, this is a perfect, this is exactly what I'm going to point them to if they don't like this show or if they do like this show. This is basically what it sounds like when you don't have interaction back and forth or if there's no delays either. So, like I said at the beginning, if you liked it, don't like it, I don't know. This was basically us trying a new thing because of our situation. But, you know, it's always good to innovate. 317 shows in, you never know. Many thanks to my co-host Dave, who I did not get to talk to this week. Thank you to people who submit to our subreddit. That is always helpful. That's reddit.com slash r slash theamp hour. That's where we keep all of our links, including stuff we don't get to talk about. It's a good place to discuss stuff. It's a good place to post links. It's a good place to see the stuff we missed. And thanks to you, our listener, of course. We especially love our Patreon sponsors. We're very grateful for them. And we love that people still are listening. Like I said, 317 shows in. Our astute and longtime listeners of the Amp Hour may recognize that in between music that I used as editing, as ways to separate out the talking points this week, that is actually from Chloroform the Amps, which is the song that was donated to us by Paul Stevenson. Thanks much to him. We've continued to use that theme song for about 200 plus episodes now. So we do appreciate that too. We'll be back next week with another episode. Thanks for listening.

Dave Jones: So, yeah, easy. I just stick with the big daddies, even if it costs me more. Thank you.