Clock debug log #1 - month rollover issue

Here I'm going to talk about a month rollover bug that my clock had been experiencing.

In a previous post I spoke about an issue that came about when I showed the clock to a friend. When the clock advanced onwards from 23:59:59 on, say, the 31st day of a month to what should have been 00:00:00 on the 1st day of the following month, it actually advanced to day 0 - which is something that wasn't supposed to be able to happen.

I'd designed in a preset input to the first stage of the day-of-month units counter such that when ever certain reset signals were asserted to reset the counter back to the start of the month, the units counter should have been pre-loaded with a 1.

But I've already discussed and fixed that issue. This post pertains to a different rollover bug that I encountered after I had debugged and fixed the one above.

And the story behind this one is that when the clock is on day 29 of a month, and it needs to advance to day 30, it would actually advance to the next month and again to day 0, instead of the 30th.

My suspicions about why this was happening surrounds the use of ripple counters in my clock. I suspected that because of the way ripple counters work, in that it can take "some time" for a change starting at the least significant bit to make its way through all of the stages of the counter. What this means is that for a very brief period of time the counter can represent a number of different values in quick succession as the flip flops transition between different states.

And because of this, I figured that the block of combinational logic that determines what day of the month it is, and based on the "length-of-month" input (30 day, 31 day or February), at some point in time the state of the counters was such that the reset logic figured that the date actually needed to advance to the next month instead of letting the counters advance to the next day.

To test this theory I needed to probe around the reset circuitry in the clock to see what it was doing. Specifically I was looking at the output of GATE68 in the calendar module. The output of this gate will be high when the day of month counter should reset and advance to the next month.

I set up my oscilloscope to trigger on a high going edge of the output of this gate, set the clock to 23:59 on Jan 29 and then let it run. The sequence of events is as illustrated below.

As you can see (although the scope probe wasn't connected in these photos) that once the clock advances past 23:59:59 (to what would technically be 24:00:00) the reset logic generates a pulse which causes the day of month counter to reset to 00 and the month counter to advance to the next month.

So I thought about it for some time, and I reckoned all that might be needed would be to add a small capacitor to this reset signal, such that while the ripple counters are sorting out their final state, if the reset logic should happen to generate a pulse during this period, the initial bulk of that pulse should be absorbed by the capacitor such that only once the ripple counters settle down, and if their state represents something which should cause a reset, then the reset signal will remain high for a significantly longer period of time - long enough to fully charge the capacitor and thus allow the reset signal to actually cause a reset.

So I tried this out:

In this photo you see a little grey probe sticking out the left hand side of the clock. This is connected to the output of GATE145 in fact. The orange wire connects to a 1nF capacitor, the brown wire connects to VSS (ground), and the green wire connects to my scope.

As you can see on the face of the clock, this did indeed seem to do the trick, as the calendar had advanced correctly from the 29th to the 30th and remained within the same month, instead of moving to the next month.

I do still see a small rise in the reset signal, I guess 1nF is only just enough to cover this, but at about 1.5V its not high enough to drive any MOSFETs and cause a reset.

For completeness, heres a photo of the newest addition to the clock (the little orange blob), tucked away all nice and cozy in there:

The schematic for the calendar module has also been updated to reflect this recent addition. And with that, I am no longer working on any issues - the clock works and functions as it should. Only time will tell if anything else pops its head up.

• GitHub repo
• calendar-module.sch EAGLE schematic file
• calendar-module.pdf

Clock design log #5 - calendar module

I think circuitry wise the calendar module is my favorite - not only does it contain a bunch of ripple counters, but it also includes a good deal of combinational logic to determine (essentially) which month it is and how many days should be counted in each month, and then generate reset pulses using even more logic. As far as circuitry goes, it kind of seems more interesting than the other modules, perhaps because its not just a bunch of counters. There's a bit of circular dependency going on as well, with the day of month counter taking inputs from the month counter, and a reset of the day of month counter advancing the month counter. To be honest I was surprised it even worked at all - having done plenty of programming in my time and knowing how circular dependencies can cause great headaches...

It also has some nifty circuitry to ensure that when the day of month counter resets it doesn't reset to 0, but 1. This ensures that when the months roll over, they will always start from day 1. That is achieved with an additional input to the first stage of the units counter for the day of month (the only such instance of a preset input in any of my ripple counters), and a small capacitor which continues to hold the preset input low for a period of time brief enough that it will only go high after the reset input has had a chance to go high first. This ensures that the counter will be reset, and then the preset value will be loaded in.

I tested out this theory to make sure that my design was correct using a 2 bit ripple counter implemented on a breadboard.

The yellow trace shows the reset signal generated when the counter represents a value of 3, and the green trace shows the preset signal that feeds the preset input of the first counter stage, which has a delayed rise time caused by a small capacitor (100nF in the photo, but my design uses a smaller value). At the botton you see some digital signals, D0 and D1 which have been formed in to a bus, and which show the value of the ripple counter before, during and after the reset event. So that all seemed to work.

During the build of this module (as usual, this post is coming some time after the actual build) I discovered some flaws in my original design, so as you look over the schematic you'll find some gates towards the beginning of the schematic with numbers that look a bit out of place. This is due to the fact that I didn't want to spend the time to go through the schematic and re-number them all to fit in nicely after determining the fix and then implementing it. Laziness, or time better spent - you decide. In the end I decided to upload the fixed schematics with this post, rather than uploading a revised version with the build log as I have done in the past.

Prior to the fix I had connected the preset input to the output of the reset circuitry for that counter, but I quickly discovered that was simply incorrect, becuase as the counter would advance from 9 to what should have been 10, it would actually go one further to 11. Maybe it was trying to suggest something, or maybe I just stuffed up. So an additional gate connected to the right reset signals was in order, and that became gate 143 on the schematic such that only the correct reset signals would result in the initial preset value being loaded.

There is also another fix that I had to wedge in, and that was to do with the reset circuitry for the day of month counter. You'll see gates 144 and 145 hanging off gate 68 in the large block of gates which point to the left. This was some reset signal surpression that was necessary to ensure that everything worked correctly when manually setting the day of month. Technically you dont need gates 68 and 144 as they simply form a buffer, but I left them in the schematic because that is exactly how my circuit ended up being built (it would have been very fiddly to remove the components so I left them in). Basically, these gates prevent the reset signal from reaching the rest of the circuitry until the month set button is released. Prior to this I found that when the reset signal was asserted, the counter would end up with a value of 2. When I discovered that I started to wonder whether I needed the capacitor at all..? Anyhow..

I also designed in the ability for the clock to handle leap years, I think that was a nice touch even if I'll only use it once every 4 years. Pressing the leap year button inverts the state of a D type flip flop, and both of the outputs of that flip flop are used within the reset circuitry of the day of month counter along with outputs from the month counter which indicates what (kind of) month it is, and that causes the day of month counter to reset appropriately after 28 or 29, 30 or 31 days as appropriate. Whether the clock is in leap mode or not will be indicated on the display board using the dot point of the units display of the day of month. If it's on, it's a leap year and the day of month will reset after 29 days. Otherwise it's a matter of resetting after 30 or 31 days. Leap year mode is automatically cleared when the month counter resets from December to January, so the following year advances normally.

And finally, there is day of the week. This is a 3 bit counter which will translate to a 2 letter day of week readout on the display board, such as MO, TU, WE etc.

As with the time module, each of the buttons is associated with a debounce circuit to prevent multiple advances on a single press due to contact bounce. The little circuit I am using is working really well.

And that about does it for the design of the calendar module. Schematics are linked below as usual!

• GitHub repo
• calendar-module.sch EAGLE schematic file
• calendar-module.pdf

Clock build log #2 - time module

Build

Build for the time module started with the bus connectors that would connect to the analogue board and further in to the stack. The connector for the analogue board is a small 8 way pin header, while the connector for the rest of the stack is a 40 pin socket - but not one with extended pins. Positioning of this board within the stack is somewhat limited, as it connects to the analogue board and in doing so it bridges power and the MCLR, AC_CLK/ and 1PPS_CLK/ signals through to the larger bus. That means this board pretty much has to sit adjacent to the analogue board for the bus connectors to line up properly between the two. It all started off looking so simple:

Then I started what would become a (part time) month and a half long process of soldering in transistors, resistors and jumper wires to create the gates that would form the logic.

While putting the seconds counters together I had an idea - what if I was to use the 1PPM clock signal that it creates to collect some statistics about how much the AC frequency drifts? I thought it would be kind of interesting to see just how wild the AC frequency was, and how often and for how long it drifts around. As part of this I thought it would give a nice insight in to how quickly things might degrade and how long it takes to recover.

So after soldering in the first 56 gates worth of transistors that make up the seconds counters and their reset logic (which generates the 1PPM_CLK/ signal) I branched off in to a small software project.

I had a Raspberry Pi sitting idle that I thought would be perfect for the task, since it has GPIO pins that I could interface to my clock. I knocked together a small program in C that, upon start, takes the current UNIX timestamp when the first 1PPM clock pulse occurrs, and for every subsequent 1PPM clock pulse it records the UNIX timestamp again, substracts the difference, and outputs the timestamp and difference. I piped that information in to a file and then left it to run for about 2 weeks until something started going funny and it started missing pulses and producing erroneous results.

I tried applying some manual fixes to some of the data, and restarted the script a couple of times before it started getting too out of hand and I gave up, sticking with about 11 days worth of solid data, or about 15,700 records. From that I was able to produce the following graph by loading all of the data in to a spreadsheet:

Since I don't know when "zero" is for the AC mains frequency, "zero" for me and my exercise was at UNIX timestamp of 1491865936, which for the non-techies means April 10th 2017 at 23:12:16 UTC (basically it's the number of seconds since January 1st 1970 00:00:00).

This answered two questions:

1. How much the clock may end up drifting - over my sample of data as much as +22 and -35 seconds
2. How quickly or slowly it drifts and recovers - could be as quick as a single day to a couple of days

Re #2, April 17th was a "bad day" where it slowed right down to the slowest it ever got, while we see that over the 13th and 14th we caught up 20 seconds and went 10 seconds faster still.

The most important thing I learned from this was that the clock would probably eventually come back to zero offset. So some of my worries about how reliable the AC frequency was going to be ended up being addressed through that little exercise. At least it wasnt going to be constantly getting slower and slower or faster and faster and never return, though long term results are yet to be seen...

Here are a couple of photos/images showing the setup I used to collect the data for the above graph, and also the schematic for what's happening on the breadboard (a MOSFET level translator, since the Raspberry Pi GPIOs are 3.3V and my clock is 9V):

Funnily enough, this is the first time I have ever used a Raspberry Pi for the thing that I guess it was intended to be used for - tinkering with electronics. My other Raspberry Pi's are running FreeBSD or Linux and acting as servers more or less. :-)

Back to the task at hand... it pretty much just sounds like a broken record from here. Solder, test, repeat. So theres not really much more to say about the build. And it seems I didn't take many more photos during the construction process, so theres a distinct lack of more things to say or show right now. So perhaps the more interesting and juicy bits are in the testing of what I have built.

Testing

Learning from the build of the analogue board, I built each counter stage individually and tested that it worked as expected before moving on to building the next. This meant that solder joints weren't buried under so many wires and hard to get at. It also gave confidence that as I was going along and things were working, if anything happened to stop working further on during the build it might likely be what I was just working on rather than something earlier on. At least then it would be a process of test the most recently built circuit and then start moving backwards if that was checking out OK.

With the seconds counters working just fine, the first major problem I came across was with the minutes counter. When I started building it one of the first things I did was to hook up the button with its debounce circuit so that I could advance that counter manually, instead of having to wait for the 1PPM_CLK/ signal to fire, although testing that the counter advanced on that clock pulse was also important.

Advancing of the units counter was fine, but I noticed an issue when it came to the reset event. The counter would advance, and at 10 count the output of the gate hooked up to the outputs of the 2nd and 4th stages would go high, inverting an SR latch to reset the counter back to 0. But after doing that, the SR latch wouldn't release that reset, so the counter could no longer advance (well, it would have if I had actually hooked up the reset inputs of the counter stages, but I hadn't at that point in time). I traced out all of the wiring I had soldered and realised the issue was not with the design or build of the time module circuitry, but with how the AC_CLK/ signal was handled back on the analogue board.

On the analogue board is a switch that allows you to isolate AC_CLK/ from the clock to stop it advancing the mains frequency prescaler, thus allowing you to reset the clock to all zeros and then set it manually without it carrying on, then set it running once you were ready. But that switch isolated the AC_CLK/ signal from the entire clock, and while it did what I had originally intended it to do, it was breaking another piece of functionality - when an SR latch it set by a connected "AND" gate to cause a reset, it needs the AC_CLK/ signal to release that reset! Ooops. So a quick re-design of the AC_CLK/ circuitry such that the switch only isolates AC_CLK/ from the input of the first counter stage mains frequency prescaler was in order, and luckily that was pretty simple, and required minimal re-work. The revised schematic will be linked below.

Here are some scope traces that show the incorrect (left) vs correct (right) behaviour:

Some explanation of what you're seeing ... the yellow trace is the inverted output of the SR latch on the minutes units counter (i.e. the one that would generate the 1PPH_CLK/ signal in this case). That output clocks the input to the minutes tens counter, hence it has gone from high to low in the left hand image. But you see that, while D4 has gone high (being the first stage of the minutes tens counter), D1 and D3 have not gone low, and the yellow trace remained low.

At the time I took that trace, the switch isolating the AC_CLK/ signal from the clock was set to the "stop" position. If I had set that to "run" things would reset correctly. But it's a bit silly to need the clock to be in "run" mode for this to work correctly, its kind of the complete opposite of what I was aiming for.

After modifying the circuitry around that switch, I took a second capture which is the right hand image you see above. As you can see things are a bit different here. As D1 goes high towards the right hand side of the trace, it remains high only for a very brief period of time, hence you just see a "glitch" in the trace for that signal. When it does go high, and with D1 and D3 representing 2+8, that generates the reset condition for that counter, so D4 goes high. You also see a very brief glitch in the yellow trace - thats the SR latch being set by the "AND" gate of the minutes unit counter and then being reset by the AC_CLK/ signal, and all with the switch being set to "stop". So that was now sorted and working correctly.

For interest sake, here's an even closer look at whats happening when the reset of the minutes unit counter occurrs, which pretty well matches the description that I gave in design notes #5 - it's nice to know that my theory turned in to reality 😎 :

At the bottom you see that I have grouped the digital channels in to some busses, B0 (D0-D3) and B1 (D4-D6), and they are displaying what you'd see on the display if the decoders were built. As we lead in from the left, we're at 09, and at the next button press D0 goes low (and for a very brief period of time we go backwards to 08) until D1 goes high and we get to a value of "A" in base 16 (aka hexadecimal) which equals 10 in our every day base 10/decimal system. At that point the reset condition is created and after 20 odd uS D1 and D3 go low while D4 goes high, and now we see that the value on the display would be 10.

Fortunately there were no more surprises that needed fixing in the build from here on, and things went smoothly. All counters worked as expected, and all clock pulses were generated fine and dandy. Somewhat boring, but also somewhat relieveing since ...

End result

... heres what I ended up with!

That was quite a bit of work!

But most importantly it works, much much much to my relief. Along with producing the 1PPD_CLK/ signal when rolling over from 23:59:59 to 00:00:00, it also produces 1PPM_CLK/ and 1PPH_CLK/ signals, although at the moment there is nothing planned that uses these, except perhaps 1PPH_CLK/ which might be used by another module to do display dimming/blanking.

Luckily I think this will be one of, if not the busiest board in the clock - it's certainly going to be one of the most densely populated. I anticipate that the calendar decoder board might be pretty complex/dense too, but we'll have to wait and see what that one brings (designs still pending at this stage...)

In closing, along with the revised analogue board schematic, I've uploaded the C source for the program I wrote to capture data for the graph, and I have also uploaded an updated schematic for the analogue board with the fix for the AC_CLK/ generator. C is not my every day programming language, so please be gentle (and you'll need wiringPi to make it work).

• offset.c C program to collect data when 1PPM clocks
• analogue-board.sch EAGLE schematic file
• analogue-board.pdf

Clock build log #1 - analogue board

Alright, alright already! I know this is what some people have been waiting for - actual freakin' construction. So I'll just get straight to it.

Build

I started with construction of the power supply, since that was going to be rather necessary. For a slight retro look I decided to use an axial capacitor as the smoothing capacitor on the input to the linear regulator. The full bridge rectifier has a 100nF capacitor in between the DC terminals as I read this can be good for filtering noise at the output of the rectifier. This then feeds via the smoothing cap in to the 7809 regulator. Another 100nF cap at the output of the regulator helps to filter noise on the output here too (not that I expect that much noise, but it can't hurt) and the positive rail then feeds through a PTC (fuse) for short circuit protection - theres a lot of fiddly wiring all done by hand, and if I happen to short something out or it breaks loose, I don't want anything getting damaged.

It was hard to find a 2.1mm DC jack that wouldn't need me to drill in to my board to create slots that the terminals could be inserted in to for soldering (i.e. nothing with pins on a 0.1" grid), but I had a flash of inspiration and figured that a surface mount jack with some loops of wire over the top of its terminals could be used to make my own "through hole" socket, so thats what I did, and this is what I ended up with for the power supply:

Quite pleased with that result, and the DC jack feels very solid. I used the off cuts of the rectifier diode leads to form the loops as they were quite hefty.

Next came a sprinkling of other building blocks and bits and pieces: the power on reset RC circuit, the AC_CLK/ generator and isolation switch, reset button, and a handful of transistors forming a few gates of logic to glue all of that together.

Soldering all of that in to place, the board now looked like this:

Then I started soldering even more transistors to start building the prescaler.

And around 100 transistors later and after much effort soldering the many many fiddly wires, I was left with this:

The 8 pin socket on the back forms a bus to the next board to carry power and the MCLR, AC_CLK/ and 1PPS_CLK/ signals over.

Confession: construction of the analogue board started in early March, and a couple of things have changed in between then and now, including the AC_CLK/ generator and some circuitry related to the reset button which provides some (better) hardware debouncing. So if you're wondering why perhaps the components you see don't quite line up with the schematics I posted recently, that would explain it. The schematics represent how it has actually been (finally) built.

The thinner blue, red and black wires are 0.22mm² or 30AWG Kynar insulated wires. They are also sometimes known as "wire wrapping wire".

And here's another slightly artsy shot, because I couldn't help myself.

So yeah, thats about it for construction of the analogue board. The next step was to test it and figure out if this was going to work. And this was where I learned my first lesson: don't build too much all at once.

Testing

Testing the power supply was easy with a multimeter, and the results predictable. 12V AC in, 22V after rectification and smoothing, fed in to the linear regulator which produced 9V DC out.

The power on reset delay circuit was also easily testable by probing the drain pin of its transistor while applying power. When power is applied you see the output high, and after a short period of time while the capacitor charges up it goes low.

That feeds in to the reset mux, along with the reset button, so testing that was a matter of probing the output of the inverter that sits after the NOR gate to which the power on reset delay and reset button are connected. The inverter outputs a positive logic signal referred to as MCLR, and it is high when ever the reset button is pressed, or when the capacitor in the power on reset delay circuit is charging just after power on. That also checked out OK.

Testing the AC_CLK/ generator is a little more complex, I really need to see the waveform it produces alongside the input AC sinewave since it happens too fast to see on a multimeter, so this needed an oscilloscope.

Backstory: I'd been hoping to win something as part of Keysight's Oscilloscope month promotion, but alas I wasn't so lucky. Towards the end I caved and ended up buying myself a Keysight "premium used" MSOX2024A which ended up costing me about half what a brand new model would have cost. I have to say I'm incredibly impressed so far, it looks for all the world like a brand new scope, and came with 5 years warranty. Highly recommend you check out this route first if you're looking to buy a Keysight... So yes, some of the material for this post dates back to then and even earlier. I also started a new job within the past month and a bit, so that has kind of slowed things down a bit too.

Using my newly acquired scope I took some measurements of the rise and fall times of the output of the AC_CLK/ generator to see what its waveform looked like. A very early version of this part of the circuit used a BC547 and I wasn't happy with the output, in particular that the output voltage was only around 4.5V which was uncomfortably close to the gate threshold voltage. So to be safe I re-designed it to use a MOSFET instead, and was much happier with the output voltage at 9V. The rise and fall times were nice and sharp at no more than about 15uS (microseconds) at worst (fall time was just a smidge over 3uS) - also much faster than the original BC547 based circuit. Here are some scope traces of that:

That is of course looking at the time taken to transition between ground and peak voltage - actual "apparent" rise/fall time would be much quicker, because the transistors don't need the full 9V to turn on or off, that happens closer to 4.5V.

The output of the AC_CLK/ generator feeds in to a NOR gate along with the MCLR signal, such that when ever MCLR is asserted the AC_CLK/ signal is held high. When MCLR is not asserted, the AC_CLK/ signal is free to pass through and oscillate. So testing that was a matter of probing the output of the inverter that forms the output of that logic. This was also easy to test with a multimeter in frequency mode, I got ~50hz, so that is all working fine.

On a roll I thought.

Then it came to the prescaler. Was I going to get lucky and was it going to work at all? All of the simulations and breadboarding said it should, but now we're in the real world and at bigger scale.

I used my multimeter in frequency mode to probe the output of the first stage. It was giving me 25hz which is what I expect (since every stage of the prescaler acts as a divide by 2 of its input frequency. The second stage was giving me 12.5hz - brilliant. And the 3rd stage was giving me 6.25hz. "Amazing" I'm thinking, "did I get this all in one go?"

Then I probed the output of the 4th stage and nothing. Zip. Nada. Ah crap, my luck had run out. Since I didn't actually have my oscilloscope at this point in time, I fabricobled a crude logic probe out of an LED, transistor, and a couple of resistors to allow me to get a quick visual from the circuit:

If you would like to build your own, here's the schematic of what I built (you may need to adjust the value of R2, I picked 2.2K to work with the 9V of my power supply, but other logic circuits might need something a little less, perhaps 1K for a 5V circuit for example):

Using this I probed around at the various inputs and outputs to the 4th stage to see where the signal was getting lost, and it just happened to be a solder joint that hadn't properly wetted from the source pin of one transistor to the ground rail. A tiny bit of reflow sorted that right out, and the 4th stage was giving me 3.125hz. Bingo. That was where lesson #1 was learned, because the solder joint was buried in between the wiring, it was a bit tricky getting down in there to make the repair. For future portions of the build I decided I would build and test counter stages one by one to avoid problems like that.

The 5th stage gave me 1.5625hz, and finally somewhere around 0.78125hz at the 6th stage (logical conclusion, because my multimeter can't display that many decimal places). Great, they are all working.

I then proceeded to hook the second, fifth and sixth stages (which represent BCD values of 2+16+32) up to the 3 input NOR gate which feeds the reset circuitry, and probed the non-inverting output of the SR latch, and what do you know, I was getting a very brief pulse once every second.

And with that, the analogue board is complete! It is producing all of the necessary clocking and other signals that are needed to move forward to the next board which will be the "time module", counting seconds, minutes and hours, so I hopefully shouldn't need to pay any more attention to this one.