[harrison]

I think whatever works, works! If you want to have a long cable in there and it seems happy with it then go with that.

[harrison]

Hi Avik,
I am not too surprised by this. The problem is that the digital communications (which use I2C protocol) are sensitive to the capacitance of the connection (i.e. wires) between the differnt components. The longer cable has more capacitance (there is more copper there – so it takes longer for the system to drive the digital signal high/low) and hence the digital signal has slower rise/fall time when the longer cable is used. When the cable is TOO long this rise time becomes too slow, meaning the chip on the pump is unable to “see” the digital signal properly – it gets corrupted.

I think in later versions of the device we will work out additional mechanisms for buffering this connection so that it is possible to reliably use longer cables on the pump side. But for now it seems sensitive to very long connections (though the software does make it a bit more robust in some cases).

[harrison]

It should really only be the PCB.
You say you don’t see ANY putty window. That is very strange – if the device crashes it should not be possible for it to close the putty window. The window should stay open at all times, even if you unplugged the control computer from your PC (Though it would throw up an error message in Putty).

If Putty is literally gone, are you sure your PC isn’t restarting or crashing somehow? For example, is it a Windows computer restarting itself to install updates (which happens painfully often if you leave them running overnight!)

If the device itself is crashign then the putty window should remain open and give error messages.

[harrison]

Potentialy yes. Though I am a little confused – if the control computer doesn’t work beyond one reactor at a time, how is it that you have multiple reactorS running such that they can stop working?

What does the PuTTY window say the next day when they stop working?

As to the cause: If the control computer is faulty then that is the most likely. Beyond that, the next most probable reason could be that you are getting liquid on the moisture sensing track at the top, which crashes the software. Are you running it in Turbidostat mode, or something else?

As an aside, if you are having any issues like this I would recommend updating to the V2.0 Software (see the Software page on this site) which makes it much more robust.

[harrison]

Thanks Shilan,
So based on what you said it seems that the reactors are fine – since all of them work if plugged into M0. However, you are saying that all of the other ports (i.e. M1-M7) are not working for unclear reasons. Is my understanding correct?

If that is the case I would say the most likely cause is a problem with the soldering of the Multiplexer chip, which is one of the circuits on the Control PCB (i.e. the PCB that has the M0-M7 ports on it).

Labmaker should be able to replace it for you if that is the case. Send them an email describing the problem and link them to this thread. They should be able to fix a replacement, they have been very good at this in the past!

[harrison]

Hi Avik,
Have you got the most recent software installed?
In particular, on this page: https://chi.bio/software/ there is a new Software V2.0 setup guide which should fix this.
The issue is probably that with your current software version it can take a while to recover from some communication issues. The updated software fixes this by updating the Linux Kernel to change how it handles communications errors.
Let me know how it goes!

[harrison]

OK, I assume these devices are all individually powered on with the wall plug?

Can you try and see what happens if you connect one reactor to M0 and then a second to M2? Or M0 and M3? etc.

What if you connect different reactors to M0? Do only some of the reactors work (while others don’t)?

I want to diagnose whether there is a problem with the control computer (i.e. one of the plugs is broken on it), or if there is a problem with one of the individual reactors.

[harrison]

Hi Shilan,
Can you explain what error you get? Perhaps share the log in PuTTy?
It should be that if you connect all the reactors up to the control computer via USB, and power all the reactors on, then you can access them all in the UI.

Harrison

[harrison]

Have emailed you 🙂

[harrison]

Yeah, i agree cabling isa pain and something I have thought about changing. At present there isn’t too much to be done since (even if data was wireless) we still need power transmitted between them.
One option I thought of was to have a “base” to which reactors/pumps attached. Thus, all power/communication would be routed through some table top-size structure and you just plug in the pumps/reactors to the top of this. But, would take a lot of design/manufacturing and potentially could create new failure modes (e.g. liquid spills could become more damaging…)

[harrison]

Hi Melo,

That error message would occur if you try to turn anything on when you have the GUI set to a device that is not at present plugged in and powered on.

Are you able to turn on/off LEDs and stirring for the reactor, but NOT pumps when they are plugged in to that reactor?

Which port in the control computer is your reactor connected to? Is this the same port you have selected in the M0-M7 buttons in the GUI? If the answer is not M0 for both try setting both to M0.

Do all your reactors have pumps plugged in to them?

[harrison]

Good ideas!

I think the pump board USB connectors have been updated after your past orders. Labmaker found a different variant which seems to hold on stronger.

The temperature sensor idea is good, though it means we would have one per reactor rather than one per control computer. Originally I put the sensor on the end of a long sticking-out part of PCB in an attempt to thermally insulate it, but as you said it doesn’t work that well. Often it is as much as ~7-10C higher than it should be…

The feedback on stirring/heating could potentially work. I think it would be feasible to tell if the motor is rotating or not, but also figuring out if there is a stir bar there would be tough… Are you hoping that the motor will draw a significantly different current if it is/isn’t interacting with a stir bar? I suppose there will be SOME change in current draw, but it might be of a small magnitude and hence difficult to distinguish between inter-device variability in that component (fan current draw). If we want to implement this could add a low-value resistor downstream of each of them and then measure voltage across it (and hence current) with an ADC (which would need to be added, there are cheap/simple I2C ones that would fit).

For the water sensor change is the primarily purpose you imagine to ensure that we can always diagnose when this has caused the crash? (versus some other possible hardware issue)

[harrison]

If I understand correctly, the system keeps ticking over and doing measurements, and it thinks the pumps are going fine, but they have actually stopped?

Indeed this is something unlike any issues I have seen. What version of the code/software are you using? Is there anything else non-standard running on your beaglebone?

My best guess at a software reason would be if somehow the pump modulation threads are failing somehow, or the system is reaching some maximum in number of threads, but I don’t know why this would be occuring in your case. It has never been an issue for ours (or as far as I know, for other peoples’). Perhaps you can try putting a print statement into the pumpmodulation (or after that, into the setPWM function) which will print something out when the device attempts to communicate via I2C with the pumps? THat way you would be able to see if there was some software issue (I.e. the communications to the pumps are not even being attempted after the freeze)

In terms of hardware, the pumps are “downstream” of an I2C repeater in each reactor, which potentially could be locking up somehow. I recently made a new software setup V2.0 (see the software page) that can overcome some such errors. But, it seems unlikely that this would be the cause of your problem since all three reactors’ pumps are locking up at the same time (at least I think this is the case from your message?), which would be highly unlikely if it was a reactor-specific hardware fault.

Potentially if it is a hardware issue the code could be adjusted to trigger the watchdog circuit intermittently (which powers down the reactors, which happens when you restart the code), but this wouldn’t solve it if the threading was at fault.

[harrison]

Hi Wilson,

You say you provided 3.3V where it “should be giving it”. Does this mean you are trying to replicate the watchdog signal?

0x60 is indeed one of the PWM chips. If I recall correctly 0x70 is a second I2C address that is reserved by the pwm chip, but we don’t communicate with it directly.

0x1b is the thermometer on the beaglebone control board. You can see what every address corresponds to looking at lines 346-358 (and line 383 for the multiplexer) in the app.py file (current version on github: https://github.com/HarrisonSteel/ChiBio/blob/master/app.py)

I am not sure of a solution to your last point. This is not something I have seen or heard of happening before. I must say at this point it seems you are chasing after some tricky hardware issues – which ultimately should not be solved by Labmaker.

I’ll upload the latest version of the schematics to the site now…

[harrison]

OK. If you have a SD card can you follow version 2 of the software setup guide? This will completely flash the beaglebone with both operating system and software. In theory it doesn’t require you to log into the beaglebone, just power it on with the SD card inserted. Then, you can remove the SD card and connect it as before.

[Author]

Posts