• Battery Powered Project

    From Simple Simon@3:770/3 to All on Tue Jan 12 12:14:24 2021
    I am working on a battery powered car and want the Pi to shut down automatically if the battery starts to go flat to try to prevent SD card corruption. I am a beginner to bash scripts! I will run this via crontab...

    #!/bin/bash
    powerstatus=$(vcgencmd get_throttled)
    if [ $powerstatus="throttled=0x1" ]
    then
    echo Under Voltage Detected - Shutting Down
    sudo halt
    else
    echo Voltage Normal
    fi

    Obviously it is not working!! Could someone correct and explain for me
    please.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Chris Green@3:770/3 to Simple Simon on Tue Jan 12 12:40:23 2021
    Simple Simon <please.dontspa@me.com> wrote:
    I am working on a battery powered car and want the Pi to shut down automatically if the battery starts to go flat to try to prevent SD card corruption. I am a beginner to bash scripts! I will run this via crontab...

    #!/bin/bash
    powerstatus=$(vcgencmd get_throttled)
    if [ $powerstatus="throttled=0x1" ]
    then
    echo Under Voltage Detected - Shutting Down
    sudo halt
    else
    echo Voltage Normal
    fi

    Obviously it is not working!! Could someone correct and explain for me please.

    You need spaces in the test:-

    if [ $powerstatus = "throttled=0x1" ]

    Without the spaces you are just testing if '$powerstatus="throttled=0x1"'
    is not an empty string, and it never will be an empty string.


    --
    Chris Green
    ·

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Simple Simon@3:770/3 to Chris Green on Tue Jan 12 12:50:36 2021
    Chris Green wrote:
    Simple Simon <please.dontspa@me.com> wrote:
    I am working on a battery powered car and want the Pi to shut down
    automatically if the battery starts to go flat to try to prevent SD card
    corruption. I am a beginner to bash scripts! I will run this via crontab... >>
    #!/bin/bash
    powerstatus=$(vcgencmd get_throttled)
    if [ $powerstatus="throttled=0x1" ]
    then
    echo Under Voltage Detected - Shutting Down
    sudo halt
    else
    echo Voltage Normal
    fi

    Obviously it is not working!! Could someone correct and explain for me
    please.

    You need spaces in the test:-

    if [ $powerstatus = "throttled=0x1" ]

    Without the spaces you are just testing if '$powerstatus="throttled=0x1"'
    is not an empty string, and it never will be an empty string.


    Brilliant - many thanks - works a treat - I was almost there!!

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From A. Dumas@3:770/3 to Simple Simon on Tue Jan 12 15:15:35 2021
    On 12-01-2021 13:50, Simple Simon wrote:
    Brilliant - many thanks - works a treat - I was almost there!!

    It won't matter here, I guess, and you needed it for debugging, but
    normally never output to the console from a background script. So
    perhaps remove the echos now that it works.

    Slightly more importantly, it's good practice to put variables in quotes
    when testing them (the part between brackets) because you never know
    what may be inside. If there's a space, or the variable is empty, your
    script will break.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Vincent Coen@2:250/1 to Simple Simon on Tue Jan 12 14:35:04 2021
    Hello Simple!

    Tuesday January 12 2021 12:14, you wrote to All:

    I am working on a battery powered car and want the Pi to shut down automatically if the battery starts to go flat to try to prevent SD
    card corruption. I am a beginner to bash scripts! I will run this via crontab...

    #!/bin/bash
    powerstatus=$(vcgencmd get_throttled)
    if [ $powerstatus="throttled=0x1" ]
    then
    echo Under Voltage Detected - Shutting Down
    sudo halt
    else
    echo Voltage Normal
    fi

    Obviously it is not working!! Could someone correct and explain for me please.

    You have another 2/3 possible issues :-

    1. crontab will only run the script at the defined point of time so you need to run it often OR better still as a subtask from a routine that WILL run if the battery is low at a given point (not almost flat) to shutdown the system , NOTE the term shutdown by using similar to shutdown -h now and not halt which may well not close down any processes running first and possbly cause curruptions on your system disk / SD etc.

    2. As a follow on from 1 , you need a low level process that is constantly running say with a sleep of nn seconds that tests the battery state and if good
    sleeps or runs process similar to your bash script but not it and suggest say
    a little C process as it would use less resources.

    3. As a follow on from both of the above.
    You should check what battery setting you have for throttled and have it set to
    at least 25% battery power if you have a lot of proceses that can be running at
    any one time and if needed a higher value.

    I have a similar set of processes but linked to a APC UPS so see package apcupsd.

    Vincent

    --- Mageia Linux v7.1 X64/Mbse v1.0.7.17/GoldED+/LNX 1.1.5-b20180707
    * Origin: Air Applewood, The Linux Gateway to the UK & Eire (2:250/1)
  • From Pancho@3:770/3 to Simple Simon on Tue Jan 12 16:40:05 2021
    On 12/01/2021 12:14, Simple Simon wrote:
    I am working on a battery powered car and want the Pi to shut down automatically if the battery starts to go flat to try to prevent SD card corruption. I am a beginner to bash scripts! I will run this via crontab...

    #!/bin/bash
    powerstatus=$(vcgencmd get_throttled)
    if [ $powerstatus="throttled=0x1" ]
    then
    echo Under Voltage Detected - Shutting Down
    sudo halt
    else
    echo Voltage Normal
    fi

    Obviously it is not working!! Could someone correct and explain for me please.

    Not being much help, but...

    I'm aware of Under Voltage problems from entries in journalctl. It seems
    to me that listening for events logged to journalctl would be a pretty
    common thing to do. So common that there should be some standard way of
    doing it. Some kind of standard Observer pattern on jounalctl. Does
    anyone know of one?

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Theo@3:770/3 to Simple Simon on Tue Jan 12 21:43:03 2021
    Simple Simon <please.dontspa@me.com> wrote:
    Brilliant - many thanks - works a treat - I was almost there!!

    I'm curious... does this actually work for low battery detection?

    Unless you're wiring the Pi directly across a battery that's giving out somewhere around 5V (for example a 6v lead acid), most of the time you'll
    have a voltage regulator between you and the battery. That regulator will
    aim to keep providing 5V as the battery is going flat. When the voltage
    sags below 5V it means the battery is so empty it can't maintain that, which means it's in a very steep part of the voltage decline.

    I'd expect it to be so steep you don't get enough time to do any meaningful shutdown, but I could be wrong. What battery setup do you have and how does
    it work out in practice?

    Theo

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Jan Panteltje@3:770/3 to theom+news@chiark.greenend.org.uk on Wed Jan 13 09:40:12 2021
    On a sunny day (12 Jan 2021 21:43:03 +0000 (GMT)) it happened Theo <theom+news@chiark.greenend.org.uk> wrote in <7ul*5x4-x@news.chiark.greenend.org.uk>:

    Simple Simon <please.dontspa@me.com> wrote:
    Brilliant - many thanks - works a treat - I was almost there!!

    I'm curious... does this actually work for low battery detection?

    Unless you're wiring the Pi directly across a battery that's giving out >somewhere around 5V (for example a 6v lead acid), most of the time you'll >have a voltage regulator between you and the battery. That regulator will >aim to keep providing 5V as the battery is going flat. When the voltage
    sags below 5V it means the battery is so empty it can't maintain that, which >means it's in a very steep part of the voltage decline.

    That is a valid argument for say detecting a low mains voltage.
    It does not however cover a fault in the regulator itself.

    I like the idea as extra test for fault conditions
    like too much voltage drop over USB cable by high loads etc..
    You can make it log the error, or switch off high load / routines first
    before shutdown, sound an alarm, etc.
    A very common error in wallwart supplies is bad filter caps on the output, causing a high ripple on the 5V, this could perhaps detect that.
    Did no know about the low voltage check option.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From The Natural Philosopher@3:770/3 to Pancho on Wed Jan 13 13:56:52 2021
    On 12/01/2021 16:40, Pancho wrote:
    On 12/01/2021 12:14, Simple Simon wrote:
    I am working on a battery powered car and want the Pi to shut down
    automatically if the battery starts to go flat to try to prevent SD
    card corruption. I am a beginner to bash scripts! I will run this via
    crontab...

    #!/bin/bash
    powerstatus=$(vcgencmd get_throttled)
    if [ $powerstatus="throttled=0x1" ]
    then
    echo Under Voltage Detected - Shutting Down
    sudo halt
    else
    echo Voltage Normal
    fi

    Obviously it is not working!! Could someone correct and explain for me
    please.

    Not being much help, but...

    I'm aware of Under Voltage problems from entries in journalctl. It seems
    to me that listening for events logged to journalctl would be a pretty
    common thing to do. So common that there should be some standard way of
    doing it. Some kind of standard Observer pattern on jounalctl. Does
    anyone know of one?

    Well that's just another problem with systemd. When coming up with
    something similar in Apache, I could tell apache to log to a process
    that tripped certain (security relevant) events.

    There ought to be a way to tell systemd to do similar so you can have a watchdog attached...but who knows how to do it?



    --
    “Puritanism: The haunting fear that someone, somewhere, may be happy.”

    H.L. Mencken, A Mencken Chrestomathy

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Martin Gregorie@3:770/3 to The Natural Philosopher on Wed Jan 13 15:12:34 2021
    On Wed, 13 Jan 2021 13:56:52 +0000, The Natural Philosopher wrote:

    On 12/01/2021 16:40, Pancho wrote:
    On 12/01/2021 12:14, Simple Simon wrote:
    I am working on a battery powered car and want the Pi to shut down
    automatically if the battery starts to go flat to try to prevent SD
    card corruption. I am a beginner to bash scripts! I will run this via
    crontab...

    #!/bin/bash powerstatus=$(vcgencmd get_throttled)
    if [ $powerstatus="throttled=0x1" ]
    then echo Under Voltage Detected - Shutting Down sudo halt else echo
    Voltage Normal fi

    Obviously it is not working!! Could someone correct and explain for me
    please.

    Not being much help, but...

    I'm aware of Under Voltage problems from entries in journalctl. It
    seems to me that listening for events logged to journalctl would be a
    pretty common thing to do. So common that there should be some standard
    way of doing it. Some kind of standard Observer pattern on jounalctl.
    Does anyone know of one?

    One way to get a rapid notification of entries written to a log is the
    use 'tail' in a script.

    Running "sudo tail -f /var/log/messages" from a console shows you each
    message as it is written to /var/log/messages, so a script started a boot
    time to run with sufficient privilege to read those log entries could
    easily use tail to read messages as they are written and filter out the
    ones of interest with grep or awk and use them to , for instance force a
    clean shutdown on low battery, something like

    #!/bin/bash
    tail -f /var/log/messages | awk <<ENDPROG
    /Under voltage/ { shutdown -h NOW }
    ENDPROG

    In this example tail is reading the logfile and piping messages into awk, which as running an awk script from a 'here document', i.e. the awk
    script is all the lines from the line after <<ENDPROG and before the
    'ENDPROG line.

    As messages are written to the logfile, tail reads and passes them to awk, which shuts the Pi down if it gets passed a line containing 'Under
    voltage' and ignores all other lines. Onve the script is written and
    tested, put it in /usr/local/bin

    tail and awk (aka gawk) is a standard Linux utility that IIRC is
    installed by default as part of the Raspbian toolset. Its manpage is also
    its manual and describes the awk language - there's also an O'Reilly
    book, "sed & awk" that contains the awk language specification plus a
    tutorial with examples of doing quite complex things with it, such as
    building a sorted list of all the words in a document.

    The stuff I've shown above is pretty much the standard way of running a
    process in the background iunder any flavour of Linux.

    Well that's just another problem with systemd.

    Systemd isn't the issue here because the example script I've shown will
    run equally well under systemd or the old SysV init.

    The only difference is in setting things up so the script gets launched
    at boot time.

    - Under systemd you'd define a service to manage it.

    - Under sysV init you'd write a control script, put that in /etc/init.d
    and then hack around with with the contents of the /etc/rc?.d
    directories to specify the runlevels at which its started and those at
    which its not.


    --
    --
    Martin | martin at
    Gregorie | gregorie dot org

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Jim H@3:770/3 to theom+news@chiark.greenend.org.uk on Wed Jan 13 18:02:51 2021
    On 12 Jan 2021 21:43:03 +0000 (GMT), in <7ul*5x4-x@news.chiark.greenend.org.uk>, Theo <theom+news@chiark.greenend.org.uk> wrote:

    Simple Simon <please.dontspa@me.com> wrote:
    Brilliant - many thanks - works a treat - I was almost there!!

    I'm curious... does this actually work for low battery detection?

    Unless you're wiring the Pi directly across a battery that's giving out >somewhere around 5V (for example a 6v lead acid), most of the time you'll >have a voltage regulator between you and the battery. That regulator will >aim to keep providing 5V as the battery is going flat. When the voltage
    sags below 5V it means the battery is so empty it can't maintain that, which >means it's in a very steep part of the voltage decline.

    I'd expect it to be so steep you don't get enough time to do any meaningful >shutdown, but I could be wrong. What battery setup do you have and how does >it work out in practice?

    Yes, except...

    At the low current draw of the Pi, unless the battery is rather small
    a cutoff of 5 V for a 6 V battery is 1.67 Volts per cell (Vpc) and
    that's way too low if you want to prevent premature failure due to overdischarge.

    You might get away with this long term if the battery capacity is low
    to begin with because then 1.67 Vpc might be say 80 - 90% discharged.
    But for a larger battery, say an automotive starting battery, a
    voltage of 1.67 Vpc at the current rate drawn by a Pi is way
    overdischarged and the battery will experience short life.

    The upside to all of this is that there probably won't be many
    outages. If regular outages are expected, then a much higher cutoff
    voltage needs to be set to prolong battery life... the bigger the
    battery the higher the cutoff.

    --
    Jim H

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Joe@3:770/3 to Theo on Wed Jan 13 22:04:22 2021
    On 12 Jan 2021 21:43:03 +0000 (GMT)
    Theo <theom+news@chiark.greenend.org.uk> wrote:

    Simple Simon <please.dontspa@me.com> wrote:
    Brilliant - many thanks - works a treat - I was almost there!!

    I'm curious... does this actually work for low battery detection?

    Unless you're wiring the Pi directly across a battery that's giving
    out somewhere around 5V (for example a 6v lead acid), most of the
    time you'll have a voltage regulator between you and the battery.
    That regulator will aim to keep providing 5V as the battery is going
    flat. When the voltage sags below 5V it means the battery is so
    empty it can't maintain that, which means it's in a very steep part
    of the voltage decline.

    I'd expect it to be so steep you don't get enough time to do any
    meaningful shutdown, but I could be wrong. What battery setup do you
    have and how does it work out in practice?


    Lithium cells have a linear-ish discharge curve, from 4.2V down to
    around 3V. Any sensibly designed lithium battery will cut off its
    output at the chosen lower bound, because if it completely discharges,
    it's dead forever. Not a 'steep' decline, a 'fall off the wall' decline.

    It's a good idea to have some independent means of anticipating this
    point.

    --
    Joe

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Theo@3:770/3 to Joe on Wed Jan 13 23:37:18 2021
    Joe <joe@jretrading.com> wrote:
    Lithium cells have a linear-ish discharge curve, from 4.2V down to
    around 3V. Any sensibly designed lithium battery will cut off its
    output at the chosen lower bound, because if it completely discharges,
    it's dead forever. Not a 'steep' decline, a 'fall off the wall' decline.

    It's a good idea to have some independent means of anticipating this
    point.

    But lithium ion cells don't work for this application. We need 5V.
    One cell is too little, two cells is too much.
    One cell would start at 4.2V which is too low to reliably run a Pi.
    Two cells would start at 8.4V which is dangerously high for the Pi, and the proposed 5V cutoff would only come in below 2.5V per cell which is bad for
    the cells.

    As mentioned 6V lead acid doesn't work for a starting battery - it might be
    OK for a 'leisure' battery that is fine with deep discharge, but again we're
    on a part of the curve where the battery is absolutely flat and declining
    fast.

    Possibly LiFePO4 might be better, but a pair of cells at 3.6V fully charged
    is still too dangerous for the Pi.

    4x alkaline cells might just about manage it - 1.6V per cell when new, ie
    6.4V total, which is a bit dangerous but maybe OK. A gradual decline - I
    don't know where the Pi low voltage detection is but flat would be about
    1.1V per cell or 4.4V total which is probably detectable. So if you don't
    mind throwing away cells at 20% capacity it might just work.

    Theo

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Martin Gregorie@3:770/3 to Theo on Thu Jan 14 00:43:27 2021
    On Wed, 13 Jan 2021 23:37:18 +0000, Theo wrote:

    As mentioned 6V lead acid doesn't work for a starting battery - it might
    be OK for a 'leisure' battery that is fine with deep discharge, but
    again we're on a part of the curve where the battery is absolutely flat
    and declining fast.

    In the soaring world we use 12v SLA (sealed lead-acid) batteries to power
    our radios, flight recorders FLARM and and various navigation systems.
    Most instrument panels draw 400-800 mA when fully powered up, so a 7AH
    battery is good for flights of over 7 hours.

    I use Yuasa NP7-12 batteries. This is a 12v 7.2Ah sealed battery, so can
    be installed any way up and is also sold for use in small business UPSs.

    In my glider I use a 12v->5v 3A DC converter (under 6 quid from eBay,
    where they are sold for blinging up cars with festoons of LEDs) for
    powering instruments which want a 5v supply. Mains chargers which shut
    off or switch to float mode at full charge are available for these
    batteries.

    If you go this way, and want to implement low-voltage shutdown for the
    Pi, connect your low-voltage detector inputs across the battery and set
    the shut-down point to 10.5v.

    If the low-voltage detector is a Pi hat, make a voltage divider from a
    68K and a 27K resistor, put that across the battery and connect leads
    from each side of the 27K resistor to the low-voltage detector: when the battery is drawn down to 10.5 volts, there will be 3 volts at the low
    voltage detector. Or use a 10K linear potentiometer and set it to output approximately 3 volts.


    --
    --
    Martin | martin at
    Gregorie | gregorie dot org

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From The Natural Philosopher@3:770/3 to Theo on Thu Jan 14 03:59:18 2021
    On 13/01/2021 23:37, Theo wrote:
    Joe <joe@jretrading.com> wrote:
    Lithium cells have a linear-ish discharge curve, from 4.2V down to
    around 3V. Any sensibly designed lithium battery will cut off its
    output at the chosen lower bound, because if it completely discharges,
    it's dead forever. Not a 'steep' decline, a 'fall off the wall' decline.

    It's a good idea to have some independent means of anticipating this
    point.

    But lithium ion cells don't work for this application. We need 5V.
    One cell is too little, two cells is too much.
    One cell would start at 4.2V which is too low to reliably run a Pi.
    Two cells would start at 8.4V which is dangerously high for the Pi, and the proposed 5V cutoff would only come in below 2.5V per cell which is bad for the cells.

    As mentioned 6V lead acid doesn't work for a starting battery - it might be OK for a 'leisure' battery that is fine with deep discharge, but again we're on a part of the curve where the battery is absolutely flat and declining fast.

    Possibly LiFePO4 might be better, but a pair of cells at 3.6V fully charged is still too dangerous for the Pi.

    4x alkaline cells might just about manage it - 1.6V per cell when new, ie 6.4V total, which is a bit dangerous but maybe OK. A gradual decline - I don't know where the Pi low voltage detection is but flat would be about
    1.1V per cell or 4.4V total which is probably detectable. So if you don't mind throwing away cells at 20% capacity it might just work.

    Theo

    switch mode regulator will work with any of them.

    Given that you want around a one amp capability

    https://hobbyking.com/en_us/turnigy-1800mah-2s-12c-lipo-receiver-pack.html

    is a suitable pack

    and

    https://hobbyking.com/en_us/turnigy-hv-sbec-5a-switch-regulator-8-42v-input.html

    is a suitable regulator.

    HOWEVER that is still only half the story, as, presumably, you want to
    run it off mains as well - or do you?

    My point here is that a charger that will both charge the pack correctly
    AND have a spare amp to drive the Pi is not so easy.

    If you are happy to go and change batteries every so often, well then presumably you will want a much bigger battery.

    Anyway hobbyking is the go-to place for chargers regulators and
    batteries - ship world wide, and product is good quality.


    --
    Future generations will wonder in bemused amazement that the early
    twenty-first century’s developed world went into hysterical panic over a globally average temperature increase of a few tenths of a degree, and,
    on the basis of gross exaggerations of highly uncertain computer
    projections combined into implausible chains of inference, proceeded to contemplate a rollback of the industrial age.

    Richard Lindzen

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From The Natural Philosopher@3:770/3 to Joe on Thu Jan 14 03:46:08 2021
    On 13/01/2021 22:04, Joe wrote:


    Lithium cells have a linear-ish discharge curve, from 4.2V down to
    around 3V.

    They do.

    Any sensibly designed lithium battery will cut off its
    output at the chosen lower bound,

    No, it wont. That isn't in the *battery* - that's in whatever is drawing current off it. I have many lthium packs with no such ptotection



    because if it completely discharges,
    it's dead forever. Not a 'steep' decline, a 'fall off the wall' decline.

    No, again it goes on down to zero, 3V is just approximately the point at
    which irreversible damage starts to happen.


    It's a good idea to have some independent means of anticipating this
    point.

    The way to run a lithium in this application is to use a 2 cell lithium,
    a constant voltage constant current charger limited to 8.4V and probably
    no more than the battery mAh capacity divided by one hour... and a
    switched mode 5V regulator to feed the Pi and then somehow monitor raw
    battery voltage and switch it all off when it gets to say 6.6V and hope
    that the SMPS and monitoring circuit don't then still draw enough to
    damage the battery, or even if it is take it on the chin and replace the battery when that happens.

    I would imagine that the whole setup would cost more than the Pi itself



    --
    “It is dangerous to be right in matters on which the established
    authorities are wrong.”

    ― Voltaire, The Age of Louis XIV

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Folderol@3:770/3 to The Natural Philosopher on Thu Jan 14 09:23:24 2021
    On Thu, 14 Jan 2021 03:46:08 +0000
    The Natural Philosopher <tnp@invalid.invalid> wrote:

    On 13/01/2021 22:04, Joe wrote:


    Lithium cells have a linear-ish discharge curve, from 4.2V down to
    around 3V.

    They do.

    Any sensibly designed lithium battery will cut off its
    output at the chosen lower bound,

    No, it wont. That isn't in the *battery* - that's in whatever is drawing >current off it. I have many lthium packs with no such ptotection



    because if it completely discharges,
    it's dead forever. Not a 'steep' decline, a 'fall off the wall' decline.

    No, again it goes on down to zero, 3V is just approximately the point at >which irreversible damage starts to happen.


    It's a good idea to have some independent means of anticipating this
    point.

    The way to run a lithium in this application is to use a 2 cell lithium,
    a constant voltage constant current charger limited to 8.4V and probably
    no more than the battery mAh capacity divided by one hour... and a
    switched mode 5V regulator to feed the Pi and then somehow monitor raw >battery voltage and switch it all off when it gets to say 6.6V and hope
    that the SMPS and monitoring circuit don't then still draw enough to
    damage the battery, or even if it is take it on the chin and replace the >battery when that happens.

    To get round that use a latching relay 'start' circuit, and when the
    battery voltage gets too low unlatch the relay- Zero power demand then.


    --
    W J G

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Ahem A Rivet's Shot@3:770/3 to Theo on Thu Jan 14 10:18:57 2021
    On 13 Jan 2021 23:37:18 +0000 (GMT)
    Theo <theom+news@chiark.greenend.org.uk> wrote:

    Joe <joe@jretrading.com> wrote:
    Lithium cells have a linear-ish discharge curve, from 4.2V down to
    around 3V. Any sensibly designed lithium battery will cut off its
    output at the chosen lower bound, because if it completely discharges,
    it's dead forever. Not a 'steep' decline, a 'fall off the wall' decline.

    It's a good idea to have some independent means of anticipating this
    point.

    But lithium ion cells don't work for this application. We need 5V.
    One cell is too little, two cells is too much.

    One cell and a DC-DC converter to give you a regulated supply,
    monitor the cell so you can shut down (well) before there's too little to
    run the converter.

    --
    Steve O'Hara-Smith | Directable Mirror Arrays C:\>WIN | A better way to focus the sun
    The computer obeys and wins. | licences available see
    You lose and Bill collects. | http://www.sohara.org/

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From The Natural Philosopher@3:770/3 to Folderol on Thu Jan 14 11:04:03 2021
    On 14/01/2021 09:23, Folderol wrote:
    On Thu, 14 Jan 2021 03:46:08 +0000
    The Natural Philosopher <tnp@invalid.invalid> wrote:

    On 13/01/2021 22:04, Joe wrote:


    Lithium cells have a linear-ish discharge curve, from 4.2V down to
    around 3V.

    They do.

    Any sensibly designed lithium battery will cut off its
    output at the chosen lower bound,

    No, it wont. That isn't in the *battery* - that's in whatever is drawing
    current off it. I have many lthium packs with no such ptotection



    because if it completely discharges,
    it's dead forever. Not a 'steep' decline, a 'fall off the wall' decline.

    No, again it goes on down to zero, 3V is just approximately the point at
    which irreversible damage starts to happen.


    It's a good idea to have some independent means of anticipating this
    point.

    The way to run a lithium in this application is to use a 2 cell lithium,
    a constant voltage constant current charger limited to 8.4V and probably
    no more than the battery mAh capacity divided by one hour... and a
    switched mode 5V regulator to feed the Pi and then somehow monitor raw
    battery voltage and switch it all off when it gets to say 6.6V and hope
    that the SMPS and monitoring circuit don't then still draw enough to
    damage the battery, or even if it is take it on the chin and replace the
    battery when that happens.

    To get round that use a latching relay 'start' circuit, and when the
    battery voltage gets too low unlatch the relay- Zero power demand then.


    Its getting as complicated as a renewable energy grid though isn't it?



    --
    In todays liberal progressive conflict-free education system, everyone
    gets full Marx.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From The Natural Philosopher@3:770/3 to Ahem A Rivet's Shot on Thu Jan 14 11:04:55 2021
    On 14/01/2021 10:18, Ahem A Rivet's Shot wrote:
    On 13 Jan 2021 23:37:18 +0000 (GMT)
    Theo <theom+news@chiark.greenend.org.uk> wrote:

    Joe <joe@jretrading.com> wrote:
    Lithium cells have a linear-ish discharge curve, from 4.2V down to
    around 3V. Any sensibly designed lithium battery will cut off its
    output at the chosen lower bound, because if it completely discharges,
    it's dead forever. Not a 'steep' decline, a 'fall off the wall' decline. >>>
    It's a good idea to have some independent means of anticipating this
    point.

    But lithium ion cells don't work for this application. We need 5V.
    One cell is too little, two cells is too much.

    One cell and a DC-DC converter to give you a regulated supply,
    monitor the cell so you can shut down (well) before there's too little to
    run the converter.

    It's slightly simpler and more efficient to use a step down from two cells.

    --
    The theory of Communism may be summed up in one sentence: Abolish all
    private property.

    Karl Marx

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Ahem A Rivet's Shot@3:770/3 to The Natural Philosopher on Thu Jan 14 11:46:23 2021
    On Thu, 14 Jan 2021 11:04:55 +0000
    The Natural Philosopher <tnp@invalid.invalid> wrote:

    On 14/01/2021 10:18, Ahem A Rivet's Shot wrote:
    On 13 Jan 2021 23:37:18 +0000 (GMT)
    Theo <theom+news@chiark.greenend.org.uk> wrote:

    Joe <joe@jretrading.com> wrote:
    Lithium cells have a linear-ish discharge curve, from 4.2V down to
    around 3V. Any sensibly designed lithium battery will cut off its
    output at the chosen lower bound, because if it completely discharges, >>> it's dead forever. Not a 'steep' decline, a 'fall off the wall'
    decline.

    It's a good idea to have some independent means of anticipating this
    point.

    But lithium ion cells don't work for this application. We need 5V.
    One cell is too little, two cells is too much.

    One cell and a DC-DC converter to give you a regulated supply,
    monitor the cell so you can shut down (well) before there's too little
    to run the converter.

    It's slightly simpler and more efficient to use a step down from two
    cells.

    It's only simpler if you look at what's on the converter board :)

    --
    Steve O'Hara-Smith | Directable Mirror Arrays C:\>WIN | A better way to focus the sun
    The computer obeys and wins. | licences available see
    You lose and Bill collects. | http://www.sohara.org/

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From The Natural Philosopher@3:770/3 to Ahem A Rivet's Shot on Thu Jan 14 12:21:52 2021
    On 14/01/2021 11:46, Ahem A Rivet's Shot wrote:
    On Thu, 14 Jan 2021 11:04:55 +0000
    The Natural Philosopher <tnp@invalid.invalid> wrote:

    On 14/01/2021 10:18, Ahem A Rivet's Shot wrote:
    On 13 Jan 2021 23:37:18 +0000 (GMT)
    Theo <theom+news@chiark.greenend.org.uk> wrote:

    Joe <joe@jretrading.com> wrote:
    Lithium cells have a linear-ish discharge curve, from 4.2V down to
    around 3V. Any sensibly designed lithium battery will cut off its
    output at the chosen lower bound, because if it completely discharges, >>>>> it's dead forever. Not a 'steep' decline, a 'fall off the wall'
    decline.

    It's a good idea to have some independent means of anticipating this >>>>> point.

    But lithium ion cells don't work for this application. We need 5V.
    One cell is too little, two cells is too much.

    One cell and a DC-DC converter to give you a regulated supply,
    monitor the cell so you can shut down (well) before there's too little
    to run the converter.

    It's slightly simpler and more efficient to use a step down from two
    cells.

    It's only simpler if you look at what's on the converter board :)

    No, that's not true. To step up you need to use a buck converter with a
    choke big enough to store energy to push the voltage up. To step down
    you only need a choke to smooth HF ripple, and transistor efficiency
    goes up with voltage and down with current, so that's a win win too.

    --
    "Corbyn talks about equality, justice, opportunity, health care, peace, community, compassion, investment, security, housing...."
    "What kind of person is not interested in those things?"

    "Jeremy Corbyn?"

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Martin Gregorie@3:770/3 to Ahem A Rivet's Shot on Thu Jan 14 12:56:23 2021
    On Thu, 14 Jan 2021 10:18:57 +0000, Ahem A Rivet's Shot wrote:

    On 13 Jan 2021 23:37:18 +0000 (GMT)
    Theo <theom+news@chiark.greenend.org.uk> wrote:

    Joe <joe@jretrading.com> wrote:
    Lithium cells have a linear-ish discharge curve, from 4.2V down to
    around 3V. Any sensibly designed lithium battery will cut off its
    output at the chosen lower bound, because if it completely
    discharges, it's dead forever. Not a 'steep' decline, a 'fall off the
    wall' decline.

    It's a good idea to have some independent means of anticipating this
    point.

    But lithium ion cells don't work for this application. We need 5V.
    One cell is too little, two cells is too much.

    One cell and a DC-DC converter to give you a regulated supply,
    monitor the cell so you can shut down (well) before there's too little
    to run the converter.

    The Powerboost 1000C from Adafruit does that - has 5v input an is
    connected to an RPi and a single cell LiPO cell, so is a a postage-stamp
    sized 1 amp UPS.

    It can run the attached RPi while charging the LiPO from the 5v supply
    and continue to run the RPi off the LoPO in the 5v feed is turned off.

    Needs some soldering: it comes with all the connectors that can be
    attached to it, but not soldered in. The idea is that you only install
    the ones you need.

    Note I have one, but haven't yet found the round tuit I need to get it
    and a Pi Zero up and running using it. When the project is complete I'll
    what will be effectively a pocketable system TFT touch screen and built-
    in charger.


    --
    --
    Martin | martin at
    Gregorie | gregorie dot org

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Ahem A Rivet's Shot@3:770/3 to The Natural Philosopher on Thu Jan 14 12:38:31 2021
    On Thu, 14 Jan 2021 12:21:52 +0000
    The Natural Philosopher <tnp@invalid.invalid> wrote:

    On 14/01/2021 11:46, Ahem A Rivet's Shot wrote:
    On Thu, 14 Jan 2021 11:04:55 +0000
    It's only simpler if you look at what's on the converter
    board :)

    No, that's not true. To step up you need to use a buck converter with a
    choke big enough to store energy to push the voltage up. To step down

    Sure that's all on the board :) From a system perspective it's just
    a board with battery in and 5V out :)

    you only need a choke to smooth HF ripple, and transistor efficiency
    goes up with voltage and down with current, so that's a win win too.

    Oh I agree a step down only is a simpler board, but either way it's
    an off the shelf small component.

    --
    Steve O'Hara-Smith | Directable Mirror Arrays C:\>WIN | A better way to focus the sun
    The computer obeys and wins. | licences available see
    You lose and Bill collects. | http://www.sohara.org/

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Theo@3:770/3 to Ahem A Rivet's Shot on Thu Jan 14 13:46:31 2021
    Ahem A Rivet's Shot <steveo@eircom.net> wrote:
    One cell and a DC-DC converter to give you a regulated supply, monitor the cell so you can shut down (well) before there's too little to
    run the converter.

    Which is all very nice, but is not what the OP is doing.

    They're monitoring *the input rail of the Pi*, because that's all the Pi hardware does. You put 5V in, it tells you if it sags below some threshold. You don't get to put some other voltage in to test. If you put in your raw cell voltage that will under- or over-volt the Pi and it'll either not work
    or fry[1].

    If they want to monitor some other voltage, they'll have to start from
    scratch because none of what they're doing (hardware or software) will work.

    Theo

    [1] the main SoC has some protection as it has its own voltage regulators,
    but USB-side things do use that 5V directly and you may fry it that way

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From David Higton@3:770/3 to The Natural Philosopher on Thu Jan 14 14:17:11 2021
    In message <rtpd10$7hp$1@dont-email.me>
    The Natural Philosopher <tnp@invalid.invalid> wrote:

    No, that's not true. To step up you need to use a buck converter with a
    choke big enough to store energy to push the voltage up. To step down
    you only need a choke to smooth HF ripple

    Not true. The step-down inductor needs to store energy too.

    David

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From The Natural Philosopher@3:770/3 to Theo on Thu Jan 14 14:26:10 2021
    On 14/01/2021 13:46, Theo wrote:
    Ahem A Rivet's Shot <steveo@eircom.net> wrote:
    One cell and a DC-DC converter to give you a regulated supply,
    monitor the cell so you can shut down (well) before there's too little to
    run the converter.

    Which is all very nice, but is not what the OP is doing.

    They're monitoring *the input rail of the Pi*, because that's all the Pi hardware does. You put 5V in, it tells you if it sags below some threshold. You don't get to put some other voltage in to test. If you put in your raw cell voltage that will under- or over-volt the Pi and it'll either not work or fry[1].

    If they want to monitor some other voltage, they'll have to start from scratch because none of what they're doing (hardware or software) will work.

    and the point is that any regulated supply will simply die without
    warning, so they will have to


    Theo

    [1] the main SoC has some protection as it has its own voltage regulators, but USB-side things do use that 5V directly and you may fry it that way



    --
    Those who want slavery should have the grace to name it by its proper
    name. They must face the full meaning of that which they are advocating
    or condoning; the full, exact, specific meaning of collectivism, of its
    logical implications, of the principles upon which it is based, and of
    the ultimate consequences to which these principles will lead. They must
    face it, then decide whether this is what they want or not.

    Ayn Rand.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From The Natural Philosopher@3:770/3 to David Higton on Thu Jan 14 14:28:13 2021
    On 14/01/2021 14:17, David Higton wrote:
    In message <rtpd10$7hp$1@dont-email.me>
    The Natural Philosopher <tnp@invalid.invalid> wrote:

    No, that's not true. To step up you need to use a buck converter with a
    choke big enough to store energy to push the voltage up. To step down
    you only need a choke to smooth HF ripple

    Not true. The step-down inductor needs to store energy too.

    It isn't a step down inductor.

    you just use variable PWM on the raw voltage. All you need to do is use
    a cap and some form of current limiter - a resistors works but
    dissipates power




    David



    --
    Those who want slavery should have the grace to name it by its proper
    name. They must face the full meaning of that which they are advocating
    or condoning; the full, exact, specific meaning of collectivism, of its
    logical implications, of the principles upon which it is based, and of
    the ultimate consequences to which these principles will lead. They must
    face it, then decide whether this is what they want or not.

    Ayn Rand.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Folderol@3:770/3 to The Natural Philosopher on Thu Jan 14 17:48:52 2021
    On Thu, 14 Jan 2021 11:04:03 +0000
    The Natural Philosopher <tnp@invalid.invalid> wrote:

    On 14/01/2021 09:23, Folderol wrote:

    To get round that use a latching relay 'start' circuit, and when the
    battery voltage gets too low unlatch the relay- Zero power demand then.


    Its getting as complicated as a renewable energy grid though isn't it?

    Oh come on now!
    I designed my first latching relay/(manual)shutdown thing when I was about 10. When I was in my early teens I did one that would automatically time out, using one of those new-fanged transistor things and a cap.

    --
    W J G

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Pancho@3:770/3 to Martin Gregorie on Sat Jan 16 12:24:48 2021
    On 13/01/2021 15:12, Martin Gregorie wrote:
    On Wed, 13 Jan 2021 13:56:52 +0000, The Natural Philosopher wrote:

    On 12/01/2021 16:40, Pancho wrote:
    On 12/01/2021 12:14, Simple Simon wrote:
    I am working on a battery powered car and want the Pi to shut down
    automatically if the battery starts to go flat to try to prevent SD
    card corruption. I am a beginner to bash scripts! I will run this via
    crontab...

    #!/bin/bash powerstatus=$(vcgencmd get_throttled)
    if [ $powerstatus="throttled=0x1" ]
    then echo Under Voltage Detected - Shutting Down sudo halt else echo
    Voltage Normal fi

    Obviously it is not working!! Could someone correct and explain for me >>>> please.

    Not being much help, but...

    I'm aware of Under Voltage problems from entries in journalctl. It
    seems to me that listening for events logged to journalctl would be a
    pretty common thing to do. So common that there should be some standard
    way of doing it. Some kind of standard Observer pattern on jounalctl.
    Does anyone know of one?

    One way to get a rapid notification of entries written to a log is the
    use 'tail' in a script.

    Running "sudo tail -f /var/log/messages" from a console shows you each message as it is written to /var/log/messages, so a script started a boot time to run with sufficient privilege to read those log entries could
    easily use tail to read messages as they are written and filter out the
    ones of interest with grep or awk and use them to , for instance force a clean shutdown on low battery, something like

    #!/bin/bash
    tail -f /var/log/messages | awk <<ENDPROG
    /Under voltage/ { shutdown -h NOW }
    ENDPROG


    I thought << was a way of redirecting stdin, not redirecting command
    line or program file.

    But perhaps more importantly, Linux seems to buffer pipelines, meaning
    an event watcher on the journal isn't close to immediate.

    In fact I couldn't get the following to work at all:

    sudo journalctl -f | awk '{ print $0 }'

    Let alone anything more ambitious.

    Also, although I've read that Unix pipe lines should work as a "push
    queue" event mechanism they always feel like a busy loop pull to me.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Martin Gregorie@3:770/3 to Pancho on Sat Jan 16 17:53:29 2021
    On Sat, 16 Jan 2021 12:24:48 +0000, Pancho wrote:

    I thought << was a way of redirecting stdin, not redirecting command
    line or program file.

    'man bash' is your friend - look up 'here documents' in the bash manpage

    In fact I couldn't get the following to work at all:

    sudo journalctl -f | awk '{ print $0 }'


    awk is expecting to be passed the awkscript in one of these ways:
    1) awk -f sourcefile
    2) awk -- 'program text'

    and in (2) the '--' which marks the end of awk options, is mandatory
    if you use this syntax.

    sudo journalctl -f | awk -- '{ print $0 }'

    Since journalctl only reads the systemd journal, if you want to read any
    other journal, and /var/log/messages is the default for most non-system programs you need to use tail, not journalctl, so write something like

    sudo tail -f /var/log/messages | awk -- '{ print $0 }'

    which works here.


    --
    --
    Martin | martin at
    Gregorie | gregorie dot org

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From David Higton@3:770/3 to The Natural Philosopher on Sat Jan 16 20:09:45 2021
    In message <rtpkdt$rjp$2@dont-email.me>
    The Natural Philosopher <tnp@invalid.invalid> wrote:

    On 14/01/2021 14:17, David Higton wrote:
    In message <rtpd10$7hp$1@dont-email.me>
    The Natural Philosopher <tnp@invalid.invalid> wrote:

    No, that's not true. To step up you need to use a buck converter with a choke big enough to store energy to push the voltage up. To step down
    you only need a choke to smooth HF ripple

    Not true. The step-down inductor needs to store energy too.

    It isn't a step down inductor.

    Just a short form of words - I was referring to an inductor in a step
    down regulator.

    you just use variable PWM on the raw voltage. All you need to do is use a cap and some form of current limiter - a resistors works but dissipates power

    There are regulators that use capacitors, sure (commonly step up, e.g.
    for EIA-232 line drivers), but they inherently require significant
    resistance in the switch elements to limit the current, so they're
    lossy. For a low power application the losses don't matter, but in
    the case discussed here, I don't think you'd want that level of loss.
    It works out the same as a linear regulator, according to my maths.

    Using an inductor in a conventional step-down switching regulator is
    much more efficient, and of course requires the inductor to store
    energy.

    (Apologies for the delay in replying - the news server I use changed
    its implementation of NNTP and I have had to modify my news transport
    client's code to suit.)

    David

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Pancho@3:770/3 to Martin Gregorie on Sat Jan 16 20:39:09 2021
    On 16/01/2021 17:53, Martin Gregorie wrote:


    sudo tail -f /var/log/messages | awk -- '{ print $0 }'

    which works here.



    when you say works, what do you mean?

    A quick check on my machine show that this too is buffered.

    Try writing messages using the logger command in a bash shell.

    It looked to me as if I had to write nearly 3000 bytes before it flushed
    ,and hence awk printed out.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Martin Gregorie@3:770/3 to Pancho on Sat Jan 16 22:33:46 2021
    On Sat, 16 Jan 2021 20:39:09 +0000, Pancho wrote:

    On 16/01/2021 17:53, Martin Gregorie wrote:


    sudo tail -f /var/log/messages | awk -- '{ print $0 }'

    which works here.

    Messages added to /var/log/messages with the following command are
    displayed almost instantly, though they aren't exactly readable since the
    text is shown as a hex string:

    sudo echo "Another message" 2>&1 | logsave /var/log/messages -

    Changing the listener to

    $ sudo tail -f /var/log/maillog | awk -- '{ print $0 }'

    so its looking for messages sent to /var/log/maillog and then running

    $ sendmail -?

    which runs sendmail with invalid parameters shows this fatal message
    being logged almost instantly:

    fatal: usage: sendmail [options]


    --
    --
    Martin | martin at
    Gregorie | gregorie dot org

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Martin Gregorie@3:770/3 to Pancho on Sat Jan 16 23:52:03 2021
    On Sat, 16 Jan 2021 23:30:34 +0000, Pancho wrote:

    On 16/01/2021 22:33, Martin Gregorie wrote:
    On Sat, 16 Jan 2021 20:39:09 +0000, Pancho wrote:

    On 16/01/2021 17:53, Martin Gregorie wrote:


    sudo tail -f /var/log/messages | awk -- '{ print $0 }'

    which works here.

    Messages added to /var/log/messages with the following command are
    displayed almost instantly, though they aren't exactly readable since
    the text is shown as a hex string:

    sudo echo "Another message" 2>&1 | logsave /var/log/messages -


    Why not just use the logger command?

    e.g.

    logger "Another message"

    You seem to be over complicating things: why redirect stderr to stdin
    &1 from echo, echo doesn't need sudo, logsave /var/log/messages does
    need sudo, logsave is truncating /var/log/messages.

    The immediate response you are seeing is stderr from tail, which isn't
    being piped to awk at all. This error message is generated by tail -f
    because logsave truncates the file that is being tailed, logsave -a
    appends.

    The manual says pipes are buffered. My experiments say pipes are
    buffered.

    So how come you're getting delays and I'm not?




    --
    --
    Martin | martin at
    Gregorie | gregorie dot org

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Pancho@3:770/3 to Martin Gregorie on Sat Jan 16 23:30:34 2021
    On 16/01/2021 22:33, Martin Gregorie wrote:
    On Sat, 16 Jan 2021 20:39:09 +0000, Pancho wrote:

    On 16/01/2021 17:53, Martin Gregorie wrote:


    sudo tail -f /var/log/messages | awk -- '{ print $0 }'

    which works here.

    Messages added to /var/log/messages with the following command are
    displayed almost instantly, though they aren't exactly readable since the text is shown as a hex string:

    sudo echo "Another message" 2>&1 | logsave /var/log/messages -


    Why not just use the logger command?

    e.g.

    logger "Another message"

    You seem to be over complicating things: why redirect stderr to stdin
    &1 from echo, echo doesn't need sudo, logsave /var/log/messages does
    need sudo, logsave is truncating /var/log/messages.

    The immediate response you are seeing is stderr from tail, which isn't
    being piped to awk at all. This error message is generated by tail -f
    because logsave truncates the file that is being tailed, logsave -a appends.

    The manual says pipes are buffered. My experiments say pipes are buffered.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From A. Dumas@3:770/3 to Martin Gregorie on Sun Jan 17 01:09:37 2021
    Martin Gregorie <martin@mydomain.invalid> wrote:
    sudo echo "Another message" 2>&1 | logsave /var/log/messages -

    sudo echo ... | ... doesn't work the way you probably intend. The sudo only applies to echo, not the second command. You could use instead: sudo
    logsave -a logfile command_with_parameters &>/dev/null

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From The Natural Philosopher@3:770/3 to David Higton on Sun Jan 17 07:19:57 2021
    On 16/01/2021 20:09, David Higton wrote:
    Using an inductor in a conventional step-down switching regulator is
    much more efficient, and of course requires the inductor to store
    energy.


    No, it doesn't have to store ALL the energy - like it does in a step up

    And if you don't mind massive ripple, it isn't needed *at all*.
    Consider a 6V battery deeding a 3V lamp. Simply chop the 6V on a 50%
    duty cycle and there you are. No choke needed at all.

    Add an LC filter and the cap stores all that is necessary. All the L
    does is limit the peak current into the C.



    --
    There is something fascinating about science. One gets such wholesale
    returns of conjecture out of such a trifling investment of fact.

    Mark Twain

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Pancho@3:770/3 to Martin Gregorie on Sun Jan 17 10:39:32 2021
    On 16/01/2021 23:52, Martin Gregorie wrote:
    On Sat, 16 Jan 2021 23:30:34 +0000, Pancho wrote:

    On 16/01/2021 22:33, Martin Gregorie wrote:
    On Sat, 16 Jan 2021 20:39:09 +0000, Pancho wrote:

    On 16/01/2021 17:53, Martin Gregorie wrote:


    sudo tail -f /var/log/messages | awk -- '{ print $0 }'

    which works here.

    Messages added to /var/log/messages with the following command are
    displayed almost instantly, though they aren't exactly readable since
    the text is shown as a hex string:

    sudo echo "Another message" 2>&1 | logsave /var/log/messages -


    Why not just use the logger command?

    e.g.

    logger "Another message"

    You seem to be over complicating things: why redirect stderr to stdin
    &1 from echo, echo doesn't need sudo, logsave /var/log/messages does
    need sudo, logsave is truncating /var/log/messages.

    The immediate response you are seeing is stderr from tail, which isn't
    being piped to awk at all. This error message is generated by tail -f
    because logsave truncates the file that is being tailed, logsave -a
    appends.

    The manual says pipes are buffered. My experiments say pipes are
    buffered.

    So how come you're getting delays and I'm not?


    Dunno, I thought I had explained the

    sudo echo "Another message" 2>&1 | logsave /var/log/messages -

    Causing an error in tail -f /var/log/messages.


    On further reflection (and Googling) I have discovered it was awk that
    was buffering, not the pipe. Apparently when awk is in a pipeline it
    stops being in "interactive" mode. This can be fixed using awk -W
    interactive

    For a simple example:

    In one bash shell terminal:

    $ tail -f /var/log/messages | awk -- '{ print $0 }'

    In another:

    $ logger "Another message."

    You will see nothing until you repeat the same command many times. Then
    you will see it all burst out in one go. (Less repeats if you use use a
    longer message.)


    However, this command produces an immediate response.

    $ tail -f /var/log/messages | awk -W interactive -- '{ print $0 }'

    This applies to journalctl -f, too.

    So something new learnt :-) (or possibly even relearnt, for me anyway)

    However, I'm not yet convinced this is a sensible way of observing
    events. I'm still undecided.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Chris Elvidge@3:770/3 to Martin Gregorie on Sun Jan 17 11:52:53 2021
    On 16/01/2021 05:53 pm, Martin Gregorie wrote:
    In fact I couldn't get the following to work at all:

    sudo journalctl -f | awk '{ print $0 }'

    Sorry Martin. This shouldn't have been attributed to you; it was Pancho
    that wrote it. Selecting some message text in Thunderbird seems to
    confuse it; however I should remember to check attributions before sending.

    --
    Chris Elvidge
    England

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Chris Elvidge@3:770/3 to Martin Gregorie on Sun Jan 17 11:48:55 2021
    On 16/01/2021 05:53 pm, Martin Gregorie wrote:
    In fact I couldn't get the following to work at all:

    sudo journalctl -f | awk '{ print $0 }'

    Why would you use awk at all here? You're just (re)printing the input line.
    BTW the above works here - LMDE4, GNU awk 4.2

    --
    Chris Elvidge
    England

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Pancho@3:770/3 to Chris Elvidge on Sun Jan 17 12:11:07 2021
    On 17/01/2021 11:48, Chris Elvidge wrote:
    On 16/01/2021 05:53 pm, Martin Gregorie wrote:
    In fact I couldn't get the following to work at all:

    sudo journalctl -f | awk '{ print $0 }'

    Why would you use awk at all here? You're just (re)printing the input line. BTW the above works here - LMDE4, GNU awk 4.2


    If you had read the thread, you would have seen that I was testing
    potential buffering, i.e. awk's timely response to input from the
    journalctl -f.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Andy Burns@3:770/3 to Chris Elvidge on Sun Jan 17 13:53:06 2021
    Chris Elvidge wrote:

    Selecting some message text in Thunderbird seems to confuse it

    It doesn't confuse it as such, but it only quotes the selected text,
    however it still adds an attribution for the author of the message
    you're replying to.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Martin Gregorie@3:770/3 to Pancho on Sun Jan 17 14:32:20 2021
    On Sun, 17 Jan 2021 10:39:32 +0000, Pancho wrote:

    However, I'm not yet convinced this is a sensible way of observing
    events. I'm still undecided.

    That depends: if the occurrence of events to be observed cause messages
    to be logged rather than being dealt with by the program that observes
    them, then watching a log for the occurrence might well be the best way
    to do it since this separates the observation from the consequent action, offering the possibility of more than one observing process being able to trigger the action. Where that action is irreversable, as it is in this
    case, that sounds like a good idea.

    Of course, if the observer doesn't log anything, there's no reason to use
    a general purpose logfile - just write the observations to a file or
    pipeline that's accessable to the process that will execute the action:
    as this is Linux, you can use shared memory or a semaphore (if the
    programming language supports it) to signal that an actionable event has occurred. Shared memory, event queues and asynchronous i/o are all
    available in the C and Java standard function/class libraries but may not
    be in other languages.

    OTOH if the required action is specific to the observer, i.e. doesn't
    affect anything outside the process(es) handling the data stream being observed, then simply deal with the observation inside the observing
    process.


    --
    --
    Martin | martin at
    Gregorie | gregorie dot org

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Martin Gregorie@3:770/3 to Chris Elvidge on Sun Jan 17 14:43:35 2021
    On Sun, 17 Jan 2021 11:48:55 +0000, Chris Elvidge wrote:

    On 16/01/2021 05:53 pm, Martin Gregorie wrote:
    In fact I couldn't get the following to work at all:

    sudo journalctl -f | awk '{ print $0 }'

    Why would you use awk at all here? You're just (re)printing the input
    line.

    I think that was just simple proof-of-concept test code to check that awk
    was reading the journalctl output stream. Quite reasonable if you've
    never user awk before.

    BTW the above works here - LMDE4, GNU awk 4.2

    I don't recall needing the '--' in the "awk -- 'short awk prog'" command before, but there seems to have been a bit of rationalisation going on. I notice that now (Fedora 32) /usr/bin/awk is now a symlink pointing to / usr/bin/gawk but don't know when that happened or which other distros do
    the same.


    --
    --
    Martin | martin at
    Gregorie | gregorie dot org

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From David Higton@3:770/3 to The Natural Philosopher on Sun Jan 17 15:57:26 2021
    In message <ru0oeu$f6t$1@dont-email.me>
    The Natural Philosopher <tnp@invalid.invalid> wrote:

    On 16/01/2021 20:09, David Higton wrote:
    Using an inductor in a conventional step-down switching regulator is much more efficient, and of course requires the inductor to store energy.


    No, it doesn't have to store ALL the energy - like it does in a step up

    And if you don't mind massive ripple, it isn't needed *at all*. Consider a
    6V battery deeding a 3V lamp. Simply chop the 6V on a 50% duty cycle and there you are. No choke needed at all.

    Yes, but that situation is entirely unlike what the OP was talking about,
    and is indeed inapplicable.

    Add an LC filter and the cap stores all that is necessary. All the L does
    is limit the peak current into the C.

    If it's going to limit the current, it must not saturate, in which case
    it'd store the energy that you claim is unnecessary. You'd also have
    to remove the energy from the inductor every time the switch switched
    off, otherwise the switch would break down. Basically you'd have created
    a conventional step-down converter.

    Or, if the inductor saturated but still limited the current, it would
    have to be a high enough resistance to be quite lossy.

    David

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From John Aldridge@3:770/3 to All on Sun Jan 17 17:52:40 2021
    In article <ru0oeu$f6t$1@dont-email.me>, tnp@invalid.invalid says...

    On 16/01/2021 20:09, David Higton wrote:
    Using an inductor in a conventional step-down switching regulator is
    much more efficient, and of course requires the inductor to store
    energy.


    No, it doesn't have to store ALL the energy - like it does in a step up

    And if you don't mind massive ripple, it isn't needed *at all*.
    Consider a 6V battery deeding a 3V lamp. Simply chop the 6V on a 50%
    duty cycle and there you are. No choke needed at all.

    25%?

    John

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From The Natural Philosopher@3:770/3 to David Higton on Mon Jan 18 07:05:31 2021
    On 17/01/2021 15:57, David Higton wrote:
    If it's going to limit the current, it must not saturate, in which case
    it'd store the energy that you claim is unnecessary.

    Go back and re read your electronics 101


    --
    "The great thing about Glasgow is that if there's a nuclear attack it'll
    look exactly the same afterwards."

    Billy Connolly

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Martin Gregorie@3:770/3 to Pancho on Mon Jan 18 13:41:21 2021
    On Mon, 18 Jan 2021 13:20:59 +0000, Pancho wrote:

    I'm not sure what awk interactive/buffering means. Without the "-W interactive" flag awk doesn't just buffer output, the system() call is delayed too. It's more like it is buffering input.

    Since you're using a pipe to pass messages from the log reader to the awk program, awk makes a read request which will wait for input until either
    a line of text is written to the pipe, in which case it gets read by awk
    (awk reads lines, so the message MUST be terminated by a newline) OR the
    pipe is closed, in which case awk gets an EOF and stops. If the awk
    script contains a END action, this is executed before is quits.

    The only apparent buffering you should see is due to awk waiting for the newline that terminates the line being read.

    About -W : according to the manpage for the awk version I'm using, 5.0.1,
    -W has nothing to do with waiting for anything. All it does is to change
    the option marker from - to --


    --
    --
    Martin | martin at
    Gregorie | gregorie dot org

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Pancho@3:770/3 to Martin Gregorie on Mon Jan 18 14:02:33 2021
    On 18/01/2021 13:41, Martin Gregorie wrote:
    On Mon, 18 Jan 2021 13:20:59 +0000, Pancho wrote:

    I'm not sure what awk interactive/buffering means. Without the "-W
    interactive" flag awk doesn't just buffer output, the system() call is
    delayed too. It's more like it is buffering input.

    Since you're using a pipe to pass messages from the log reader to the awk program, awk makes a read request which will wait for input until either
    a line of text is written to the pipe, in which case it gets read by awk
    (awk reads lines, so the message MUST be terminated by a newline) OR the
    pipe is closed, in which case awk gets an EOF and stops. If the awk
    script contains a END action, this is executed before is quits.

    The only apparent buffering you should see is due to awk waiting for the newline that terminates the line being read.

    About -W : according to the manpage for the awk version I'm using, 5.0.1,
    -W has nothing to do with waiting for anything. All it does is to change
    the option marker from - to --

    I'm using Raspbian Buster, default awk is mawk 1.3.3.

    Perhaps that explains our different experience. mawk buffering doesn't
    appear to be line buffering by default, i.e it buffers blocks somewhere
    between 2 to 3kB (whatever the correct name for that is).

    What OS are you using?

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Pancho@3:770/3 to Martin Gregorie on Mon Jan 18 13:20:59 2021
    On 17/01/2021 14:32, Martin Gregorie wrote:
    On Sun, 17 Jan 2021 10:39:32 +0000, Pancho wrote:

    However, I'm not yet convinced this is a sensible way of observing
    events. I'm still undecided.

    That depends: if the occurrence of events to be observed cause messages
    to be logged rather than being dealt with by the program that observes
    them, then watching a log for the occurrence might well be the best way
    to do it since this separates the observation from the consequent action, offering the possibility of more than one observing process being able to trigger the action. Where that action is irreversable, as it is in this
    case, that sounds like a good idea.

    Of course, if the observer doesn't log anything, there's no reason to use
    a general purpose logfile - just write the observations to a file or
    pipeline that's accessable to the process that will execute the action:
    as this is Linux, you can use shared memory or a semaphore (if the programming language supports it) to signal that an actionable event has occurred. Shared memory, event queues and asynchronous i/o are all
    available in the C and Java standard function/class libraries but may not
    be in other languages.

    OTOH if the required action is specific to the observer, i.e. doesn't
    affect anything outside the process(es) handling the data stream being observed, then simply deal with the observation inside the observing
    process.



    Yeah, I'm not sure what you mean, the journal/logfile is the "subject",
    the "observer" logging isn't relevant.

    FWIW I got this basic pattern to work.

    --
    journalctl -f | awk -W interactive -- '/Specific Event/ { system( "/home/pi/SpecificEventHandler.sh") }'
    --

    Response appears to be ballpark of 30ms on a rPi3. Tested with:

    date +%S-%N;logger "Specific Event"


    And /home/pi/SpecificEventHandler.sh
    --
    #!/bin/bash

    touch "awktest$(date +%H-%M-%S-%N)"

    -

    I'm not sure what awk interactive/buffering means. Without the "-W
    interactive" flag awk doesn't just buffer output, the system() call is
    delayed too. It's more like it is buffering input.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From David Higton@3:770/3 to The Natural Philosopher on Mon Jan 18 14:14:59 2021
    In message <ru3bvr$8pi$7@dont-email.me>
    The Natural Philosopher <tnp@invalid.invalid> wrote:

    On 17/01/2021 15:57, David Higton wrote:
    If it's going to limit the current, it must not saturate, in which case it'd store the energy that you claim is unnecessary.

    Go back and re read your electronics 101

    Most amusing, given my decades-long career in electronic engineering.

    In what way do think that my statement is incorrect?

    David

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Martin Gregorie@3:770/3 to Pancho on Mon Jan 18 15:15:25 2021
    On Mon, 18 Jan 2021 14:02:33 +0000, Pancho wrote:

    About -W : according to the manpage for the awk version I'm using,
    5.0.1, -W has nothing to do with waiting for anything. All it does is
    to change the option marker from - to --

    I'm using Raspbian Buster, default awk is mawk 1.3.3.

    Perhaps that explains our different experience. mawk buffering doesn't
    appear to be line buffering by default, i.e it buffers blocks somewhere between 2 to 3kB (whatever the correct name for that is).

    What OS are you using?

    Fedora 32 for these tests, which uses awk 5.0.1 - The Buster awk is very
    old, so raising a bug requesting an upgrade to the latest awk may be a
    good idea.

    Running "man 7 pipe" tells you almost everything you'd want to know about
    pipes and that the implementation changed at Linux 2.6.11 - Fedora 32 is
    using the Linux 5.9.16 kernel.

    Turns out I was wrong about buffering: before 2.6.11 the max buffer size
    was 4Kb and now its 64 Kb, which rather points at the Raspbian process scheduler as the cause of your buffering problem: if the scheduler is
    slow to pass control to awk after stuff has been written to the pipe
    buffer, that would explain the delays you're seeing. If this is the case, you'll need to recognise the problem within the program that detects it
    and, if its fatal to the system as a whole, make your program issue a
    "sudo stop" command - in a script something like this should work:

    sudo stop <EOF
    password
    EOF

    Tested from a script that displays the content of /etc/sudoers which, on
    my Fedora system requires a password.

    Raise a bug about the long pauses you're seeing seems appropriate: the
    pipeline should be passing the data on almost instantly.

    BTW, RPi isn't running at present, hence testing under Fedora.


    --
    --
    Martin | martin at
    Gregorie | gregorie dot org

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Jim H@3:770/3 to The Natural Philosopher on Mon Jan 18 17:07:59 2021
    On Thu, 14 Jan 2021 03:46:08 +0000, in <rtoeq0$pt9$1@dont-email.me>,
    The Natural Philosopher <tnp@invalid.invalid> wrote:

    On 13/01/2021 22:04, Joe wrote:


    Lithium cells have a linear-ish discharge curve, from 4.2V down to
    around 3V.

    They do.

    Any sensibly designed lithium battery will cut off its
    output at the chosen lower bound,

    No, it wont. That isn't in the *battery* - that's in whatever is drawing >current off it. I have many lthium packs with no such ptotection



    because if it completely discharges,
    it's dead forever. Not a 'steep' decline, a 'fall off the wall' decline.

    No, again it goes on down to zero, 3V is just approximately the point at >which irreversible damage starts to happen.


    It's a good idea to have some independent means of anticipating this
    point.

    The way to run a lithium in this application is to use a 2 cell lithium,
    a constant voltage constant current charger limited to 8.4V and probably
    no more than the battery mAh capacity divided by one hour... and a
    switched mode 5V regulator to feed the Pi and then somehow monitor raw >battery voltage and switch it all off when it gets to say 6.6V and hope
    that the SMPS and monitoring circuit don't then still draw enough to
    damage the battery, or even if it is take it on the chin and replace the >battery when that happens.

    Yes... with special care to observe that safe cutoff voltage so that
    when power returns the constant current, constant voltage charger
    doesn't charge the battery any faster than it can accept safely. But
    wait... the constant current needs to be all that is needed to run the
    Pi PLUS the additional safe amount to recharge the battery, but... how
    do you assure the Pi and the battery share the current appropriately
    so that the Pi runs and the battery recharges safely? Maybe a bit more
    control is needed to assure a battery voltage above a certain value
    before the Pi is rebooted.

    I would imagine that the whole setup would cost more than the Pi itself

    Likely so, but I've always wondered at comparing the COST of the Pi to
    the cost of supporting equipment vs comparing the CAPABILITIES of the
    Pi (vs desktops/laptops/etc) to the cost of support. One can't really
    expect support costs to scale down proportionally to the lower cost of
    the Pi.
    --
    Jim H

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Joe@3:770/3 to Jim H on Mon Jan 18 20:33:21 2021
    On Mon, 18 Jan 2021 17:07:59 +0000
    Jim H <invalid@invalid.invalid> wrote:

    On Thu, 14 Jan 2021 03:46:08 +0000, in <rtoeq0$pt9$1@dont-email.me>,
    The Natural Philosopher <tnp@invalid.invalid> wrote:


    The way to run a lithium in this application is to use a 2 cell
    lithium, a constant voltage constant current charger limited to 8.4V
    and probably no more than the battery mAh capacity divided by one
    hour... and a switched mode 5V regulator to feed the Pi and then
    somehow monitor raw battery voltage and switch it all off when it
    gets to say 6.6V and hope that the SMPS and monitoring circuit don't
    then still draw enough to damage the battery, or even if it is take
    it on the chin and replace the battery when that happens.

    Yes... with special care to observe that safe cutoff voltage so that
    when power returns the constant current, constant voltage charger
    doesn't charge the battery any faster than it can accept safely. But
    wait... the constant current needs to be all that is needed to run the
    Pi PLUS the additional safe amount to recharge the battery, but... how
    do you assure the Pi and the battery share the current appropriately
    so that the Pi runs and the battery recharges safely? Maybe a bit more control is needed to assure a battery voltage above a certain value
    before the Pi is rebooted.


    The way to do it is to use a purpose-made lithium battery charger IC,
    for a pound or two. A diode, an inductor, a capacitor, a resistor and a transistor are all that is additionally needed, and some ICs
    incorporate the transistor. You then need a power source that will
    provide current for both charging and a reasonable amount to run the
    device itself. The resistor sets the main charging current, typically
    0.3-0.5 of the battery capacity, independently of the device
    consumption. The diode, inductor, capacitor and transistor do the same
    jobs as in any switch-mode power supply.

    --
    Joe

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Pancho@3:770/3 to Martin Gregorie on Tue Jan 19 13:46:18 2021
    On 18/01/2021 15:15, Martin Gregorie wrote:
    On Mon, 18 Jan 2021 14:02:33 +0000, Pancho wrote:

    About -W : according to the manpage for the awk version I'm using,
    5.0.1, -W has nothing to do with waiting for anything. All it does is
    to change the option marker from - to --

    I'm using Raspbian Buster, default awk is mawk 1.3.3.

    Perhaps that explains our different experience. mawk buffering doesn't
    appear to be line buffering by default, i.e it buffers blocks somewhere
    between 2 to 3kB (whatever the correct name for that is).

    What OS are you using?

    Fedora 32 for these tests, which uses awk 5.0.1 - The Buster awk is very old, so raising a bug requesting an upgrade to the latest awk may be a
    good idea.

    Running "man 7 pipe" tells you almost everything you'd want to know about pipes and that the implementation changed at Linux 2.6.11 - Fedora 32 is using the Linux 5.9.16 kernel.


    Yeah that is kind of the point, I'm a mediocre old programmer, I don't
    want to know everything about everything. What I want is an easy life. I
    want a basic technique to work, reliably, environment agnostically as
    possible.

    I think this clarifies in my mind why I wouldn't ever use this technique
    to observe events in practice. It is too fragile.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Pancho@3:770/3 to Martin Gregorie on Tue Jan 19 15:34:00 2021
    On 19/01/2021 15:24, Martin Gregorie wrote:
    On Tue, 19 Jan 2021 13:46:18 +0000, Pancho wrote:

    I think this clarifies in my mind why I wouldn't ever use this technique
    to observe events in practice. It is too fragile.

    So raise a bug to get it fixed: this will help everybody and is, after
    all, why most Linux distros have decent bug reporting facilities. Plus
    its quite a good way of thanking the developers for their work.



    There is not a bug, just different implementations, different behaviour. Different buffering, different arguments.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Martin Gregorie@3:770/3 to Pancho on Tue Jan 19 15:24:43 2021
    On Tue, 19 Jan 2021 13:46:18 +0000, Pancho wrote:

    I think this clarifies in my mind why I wouldn't ever use this technique
    to observe events in practice. It is too fragile.

    So raise a bug to get it fixed: this will help everybody and is, after
    all, why most Linux distros have decent bug reporting facilities. Plus
    its quite a good way of thanking the developers for their work.


    --
    --
    Martin | martin at
    Gregorie | gregorie dot org

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Kees Nuyt@3:770/3 to Pancho.Dontmaileme@outlook.com on Tue Jan 19 17:03:20 2021
    On Tue, 19 Jan 2021 13:46:18 +0000, Pancho
    <Pancho.Dontmaileme@outlook.com> wrote:

    Yeah that is kind of the point, I'm a mediocre old programmer, I don't
    want to know everything about everything. What I want is an easy life. I
    want a basic technique to work, reliably, environment agnostically as possible.

    I think this clarifies in my mind why I wouldn't ever use this technique
    to observe events in practice. It is too fragile.

    FWIW:
    I use gawk to follow logfiles, and the fflush() function to
    flush the output buffers.


    Snippet of Bash script :
    tail -f -n $TLINES $LOGFILE \
    | gawk -f $SCRIPT.awk \
    | sqlite3 $DATABASE

    pseudocode of $SCRIPT.awk :
    process every input line
    console output to "/dev/stderr"
    realtime filtered, reformatted view
    SQL statements to stdout
    statistics
    After processing every input line:
    fflush()

    --
    Regards,
    Kees Nuyt

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Martin Gregorie@3:770/3 to Pancho on Tue Jan 19 16:21:35 2021
    On Tue, 19 Jan 2021 15:34:00 +0000, Pancho wrote:

    On 19/01/2021 15:24, Martin Gregorie wrote:
    On Tue, 19 Jan 2021 13:46:18 +0000, Pancho wrote:

    I think this clarifies in my mind why I wouldn't ever use this
    technique to observe events in practice. It is too fragile.

    So raise a bug to get it fixed: this will help everybody and is, after
    all, why most Linux distros have decent bug reporting facilities. Plus
    its quite a good way of thanking the developers for their work.



    There is not a bug, just different implementations, different behaviour. Different buffering, different arguments.

    Disagree: the delay you're seeing is definitely a bug, though possibly
    its a task scheduler issue. If you run less than a buffer-full of data
    through a pipe there should not be a noticeable delay under a UNIX/Linux
    OS because the buffer is in memory and the task scheduler is a
    multitasking scheduler and so can interleave both the writing and reading
    tasks without any delay except those caused by task switching and being preempted by higher priority tasks.

    You're reporting multi-second delays you can see which task(s) are
    involved: run the delayed pipe again, but this time with 'top' running in another console window to see what programs are active during the delay.

    OTOH Windows 95 pipes always had delays because those worked by:
    - the first program created a new temporary file
    - wrote the whole dataset to a file
    - closed it
    - the second program opened the file
    - read the whole dataset
    - closed the file
    - deleted the file

    It had to do that because those versions of Windows did not multi-task,
    being just a GUI handler sitting on top of MS-DOS 7 and a pipe connecting
    more than two programs could be very slow.


    --
    --
    Martin | martin at
    Gregorie | gregorie dot org

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Pancho@3:770/3 to Martin Gregorie on Tue Jan 19 17:18:21 2021
    On 19/01/2021 16:21, Martin Gregorie wrote:
    On Tue, 19 Jan 2021 15:34:00 +0000, Pancho wrote:

    On 19/01/2021 15:24, Martin Gregorie wrote:
    On Tue, 19 Jan 2021 13:46:18 +0000, Pancho wrote:

    I think this clarifies in my mind why I wouldn't ever use this
    technique to observe events in practice. It is too fragile.

    So raise a bug to get it fixed: this will help everybody and is, after
    all, why most Linux distros have decent bug reporting facilities. Plus
    its quite a good way of thanking the developers for their work.



    There is not a bug, just different implementations, different behaviour.
    Different buffering, different arguments.

    Disagree: the delay you're seeing is definitely a bug, though possibly
    its a task scheduler issue. If you run less than a buffer-full of data through a pipe there should not be a noticeable delay under a UNIX/Linux
    OS because the buffer is in memory and the task scheduler is a
    multitasking scheduler and so can interleave both the writing and reading tasks without any delay except those caused by task switching and being preempted by higher priority tasks.

    You're reporting multi-second delays you can see which task(s) are
    involved: run the delayed pipe again, but this time with 'top' running in another console window to see what programs are active during the delay.


    I think you are missing the point. If I pipe 4095 characters into mawk,
    nothing happens, if a pipe an extra char to make 4096, it prints out.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Richard Kettlewell@3:770/3 to Pancho on Tue Jan 19 19:31:50 2021
    Pancho <Pancho.Dontmaileme@outlook.com> writes:
    On 19/01/2021 16:21, Martin Gregorie wrote:
    On Tue, 19 Jan 2021 15:34:00 +0000, Pancho wrote:
    On 19/01/2021 15:24, Martin Gregorie wrote:
    On Tue, 19 Jan 2021 13:46:18 +0000, Pancho wrote:
    I think this clarifies in my mind why I wouldn't ever use this
    technique to observe events in practice. It is too fragile.

    So raise a bug to get it fixed: this will help everybody and is, after >>>> all, why most Linux distros have decent bug reporting facilities. Plus >>>> its quite a good way of thanking the developers for their work.

    There is not a bug, just different implementations, different behaviour. >>> Different buffering, different arguments.

    Disagree: the delay you're seeing is definitely a bug, though possibly
    its a task scheduler issue. If you run less than a buffer-full of data
    through a pipe there should not be a noticeable delay under a UNIX/Linux
    OS because the buffer is in memory and the task scheduler is a
    multitasking scheduler and so can interleave both the writing and reading
    tasks without any delay except those caused by task switching and being
    preempted by higher priority tasks.

    You're reporting multi-second delays you can see which task(s) are
    involved: run the delayed pipe again, but this time with 'top' running in
    another console window to see what programs are active during the delay.

    I think you are missing the point. If I pipe 4095 characters into
    mawk, nothing happens, if a pipe an extra char to make 4096, it prints
    out.

    Agreed. It is easy to reproduce.

    $ (seq 9999 | head -c 4095; sleep 2; echo) | mawk '{print}'

    This pauses before printing anything whatsoever.

    $ (seq 9999 | head -c 4096; sleep 2; echo) | mawk '{print}'

    This immediately prints whole lines up to 1040, pauses, then prints 104
    (i.e. 1041, truncated).

    $ (seq 9999 | head -c 4095; sleep 2; echo) | mawk -Wi '{print}'

    This immediately prints whole lines up to 1040, pauses, then prints
    (i.e. 1041, truncated).


    This is entirely down to mawk and is nothing to do with the kernel. The
    effect of -Wi is twofold.

    First it disables output buffering. But this is not really relevant
    here.

    Second it causes mawk to read from stdin rather than file descriptor 0.
    This is the key difference. With -Wi, it runs fgets on stdin, and gets stdio’s buffering policy: read as much as possible, but don’t block
    unless progress is impossible. Without -Wi, it uses its own internal
    buffering policy: always read a whole block even if this means blocking unnecessarily.

    I have no idea what the benefit of the latter policy is, it seems to
    make the code a lot more complicated for no clear gain (and it breaks
    your use case). It’s plainly deliberate, so in that sense not a bug,
    although it seems like a bizarre design decision to me.

    I'm using Raspbian Buster, default awk is mawk 1.3.3.
    [...]
    Fedora 32 for these tests, which uses awk 5.0.1 - The Buster awk is very old, so raising a bug requesting an upgrade to the latest awk may be a
    good idea.

    There is no such thing as mawk 5.0.1, Fedora is presumably using GNU Awk
    (which also available in Debian and its derivatives). These are totally different programs and it does not make any sense to compare their
    version numbers.

    --
    https://www.greenend.org.uk/rjk/

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Martin Gregorie@3:770/3 to Pancho on Tue Jan 19 22:15:28 2021
    On Tue, 19 Jan 2021 17:18:21 +0000, Pancho wrote:

    On 19/01/2021 16:21, Martin Gregorie wrote:
    On Tue, 19 Jan 2021 15:34:00 +0000, Pancho wrote:

    On 19/01/2021 15:24, Martin Gregorie wrote:
    On Tue, 19 Jan 2021 13:46:18 +0000, Pancho wrote:

    I think this clarifies in my mind why I wouldn't ever use this
    technique to observe events in practice. It is too fragile.

    So raise a bug to get it fixed: this will help everybody and is,
    after all, why most Linux distros have decent bug reporting
    facilities. Plus its quite a good way of thanking the developers for
    their work.



    There is not a bug, just different implementations, different
    behaviour.
    Different buffering, different arguments.

    Disagree: the delay you're seeing is definitely a bug, though possibly
    its a task scheduler issue. If you run less than a buffer-full of data
    through a pipe there should not be a noticeable delay under a
    UNIX/Linux OS because the buffer is in memory and the task scheduler is
    a multitasking scheduler and so can interleave both the writing and
    reading tasks without any delay except those caused by task switching
    and being preempted by higher priority tasks.

    You're reporting multi-second delays you can see which task(s) are
    involved: run the delayed pipe again, but this time with 'top' running
    in another console window to see what programs are active during the
    delay.


    I think you are missing the point. If I pipe 4095 characters into mawk, nothing happens, if a pipe an extra char to make 4096, it prints out.

    Thats definitely faulty behaviour: pipe operation should not depend on
    how full the pipe is.

    I've just run similar tests on my systems:
    - Fedora 32 on an *GB Lenovo T420
    - Raspbins Buster in a 512MB RPI 2B

    Both systems had full updates 4 days ago.

    Fedora 32 (awk 5.0.1) with 65Kb pipe buffer size

    File size Time to transfer:
    65535 7-10 Ms
    65536 7-11 Ms
    65537 10-31 Ms

    Raspbian (awk 1.3.3) with 40

    65535 67 Ms
    65536 67 Ms
    65537 68 Ms

    4095 63 Ms
    4096 64 Ms
    4097 64 Ms

    If your system can't match that then you have a problem that doesn't show
    up here. How recently have you updated your system?

    If the pipe hangs persist after a software update then you should raise a
    bug on it, which, if the Raspbian bug reporter is any good, also gives
    you the opportunity to see if anybody else has the same problem.

    The Raspbian awk is also quite old: Version 1.3.3 is dated November 1996.
    By contrast, the Fedora 32 awk is version 5.0.1 and dated May 2019

    I think that's old enough to be worth asking about a refresh, too.


    --
    --
    Martin | martin at
    Gregorie | gregorie dot org

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Pancho@3:770/3 to Richard Kettlewell on Wed Jan 20 10:12:00 2021
    On 19/01/2021 19:31, Richard Kettlewell wrote:

    Agreed. It is easy to reproduce.

    $ (seq 9999 | head -c 4095; sleep 2; echo) | mawk '{print}'


    Thanks for the seq example, simple, informative. I had seen that (or
    similar) done before, but couldn't quite remember/figure it out myself,
    I actually tried yesterday.

    [snip]


    I have no idea what the benefit of the latter policy is, it seems to
    make the code a lot more complicated for no clear gain (and it breaks
    your use case). It’s plainly deliberate, so in that sense not a bug, although it seems like a bizarre design decision to me.


    Naively, I would guess prioritising performance for big files, as the
    default.


    I'm using Raspbian Buster, default awk is mawk 1.3.3.
    [...]
    Fedora 32 for these tests, which uses awk 5.0.1 - The Buster awk is very
    old, so raising a bug requesting an upgrade to the latest awk may be a
    good idea.

    There is no such thing as mawk 5.0.1, Fedora is presumably using GNU Awk (which also available in Debian and its derivatives). These are totally different programs and it does not make any sense to compare their
    version numbers.



    To be fair, until this thread there was just awk, as far as I was
    concerned, which is why I see different implementations as fragile.

    It looks as if Ubuntu recently swapped default awk from mawk to gawk.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Pancho@3:770/3 to Martin Gregorie on Wed Jan 20 10:03:49 2021
    On 19/01/2021 22:15, Martin Gregorie wrote:

    I think you are missing the point. If I pipe 4095 characters into mawk,
    nothing happens, if a pipe an extra char to make 4096, it prints out.

    Thats definitely faulty behaviour: pipe operation should not depend on
    how full the pipe is.


    Richard explained it better than me. It's mawk waiting until it has a
    block of 4096 bytes (or EOF). Clearly designed behaviour.

    With 20.04, Ubuntu seems to have switched from mawk to gawk as default
    awk. I don't know if this is Ubuntu specific or Debian. So it's quite
    possible this will be reflected in the next version of Raspbian
    (Rasberry Pi OS)

    This is exactly what I mean by fragile. Someone writes a script to do
    something using awk, an OS update comes along and the app completely
    changes.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Martin Gregorie@3:770/3 to Pancho on Wed Jan 20 10:49:35 2021
    On Wed, 20 Jan 2021 10:03:49 +0000, Pancho wrote:

    On 19/01/2021 22:15, Martin Gregorie wrote:

    I think you are missing the point. If I pipe 4095 characters into
    mawk,
    nothing happens, if a pipe an extra char to make 4096, it prints out.

    Thats definitely faulty behaviour: pipe operation should not depend on
    how full the pipe is.


    Richard explained it better than me. It's mawk waiting until it has a
    block of 4096 bytes (or EOF). Clearly designed behaviour.

    With 20.04, Ubuntu seems to have switched from mawk to gawk as default
    awk. I don't know if this is Ubuntu specific or Debian. So it's quite possible this will be reflected in the next version of Raspbian
    (Rasberry Pi OS)

    This is exactly what I mean by fragile. Someone writes a script to do something using awk, an OS update comes along and the app completely
    changes.

    In Fedora systems the binary is called gawk with awk as an alias
    In Raspbian Buster the binary is /usr/bin/mawk with awk and nawk as
    aliases

    So, the same shell script should run in both places: my test scripts do
    exactly that *and* do not show the long delay you're seeing.


    --
    --
    Martin | martin at
    Gregorie | gregorie dot org

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Chris Elvidge@3:770/3 to Martin Gregorie on Wed Jan 20 11:29:47 2021
    On 20/01/2021 10:49 am, Martin Gregorie wrote:
    On Wed, 20 Jan 2021 10:03:49 +0000, Pancho wrote:

    On 19/01/2021 22:15, Martin Gregorie wrote:

    I think you are missing the point. If I pipe 4095 characters into
    mawk,
    nothing happens, if a pipe an extra char to make 4096, it prints out.

    Thats definitely faulty behaviour: pipe operation should not depend on
    how full the pipe is.


    Richard explained it better than me. It's mawk waiting until it has a
    block of 4096 bytes (or EOF). Clearly designed behaviour.

    With 20.04, Ubuntu seems to have switched from mawk to gawk as default
    awk. I don't know if this is Ubuntu specific or Debian. So it's quite
    possible this will be reflected in the next version of Raspbian
    (Rasberry Pi OS)

    This is exactly what I mean by fragile. Someone writes a script to do
    something using awk, an OS update comes along and the app completely
    changes.

    In Fedora systems the binary is called gawk with awk as an alias
    In Raspbian Buster the binary is /usr/bin/mawk with awk and nawk as
    aliases

    So, the same shell script should run in both places: my test scripts do exactly that *and* do not show the long delay you're seeing.



    In mine, Linux raspi-3plus 5.4.79-v7+ #1373 SMP Mon Nov 23 13:22:33 GMT
    2020 armv7l GNU/Linux,

    # ls -l /usr/bin/?awk
    -rwxr-xr-x 1 root root 537K Sep 14 2018 /usr/bin/gawk*
    -rwxr-xr-x 1 root root 93K Apr 8 2012 /usr/bin/mawk*
    lrwxrwxrwx 1 root root 22 May 27 2020 /usr/bin/nawk -> /etc/alternatives/nawk*

    so mawk is well out of date!

    /usr/bin/awk -> /etc/alternatives/awk -> /usr/bin/gawk

    /usr/bin/mawk -W version says:
    mawk 1.3.3 Nov 1996, Copyright (C) Michael D. Brennan

    compiled limits:
    max NF 32767
    sprintf buffer 1020

    So really, well worth linking awk to gawk if you can.

    --
    Chris Elvidge
    England

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From A. Dumas@3:770/3 to Chris Elvidge on Wed Jan 20 12:18:16 2021
    Chris Elvidge <chris@mshome.net> wrote:
    In mine, [...]

    Ah, you already said the same, sorry.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From A. Dumas@3:770/3 to Martin Gregorie on Wed Jan 20 12:18:16 2021
    Martin Gregorie <martin@mydomain.invalid> wrote:
    In Raspbian Buster the binary is /usr/bin/mawk with awk and nawk as
    aliases

    Not here, or did you not mean aliases to mawk? It's /usr/bin/awk -> /etc/alternatives/awk -> /usr/bin/gawk. /usr/bin/mawk is a separate binary
    with these properties:

    $ /usr/bin/mawk -W version
    mawk 1.3.3 Nov 1996, Copyright (C) Michael D. Brennan

    compiled limits:
    max NF 32767
    sprintf buffer 1020

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Martin Gregorie@3:770/3 to A. Dumas on Wed Jan 20 13:18:50 2021
    On Wed, 20 Jan 2021 12:18:16 +0000, A. Dumas wrote:

    Martin Gregorie <martin@mydomain.invalid> wrote:
    In Raspbian Buster the binary is /usr/bin/mawk with awk and nawk as
    aliases

    Not here, or did you not mean aliases to mawk? It's /usr/bin/awk -> /etc/alternatives/awk -> /usr/bin/gawk. /usr/bin/mawk is a separate
    binary with these properties:

    Well, that is effectively aliases, just selective ones and is what I have
    here. I've newer switched that alternative on my RPi, so I've been
    running mawk as awk.

    Fedora does not use alternatives - just /usr/bin/gawk with /usr/bin/awk
    as an alias.


    --
    --
    Martin | martin at
    Gregorie | gregorie dot org

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Martin Gregorie@3:770/3 to Chris Elvidge on Wed Jan 20 13:02:24 2021
    On Wed, 20 Jan 2021 11:29:47 +0000, Chris Elvidge wrote:

    On 20/01/2021 10:49 am, Martin Gregorie wrote:
    On Wed, 20 Jan 2021 10:03:49 +0000, Pancho wrote:

    On 19/01/2021 22:15, Martin Gregorie wrote:

    I think you are missing the point. If I pipe 4095 characters into
    mawk,
    nothing happens, if a pipe an extra char to make 4096, it prints
    out.

    Thats definitely faulty behaviour: pipe operation should not depend
    on how full the pipe is.


    Richard explained it better than me. It's mawk waiting until it has a
    block of 4096 bytes (or EOF). Clearly designed behaviour.

    With 20.04, Ubuntu seems to have switched from mawk to gawk as default
    awk. I don't know if this is Ubuntu specific or Debian. So it's quite
    possible this will be reflected in the next version of Raspbian
    (Rasberry Pi OS)

    This is exactly what I mean by fragile. Someone writes a script to do
    something using awk, an OS update comes along and the app completely
    changes.

    In Fedora systems the binary is called gawk with awk as an alias In
    Raspbian Buster the binary is /usr/bin/mawk with awk and nawk as
    aliases

    So, the same shell script should run in both places: my test scripts do
    exactly that *and* do not show the long delay you're seeing.



    In mine, Linux raspi-3plus 5.4.79-v7+ #1373 SMP Mon Nov 23 13:22:33 GMT
    2020 armv7l GNU/Linux,

    # ls -l /usr/bin/?awk -rwxr-xr-x 1 root root 537K Sep 14 2018
    /usr/bin/gawk*
    -rwxr-xr-x 1 root root 93K Apr 8 2012 /usr/bin/mawk*
    lrwxrwxrwx 1 root root 22 May 27 2020 /usr/bin/nawk -> /etc/alternatives/nawk*

    so mawk is well out of date!

    /usr/bin/awk -> /etc/alternatives/awk -> /usr/bin/gawk

    /usr/bin/mawk -W version says:
    mawk 1.3.3 Nov 1996, Copyright (C) Michael D. Brennan

    compiled limits:
    max NF 32767 sprintf buffer 1020

    So really, well worth linking awk to gawk if you can.

    I think that requires the Debian maintainers to switch from mawk to gawk
    and then everybody else to wait while the new codes percolates up through
    the distro dependencies.

    However, I have realised that the *content* of the input file may be
    affecting awk, since its designed to work with lines of text.

    I started out by grabbing a suitably large JPG image and adjusting its
    length using the 'truncate' utility, which makes files of exactly the
    specified size. Then I read its manpage a little more carefully and
    discovered that, if it is used to lengthen a file, the new space is
    filled with binary zeros: something awk may not handle well since its
    expecting textual input.

    As a result, I wrote another awk script for creating files of pseudo-text
    of exactly the specified length. Each line in a generated file is up to
    37 characters:

    0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ\n

    and can also generate very short final line if the required amount of
    data isn't an exact multiple of 37:

    empty '' (empty)
    1 char '\n' (1 char
    2 chars '0\n'
    3 chars '01\n'
    ...

    The test results I posted were all the result of running a command like
    this:

    "cat filename.txt | awk -- 'script'"

    that uses 'cat' to fill a pipeline with one of the files created with my
    awk file filling program and 'script' is an awk program that totals the
    lengths of the lines read and displays the total.

    So, questions for Pancho:

    - how did you create your input files?

    - Is there any chance that they don't contain any 'newline' characters?

    I'm asking because its quite possible that awk would stall, waiting for a
    the next character, if it had read several KB of data without finding a
    newline character or EOF.


    --
    --
    Martin | martin at
    Gregorie | gregorie dot org

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Chris Elvidge@3:770/3 to Martin Gregorie on Wed Jan 20 14:14:40 2021
    On 20/01/2021 01:18 pm, Martin Gregorie wrote:
    On Wed, 20 Jan 2021 12:18:16 +0000, A. Dumas wrote:

    Martin Gregorie <martin@mydomain.invalid> wrote:
    In Raspbian Buster the binary is /usr/bin/mawk with awk and nawk as
    aliases

    Not here, or did you not mean aliases to mawk? It's /usr/bin/awk ->
    /etc/alternatives/awk -> /usr/bin/gawk. /usr/bin/mawk is a separate
    binary with these properties:

    Well, that is effectively aliases, just selective ones and is what I have here. I've newer switched that alternative on my RPi, so I've been
    running mawk as awk.

    Fedora does not use alternatives - just /usr/bin/gawk with /usr/bin/awk
    as an alias.



    /usr/bin/awk -> /etc/alternatives/awk -> /usr/bin/gawk
    Not aliases, soft links, AIUI

    alias awk='gawk' would not work if user not logged in (e.g in cron)

    --
    Chris Elvidge
    England

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From A. Dumas@3:770/3 to A. Dumas on Wed Jan 20 14:45:33 2021
    A. Dumas <alexandre@dumas.fr.invalid> wrote:
    Martin Gregorie <martin@mydomain.invalid> wrote:
    On Wed, 20 Jan 2021 12:18:16 +0000, A. Dumas wrote:

    Martin Gregorie <martin@mydomain.invalid> wrote:
    In Raspbian Buster the binary is /usr/bin/mawk with awk and nawk as
    aliases

    Not here, or did you not mean aliases to mawk? It's /usr/bin/awk ->
    /etc/alternatives/awk -> /usr/bin/gawk. /usr/bin/mawk is a separate
    binary with these properties:

    Well, that is effectively aliases, just selective ones and is what I have
    here. I've newer switched that alternative on my RPi, so I've been
    running mawk as awk.

    Fedora does not use alternatives - just /usr/bin/gawk with /usr/bin/awk
    as an alias.

    But I also haven't changed those aliases, or at least don't remember. I
    would find it very strange if mawk was the default awk anywhere, but especially on Debian/Raspbian which I have used for quite a few years and never noticed that awk was linked to mawk. Pretty sure it isn't, in a standard installation.

    Oh right, what Chris said (again...): not aliases but symbolic links.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From A. Dumas@3:770/3 to Martin Gregorie on Wed Jan 20 14:44:16 2021
    Martin Gregorie <martin@mydomain.invalid> wrote:
    On Wed, 20 Jan 2021 12:18:16 +0000, A. Dumas wrote:

    Martin Gregorie <martin@mydomain.invalid> wrote:
    In Raspbian Buster the binary is /usr/bin/mawk with awk and nawk as
    aliases

    Not here, or did you not mean aliases to mawk? It's /usr/bin/awk ->
    /etc/alternatives/awk -> /usr/bin/gawk. /usr/bin/mawk is a separate
    binary with these properties:

    Well, that is effectively aliases, just selective ones and is what I have here. I've newer switched that alternative on my RPi, so I've been
    running mawk as awk.

    Fedora does not use alternatives - just /usr/bin/gawk with /usr/bin/awk
    as an alias.

    But I also haven't changed those aliases, or at least don't remember. I
    would find it very strange if mawk was the default awk anywhere, but
    especially on Debian/Raspbian which I have used for quite a few years and
    never noticed that awk was linked to mawk. Pretty sure it isn't, in a
    standard installation.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Martin Gregorie@3:770/3 to Chris Elvidge on Wed Jan 20 14:51:33 2021
    On Wed, 20 Jan 2021 14:14:40 +0000, Chris Elvidge wrote:

    On 20/01/2021 01:18 pm, Martin Gregorie wrote:
    On Wed, 20 Jan 2021 12:18:16 +0000, A. Dumas wrote:

    Martin Gregorie <martin@mydomain.invalid> wrote:
    In Raspbian Buster the binary is /usr/bin/mawk with awk and nawk as
    aliases

    Not here, or did you not mean aliases to mawk? It's /usr/bin/awk ->
    /etc/alternatives/awk -> /usr/bin/gawk. /usr/bin/mawk is a separate
    binary with these properties:

    Well, that is effectively aliases, just selective ones and is what I
    have here. I've newer switched that alternative on my RPi, so I've been
    running mawk as awk.

    Fedora does not use alternatives - just /usr/bin/gawk with /usr/bin/awk
    as an alias.



    /usr/bin/awk -> /etc/alternatives/awk -> /usr/bin/gawk Not aliases, soft links, AIUI

    alias awk='gawk' would not work if user not logged in (e.g in cron)

    Indeed: that's what I meant.




    --
    --
    Martin | martin at
    Gregorie | gregorie dot org

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Richard Kettlewell@3:770/3 to A. Dumas on Wed Jan 20 15:07:16 2021
    A. Dumas <alexandre@dumas.fr.invalid> writes:
    But I also haven't changed those aliases, or at least don't remember. I
    would find it very strange if mawk was the default awk anywhere, but especially on Debian/Raspbian which I have used for quite a few years and never noticed that awk was linked to mawk. Pretty sure it isn't, in a standard installation.

    If both are installed, gawk wins (priority 10 vs priority 5).
    However mawk is Priority: required, while gawk is Priority: optional.

    So mawk is the default in the sense that it’s certain to be installed
    while gawk is not, but gawk is the default in the alternative sense that
    if you have both, awk->gawk.

    --
    https://www.greenend.org.uk/rjk/

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Pancho@3:770/3 to Martin Gregorie on Wed Jan 20 15:49:49 2021
    On 20/01/2021 13:02, Martin Gregorie wrote:

    So, questions for Pancho:

    - how did you create your input files?

    - Is there any chance that they don't contain any 'newline' characters?

    I'm asking because its quite possible that awk would stall, waiting for a
    the next character, if it had read several KB of data without finding a newline character or EOF.



    I used:

    tail -f testfile | mawk...

    I used vi to generate files, with newlines, which I appended into the
    testfile, or just echo "stuff..." appended into the test file.

    But Richard's example is perfect to demonstrate the problem:

    $ (seq 9999 | head -c 4095; sleep 2; echo) | mawk '{print}'

    Not only does seq quickly generate a byte stream, but it allows you to
    see exactly what byte you are on. I'm not sure why he used the brackets
    () but I left them in as they don't hurt.

    I guess he used the echo because he is habitually tidy :-).

    Maybe the answer is that for reliability I should have used perl instead
    of awk? Maybe perl is more standard?

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From A. Dumas@3:770/3 to All on Wed Jan 20 16:38:42 2021
    Op 20-01-2021 om 16:07 schreef Richard Kettlewell:
    A. Dumas <alexandre@dumas.fr.invalid> writes:
    But I also haven't changed those aliases, or at least don't remember. I
    would find it very strange if mawk was the default awk anywhere, but
    especially on Debian/Raspbian which I have used for quite a few years and
    never noticed that awk was linked to mawk. Pretty sure it isn't, in a
    standard installation.

    If both are installed, gawk wins (priority 10 vs priority 5).
    However mawk is Priority: required, while gawk is Priority: optional.

    So mawk is the default in the sense that it’s certain to be installed
    while gawk is not, but gawk is the default in the alternative sense that
    if you have both, awk->gawk.

    Thanks for the explanation!

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Chris Elvidge@3:770/3 to Richard Kettlewell on Wed Jan 20 15:45:19 2021
    On 20/01/2021 03:07 pm, Richard Kettlewell wrote:
    A. Dumas <alexandre@dumas.fr.invalid> writes:
    But I also haven't changed those aliases, or at least don't remember. I
    would find it very strange if mawk was the default awk anywhere, but
    especially on Debian/Raspbian which I have used for quite a few years and
    never noticed that awk was linked to mawk. Pretty sure it isn't, in a
    standard installation.

    If both are installed, gawk wins (priority 10 vs priority 5).
    However mawk is Priority: required, while gawk is Priority: optional.

    So mawk is the default in the sense that it’s certain to be installed
    while gawk is not, but gawk is the default in the alternative sense that
    if you have both, awk->gawk.


    So, all in all, a TITSUP. (Total Inability To Support Users, Properly).

    --
    Chris Elvidge
    England

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Martin Gregorie@3:770/3 to Pancho on Wed Jan 20 17:19:56 2021
    On Wed, 20 Jan 2021 15:49:49 +0000, Pancho wrote:

    On 20/01/2021 13:02, Martin Gregorie wrote:

    So, questions for Pancho:

    - how did you create your input files?

    - Is there any chance that they don't contain any 'newline' characters?

    I'm asking because its quite possible that awk would stall, waiting for
    a the next character, if it had read several KB of data without finding
    a newline character or EOF.



    I used:

    tail -f testfile | mawk...

    I used vi to generate files, with newlines, which I appended into the testfile, or just echo "stuff..." appended into the test file.

    But Richard's example is perfect to demonstrate the problem:

    $ (seq 9999 | head -c 4095; sleep 2; echo) | mawk '{print}'

    Not only does seq quickly generate a byte stream, but it allows you to
    see exactly what byte you are on. I'm not sure why he used the brackets
    () but I left them in as they don't hurt.

    I guess he used the echo because he is habitually tidy :-).

    Maybe the answer is that for reliability I should have used perl instead
    of awk?

    Dunno about that: awk takes a bit of getting used to because its a
    specialised tool for extracting data from text files: I suggested using
    it in this case because its brilliant for tasks like scanning log files
    for a set of error messages and doing appropriate stuff for each one it
    spots. As you've seen, the code needed to recognise 'battery low'
    warnings in, say, /var/log/messages and issue a 'shutdown -h now' command
    if one is found would be a trivially small program because it
    automatically reads lines and splits them into words before using a regex
    to trigger actions on any lines that the regex matches: about all you
    need to write is the regexes and the code that each of them triggers.

    By comparison C, Java, Perl or Python, ... are all general purpose
    programming languages and so you need to code the file reading loop and (probably) the scan routine that recognises interesting lines in the file
    as well as the code to do something useful when a line is recognised.

    IOW, to spot a 'battery low' message in the /var/log/messages logfile and
    issue a STOP command the Raspbian would be a one liner in awk:

    awk -- '/battery low/ { system("shutdown -h now") }' /var/log/messages

    as compared with at least 10-20 lines in any of the other languages I
    mentioned - and they'll take longer to write simply because the loop and
    the line parser will need to be coded and debugged.

    The above, of course, is assuming that you don't want to simply build the shutdown command into the program that's watching battery voltage

    A lot of programmers would do it that way while others, of which I'm one,
    like to keep different activities in single purpose programs - and,
    though its unlikely in this particular case, would keep them separate
    because you might someday need to let another program trigger the boojum
    as well as the one you're designing right now.

    Maybe perl is more standard?

    Dunno. These days I bet more people would try to write it in Python or Javascript than would use Perl.


    --
    Martin | martin at
    Gregorie | gregorie dot org

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Richard Kettlewell@3:770/3 to Pancho on Wed Jan 20 16:54:17 2021
    Pancho <Pancho.Dontmaileme@outlook.com> writes:
    I used:

    tail -f testfile | mawk...

    I used vi to generate files, with newlines, which I appended into the testfile, or just echo "stuff..." appended into the test file.

    But Richard's example is perfect to demonstrate the problem:

    $ (seq 9999 | head -c 4095; sleep 2; echo) | mawk '{print}'

    Not only does seq quickly generate a byte stream, but it allows you to
    see exactly what byte you are on. I'm not sure why he used the
    brackets () but I left them in as they don't hurt.

    If you take them out it means something different.

    I guess he used the echo because he is habitually tidy :-).

    I am l-) in this case I wanted one more byte to make up the 4096-byte
    input block.

    Maybe the answer is that for reliability I should have used perl
    instead of awk? Maybe perl is more standard?

    Perl only has one implementation (albeit many versions of it) so it’s
    more predictable in that sense. I doubt it has mawk’s weird approach to
    IO, too.

    --
    https://www.greenend.org.uk/rjk/

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From druck@3:770/3 to A. Dumas on Thu Jan 21 10:07:09 2021
    On 20/01/2021 14:44, A. Dumas wrote:
    But I also haven't changed those aliases, or at least don't remember. I
    would find it very strange if mawk was the default awk anywhere, but especially on Debian/Raspbian which I have used for quite a few years and never noticed that awk was linked to mawk. Pretty sure it isn't, in a standard installation.

    I've just checked /etc/alternatives/awk on my bakers dozen Pi's, and the
    two running Raspbian with the Mate desktop are gawk, all the others
    running Raspbian Pixel and Raspbian Lite are using mawk, so it would
    seem mawk is the default on standard installations.

    ---druck

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Pancho@3:770/3 to Martin Gregorie on Thu Jan 21 10:24:46 2021
    On 20/01/2021 17:19, Martin Gregorie wrote:
    On Wed, 20 Jan 2021 15:49:49 +0000, Pancho wrote:

    On 20/01/2021 13:02, Martin Gregorie wrote:

    So, questions for Pancho:

    - how did you create your input files?

    - Is there any chance that they don't contain any 'newline' characters?

    I'm asking because its quite possible that awk would stall, waiting for
    a the next character, if it had read several KB of data without finding
    a newline character or EOF.



    I used:

    tail -f testfile | mawk...

    I used vi to generate files, with newlines, which I appended into the
    testfile, or just echo "stuff..." appended into the test file.

    But Richard's example is perfect to demonstrate the problem:

    >> $ (seq 9999 | head -c 4095; sleep 2; echo) | mawk '{print}'

    Not only does seq quickly generate a byte stream, but it allows you to
    see exactly what byte you are on. I'm not sure why he used the brackets
    () but I left them in as they don't hurt.

    I guess he used the echo because he is habitually tidy :-).

    Maybe the answer is that for reliability I should have used perl instead
    of awk?

    Dunno about that: awk takes a bit of getting used to because its a specialised tool for extracting data from text files: I suggested using
    it in this case because its brilliant for tasks like scanning log files
    for a set of error messages and doing appropriate stuff for each one it spots. As you've seen, the code needed to recognise 'battery low'
    warnings in, say, /var/log/messages and issue a 'shutdown -h now' command
    if one is found would be a trivially small program because it
    automatically reads lines and splits them into words before using a regex
    to trigger actions on any lines that the regex matches: about all you
    need to write is the regexes and the code that each of them triggers.

    By comparison C, Java, Perl or Python, ... are all general purpose programming languages and so you need to code the file reading loop and (probably) the scan routine that recognises interesting lines in the file
    as well as the code to do something useful when a line is recognised.

    IOW, to spot a 'battery low' message in the /var/log/messages logfile and issue a STOP command the Raspbian would be a one liner in awk:

    awk -- '/battery low/ { system("shutdown -h now") }' /var/log/messages

    as compared with at least 10-20 lines in any of the other languages I mentioned - and they'll take longer to write simply because the loop and
    the line parser will need to be coded and debugged.


    tail -f log | perl -ne 'system('shutdown','-h now') if /battery low/'


    The above, of course, is assuming that you don't want to simply build the shutdown command into the program that's watching battery voltage

    A lot of programmers would do it that way while others, of which I'm one, like to keep different activities in single purpose programs - and,
    though its unlikely in this particular case, would keep them separate
    because you might someday need to let another program trigger the boojum
    as well as the one you're designing right now.



    I would prefer an explicit observer mechanism, bundled with journalctl,
    but clearly I'm a MS fanboi.

    It's not the behaviour of mawk I object to, per se, but the fact that
    awk can change behaviour so easily with system config. Really speaking,
    I've only been willing to touch Linux in the last 5-6 years (after a 20
    year Unix hiatus) because Vagrant and then Docker allowed me to create a reliable environment.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From A. Dumas@3:770/3 to All on Thu Jan 21 12:30:52 2021
    Op 21-01-2021 om 11:07 schreef druck:
    On 20/01/2021 14:44, A. Dumas wrote:
    But I also haven't changed those aliases, or at least don't remember. I
    would find it very strange if mawk was the default awk anywhere, but
    especially on Debian/Raspbian which I have used for quite a few years and
    never noticed that awk was linked to mawk. Pretty sure it isn't, in a
    standard installation.

    I've just checked /etc/alternatives/awk on my bakers dozen Pi's, and the
    two running Raspbian with the Mate desktop are gawk, all the others
    running Raspbian Pixel and Raspbian Lite are using mawk, so it would
    seem mawk is the default on standard installations.

    Huh. Thanks for checking. Perhaps I am the "victim" of an automated
    switch from mawk to gawk after installing it as a dependency from other
    things I always install.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From druck@3:770/3 to A. Dumas on Thu Jan 21 13:09:39 2021
    On 21/01/2021 11:30, A. Dumas wrote:
    Op 21-01-2021 om 11:07 schreef druck:
    On 20/01/2021 14:44, A. Dumas wrote:
    But I also haven't changed those aliases, or at least don't remember. I
    would find it very strange if mawk was the default awk anywhere, but
    especially on Debian/Raspbian which I have used for quite a few years
    and
    never noticed that awk was linked to mawk. Pretty sure it isn't, in a
    standard installation.

    I've just checked /etc/alternatives/awk on my bakers dozen Pi's, and
    the two running Raspbian with the Mate desktop are gawk, all the
    others running Raspbian Pixel and Raspbian Lite are using mawk, so it
    would seem mawk is the default on standard installations.

    Huh. Thanks for checking. Perhaps I am the "victim" of an automated
    switch from mawk to gawk after installing it as a dependency from other things I always install.

    I think that is the case, as I don't recall ever deliberately installing
    gawk on those two machine.

    ---druck

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Jim H@3:770/3 to joe@jretrading.com on Thu Jan 21 20:36:01 2021
    On Mon, 18 Jan 2021 20:33:21 +0000, in <20210118203321.00c28e56@jresid.jretrading.com>, Joe
    <joe@jretrading.com> wrote:

    On Mon, 18 Jan 2021 17:07:59 +0000
    Jim H <invalid@invalid.invalid> wrote:

    On Thu, 14 Jan 2021 03:46:08 +0000, in <rtoeq0$pt9$1@dont-email.me>,
    The Natural Philosopher <tnp@invalid.invalid> wrote:


    The way to run a lithium in this application is to use a 2 cell
    lithium, a constant voltage constant current charger limited to 8.4V
    and probably no more than the battery mAh capacity divided by one
    hour... and a switched mode 5V regulator to feed the Pi and then
    somehow monitor raw battery voltage and switch it all off when it
    gets to say 6.6V and hope that the SMPS and monitoring circuit don't
    then still draw enough to damage the battery, or even if it is take
    it on the chin and replace the battery when that happens.

    Yes... with special care to observe that safe cutoff voltage so that
    when power returns the constant current, constant voltage charger
    doesn't charge the battery any faster than it can accept safely. But
    wait... the constant current needs to be all that is needed to run the
    Pi PLUS the additional safe amount to recharge the battery, but... how
    do you assure the Pi and the battery share the current appropriately
    so that the Pi runs and the battery recharges safely? Maybe a bit more
    control is needed to assure a battery voltage above a certain value
    before the Pi is rebooted.


    The way to do it is to use a purpose-made lithium battery charger IC,
    for a pound or two. A diode, an inductor, a capacitor, a resistor and a >transistor are all that is additionally needed, and some ICs
    incorporate the transistor. You then need a power source that will
    provide current for both charging and a reasonable amount to run the
    device itself. The resistor sets the main charging current, typically
    0.3-0.5 of the battery capacity, independently of the device
    consumption. The diode, inductor, capacitor and transistor do the same
    jobs as in any switch-mode power supply.

    This type of charger will work PROVIDED the battery doesn't run so low
    that it's reluctant to take a recharge. In that case you need a REAL
    LiIon charger (what you describe above doesn't even come close) that
    detects this condition and charges at a very low rate until the
    voltage rises to the point that the battery will accept charge.

    Simple chargers like you describe above work for cell phones because
    the logic that protects the battery is in the cell phone.

    --
    Jim H

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Anssi Saari@3:770/3 to druck on Fri Jan 22 10:08:25 2021
    druck <news@druck.org.uk> writes:

    Huh. Thanks for checking. Perhaps I am the "victim" of an automated
    switch from mawk to gawk after installing it as a dependency from
    other things I always install.

    I think that is the case, as I don't recall ever deliberately
    installing gawk on those two machine.

    Could be. I have two Pis, one is running Raspbian 8 and
    /etc/alternatives/awk points to mawk. The other is running Raspbian 10
    and /etc/alternatives/awk points to gawk. I don't think I have installed
    gawk on the latter but I don't really remember for sure.

    By the way, since I haven't seen this mentioned, Debian provides a tool
    to manage those symlinks in /etc/alternatives, it's called
    update-alternatives. IMO it has an obscure syntax and is hard to use but
    I'd say it's still better than manually modifying the symlinks.

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Chris Green@3:770/3 to Anssi Saari on Fri Jan 22 10:25:14 2021
    Anssi Saari <as@sci.fi> wrote:
    druck <news@druck.org.uk> writes:

    Huh. Thanks for checking. Perhaps I am the "victim" of an automated
    switch from mawk to gawk after installing it as a dependency from
    other things I always install.

    I think that is the case, as I don't recall ever deliberately
    installing gawk on those two machine.

    Could be. I have two Pis, one is running Raspbian 8 and
    /etc/alternatives/awk points to mawk. The other is running Raspbian 10
    and /etc/alternatives/awk points to gawk. I don't think I have installed
    gawk on the latter but I don't really remember for sure.

    By the way, since I haven't seen this mentioned, Debian provides a tool
    to manage those symlinks in /etc/alternatives, it's called update-alternatives. IMO it has an obscure syntax and is hard to use but
    I'd say it's still better than manually modifying the symlinks.

    Yes, update-alternatives is an exercise in being as arcane as possible
    I'm sure! :-)

    I think, in general, if you explicitly install gawk or mawk then they
    will set themselves as the preferred (by update-alternatives) version
    of awk.

    --
    Chris Green
    ·

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)
  • From Martin Gregorie@3:770/3 to Anssi Saari on Fri Jan 22 10:29:19 2021
    On Fri, 22 Jan 2021 10:08:25 +0200, Anssi Saari wrote:

    By the way, since I haven't seen this mentioned, Debian provides a tool
    to manage those symlinks in /etc/alternatives, it's called update-alternatives. IMO it has an obscure syntax and is hard to use but
    I'd say it's still better than manually modifying the symlinks.

    I think you'll find that all Linux distros (and the remaining Unixen)
    have a similar tool, though the name may differ between distros: Fedora
    calls the tool 'alternatives'.

    As always, the apropos tool is your friend here: "apropos alternate"
    finds 'alternatives' on my laptop, a Fedora system, and
    'update_alternatives' on my RPi.


    --
    --
    Martin | martin at
    Gregorie | gregorie dot org

    --- SoupGate-Win32 v1.05
    * Origin: Agency HUB, Dunedin - New Zealand | Fido<>Usenet Gateway (3:770/3)