CSVfix patches for regex and exec
CSVfix is a tool for manipulating CSV files. Along with the usual column re-ordering and filtering, CSVfix offers a powerful per-cell data transformation using a simple expression language, as well as regular-expression for string matching and editing. And if this is not enough, CSVfix can execute external process - for every cell that needs to be processed. And oh, it's available for Windows too!I find this tool to be very handy in what I need to do, so when I encountered a bug in its regex processing (for its "edit" command), I immediately checked if there is any updates to this tool. Unfortunately, its development seems to have ceased in 2015; and no other people seem to have picked up the development (I did find some forks, but they were all older copies from when it was still hosted in google code).
So I set out to figure out about the problem and hopefully rectify it. I found that the problem was in its regex library, which was a home-grown library (apparently adapted from an algorithm book). It is 2020 as of this time of writing, and C++ now comes with its own STL regex library (std::regex). I decided to rip off the custom regex lib and replace it with the STL regex instead, while keeping the rest of the class interface identical, therefore no other part of the code needed to be changed. This instantly fixed the problem, and as a bonus, now we can use ECMAScript-compatible regex instead of just the basic regex.
Later, I found out that the "exec" command also had a bug (the flag "-ix" did not work properly), so I traced this and fixed it too.
Oh, and during the process, I tried to run its testsuite - and while most of them passed, some did fail. Mainly because of CRLF/LF inconsistencies, so I changed those the test data to use LF. It is also a warning that this tool only works with platform "newline" - CRLF in Windows, and LF in Linux - so if files were to be exchanged between platforms, they must be properly translated first before use.
Here are the individual patches.
- regex patch
- exec patch
- test-case patch
They apply on top of the commit 93804d4 from 2015-02, which was the latest when I wrote this. They are licensed in the same way as the original CSVfix is licensed.
If CSVfix is not powerful enough for you, there are other similar tools:
1. miller is a tool in very similar spirit with CSVfix, but it is (much) more sophisticated. Its "data transformation language" looks more expressive than the one in CSVfix. If you have a problem you cannot solve with CSVfix, miller will probably help you. As a bonus, it is still in active development - that means bugs will be squashed. It is written in C, you will need to compile it if it is not in your package repository. (Fatdog, naturally, has it in its repository).
2. csvkit is a collection of tools that more or less perform the same functions as CSVfix. It supports direct conversion to/from Excel files, importing/exporting into databases (sqlite and postgresql as documented, perhaps others too), as well as running direct SQL queries from CSV files (and databases too). It is written in Python3 so you can install it using pip3. Fatdog has this in its repository too (so you can install it using package manager instead of pip3).
3. rbql basically enables you to run SQL-like on CSV files; but its power is its ability to run python (or javascript, depending on which backend you choose) code for every cell. Fatdog has it in its repository too, although you can just use pip to install it, if you don't run Fatdog.
No comments - Edit - Delete
The Good Book
Everyone keep two books. The "good book", and the "bad book". People that we know, we like, and generally we like to be associated with, is in our "good book". People that have hurt us, or people who behave in a manner that we despise, on the other hand, get their names written in our "bad book".Being listed in either book have consequences, both good and bad.
Now, God carries two books too. Would you rather be in men's good books, or God's good book? How do you live your life - are you striving to be in God's good book, or to be in other men's good books?
No comments - Edit - Delete
Welcome to the new decade ... or not!
This new year 2020 is a new year that is divisble by ten. It changes the second digit from "1" in 2019 to "2" in 2020.A lot of people are wishing me happy new year and also welcoming me to the new decade. I am grateful for their well-wishes.
Except for one thing.
2020 is not the start of the new decade. Just like year 2000 is not the start of the new millenium. The new decade starts at year 2021. Just like the 21st century starts at 2001, not at 2000. Year 2000 belongs to the 20th century.
In case you don't see why: it is because we start counting our calender at Year 1 AD. So the first decade - all ten years of it - would be the counting numbers from 1 to 10. The second decade starts at year 11 AD.
But of course, I'm being pedantic. Who cares anyway. Calendars have always been subject to various "adjustments"; and different cultures use different calendars. Those still using Julian calendars have not celebrated the new year yet, it is still 9 days away at the time of writing.
PS: We do not have Year Zero. The year before 1 AD is year 1 BC.
No comments - Edit - Delete
Fatdog64 810 Beta is Released
Maintenance release.Release Notes and Forum announcements
Download locations: ibiblio.org, aarnet, uoc.gr, nluug.nl.
No comments - Edit - Delete
The Author of Life
I am an author. I write both fiction and non-fiction works.When I write fiction stories, I make and write my own characters. These characters are people who live in my fictional world, fictional universe, oblivious to the fact that they are all fictitious. For them, everything that happens in their world is as real as it could be.
Now, of course, those characters I have created: they are not independent of me. They exist because of me, they exist in me. Eventually, I do get to decide that they see, that they hear, what they feel, what they do. I determine their fate, their destiny. I set how life is going to be for them, and what revolves in their world.
But they are not just characters, or puppets. In my mind, they are alive, and I'm only writing part of their life that I happen to see. Not only that, I love my characters, and I care about what happens to them. Certain characters are lovable, and some are detestable - but I certainly care about all of them.
I wish - if it were ever possible - to actually meet my characters, in their own universe. See how they live, how they feel. Feel their joy, and suffer their sadness. Be one of them. And tell them, how much I care and love all of them. That all of their life has meaning to me, their author.
---
If you are an author, or an artist of any kind that loves your own created arts, it is easy to see that you, too, is a work of art of an omnipotent Author, that is, God.
When God paints, you see the flowers, country side, and the star constellations.
When God sculpts, you see the mountains and the Laniakea supercluster.
When God builds, you see the quarks self-assembling to atoms and the visible universe.
When God engineers, you see protein machines and solar systems and life.
And when God writes, you see yourself.
You exist because of Him, your Author.
He cares about you, more than you care about your own creation.
He loves you so much that He gives you life, and more: something that you can never give to your own creation: a free and independent mind to decide what you want to do about Him - whether to love Him back, or to reject and deny Him.
He loves you so much that He came into this world, to feel its joy and suffering, and to tell you that He love you, and to tell you that your life is meaningful, and that there are more to your life than just this world; He showed the way to Himself.
I thank my Author for giving me life.
I thank my Author for loving me more than I love myself.
I thank you, Lord Jesus, and I long to see you face-to-face when the time comes.
No comments - Edit - Delete
Life is like a harddisk ...
With every new day, we savour new experience... keeping it in our memory. As time passes, we buried our moments deeper in the hierarchy of directories. Over time, we forget what we know.Sometime, we feel that life is too much for us - that's when it's time to dig into our repositories and delete unnecessary files and folders, so that new experience can come in.
There are times that we still feel burning anger and hatred or sick to the stomach of somebody or something, though we thought we have forgotten it... it's because the file is still in you, in the recycled bin or trash folder. You need to really empty it to let go, and life will begin a fresh.
The only thing is, we cannot buy a new life like we replace a damaged harddisk or upgrade its capacity... but then, who needs to upgrade? Unlike a harddisk that comes with a predefined storage size, our capacity to absorb experience and live to the fullest is unbounded - provided we know how to not keep the bad stuff.
---
Originally posted 18 August 2006, before the advent of SSD.
No comments - Edit - Delete
What is reality
What’s reality, he thinks. Alone, in this world, he can no longer differentiate between dreams and reality. He has no reference to compare things with, and when this happens, there is no way to weight any evidence against any standard. He shouts, and echoes comes back to him, and he does not know whether someone hears his shout or not. But he cannot be sure whether he just thinks he shouts or he actually does shout. Nothing to check against. It’s the point of when and where reality blurs with dreams and nightmares.---
The Book of Ten Children, Reunion:4. Originally posted in 11 November 2006.
No comments - Edit - Delete
Tomorrow is promised to no one
“I am still making peace with them, every single day,” she said softly. “That’s why I don’t want you to go when you were still upset with me. If bad things happen to you – perhaps you lose concentration in your battle against the bad guys, because of me – then really, I’m not sure I can live with it. It’s already difficult with my parents. I can’t let it happen to another person that I really care about. At least, before you go, I want to have peace between us. I can’t wait until the next time you come again, because tomorrow is promised to no-one.” she said with trembling voice.---
The Book of Ten Children, 5:5
No comments - Edit - Delete
Fatdog64 801 is Released
It is a rolled-up updates, containing bugfixes and minor feature updates.Release Notes and Forum announcements
Get it from the usual locations:
Primary site - ibiblio.org (US)
nluug.nl - European mirror
aarnet.edu - Australian mirror
uoc.gr - European mirror
ISO Builder: Get it from here: http://distro.ibiblio.org/fatdog/iso/builder/ and choose the builder dated 2019.05 and the package list for 801.
Enjoy.
No comments - Edit - Delete
Today is Ash Wednesday
Count your blessings!No comments - Edit - Delete
