Software Project: ped-editor v1.2.0 shipped!

A new feature release of the editor. The feature is that either when you pipe content into the editor like “find ~ | ped” or you use the F10 shell command dialog the buffer will update in the background for as long as the command produces output. You can press Ctrl-F in that buffer and toggle following, i.e. the screen will automatically show you the end of the stream and the latest content. If you want to follow a file on disk you can use “tail -f filename.log” as the command and the resulting buffer can be used to view this information. All of the regular read only buffer commands are available, copying content, searching with regular expressions, etc…

There are new tests since the underlying StreamEditor class is used in the StreamSelect component and so the existing tests caught a lot of side-effects that I had to clean up as I did this. I have to say having sensitive tests has made this whole thing a lot safer, I probably would have pushed my first cut on this in the past and then discovered the problems later. Now, it all works and is covered with regression tests.

I’ve also updated the wiki usage documentation with up to date screen shots and more information about how the editor works.

Software Project: ped v1.1.2, pypi and packaging

I finally built a standard python package for ped and released it to pypi which means it can be installed now by doing: pip install ped-editor ( just make sure that ~/.local/bin is on your path ). It is a sign of my brain recovering from years of being a manager ( and by definition very distracted all the time ) that I figured out the packaging stuff in a few hours. I have to say I looked at it many times over the years and I just didn’t get it so I put it off.

I’ve even started following a software development process, running and adding tests, building documentation, pushing releases. I’ve been the one making other folks do those things, but I haven’t had to do them myself for a while. There was considerable grinding and screeching as those old wheels started to turn again in my head.

Now that I have this project in shape I’m going to shift my attention to the backup software ( https://github.com/jpfxgood/bkp ) and do all these things there. Hopefully it’ll go a bit quicker since it won’t include the learning/relearning that I did on this project.

Also, I need to rehab a bike for my wife, but that is a different thread…

Software Project: ped the editor, v1.0.0

After all the work getting the editor ( https://github.com/jpfxgood/ped ) to Python 3 and getting the test coverage to 78% and fixing a lot of bugs, I also wanted to do some improvements. I worked hard on getting the optimal redraw to work and the terminal cursor was still being a problem so I turned it off and now render my own cursor. I also had to redo a lot of the code that managed multiple views so that they would render efficiently and also keep their cursor positions when you changed frames or current files and updates in other windows are properly reflected in all the windows. I also made it easier to see which frame is current when there are multiple frames and cleaned up a bunch of over refreshing in the dialog code as well.

I made it so that editing a new file doesn’t create the file until you save it and also the editor now warns you if the file you are editing has been modified on disk.

I optimized the code for recoloring the current file and now it runs very fast reducing or eliminating any flashes when it has to catch up.

So, given all the improvements and the testing I decided to mark that moment with a v1.0.0 release. I’ll try to do patch releases relatively frequently as I find and fix other things ( two seconds after the v1.0.0 release I found several things and fixed them…)

It’s been a lot of fun getting this code cleaned up and improving the testing, I’ve spend hours at a time in a kind of flow state just coding and debugging. I even forgot to get lunch a few times, which I never do…

Software Project Python 2to3 continued... 47% coverage, 70% on the core editor...

Whoo… a lot of bug fixing, a lot of test cases, and getting all the test cases to work in both non-wrapping and line-wrapped mode… The cool part is that now when I’ve made some minor improvements, the tests are showing me areas that are impacted by the changes and catching side effects. I should have done this a long time ago. I’m done with the core editor module and I’m going to do the module that manages windows and switches between files and such at a high level next. I’m hoping to get the overall coverage over 60% with this one… we’ll see…

Here’s the latest video capture with the coverage report at the end:

This gets me up to about 47% coverage but 70% coverage of the core editor.

Software Project Python 2to3 continued... Wow, such bugs...

The process of converting the editor ( https://github.com/jpfxgood/ped ) to Python3 and writing tests is going well. I am fixing a LOT of minor bugs and edge conditions that don’t really come up a lot in my normal use of the editor but are very evident doing detailed test cases. I came up with some cool ways to test features that are only available interactively by using the keyboard macro feature of the editor to feed the required interaction to the editor main processing loop. It reminds me of testing 1-2-3 for DOS where virtually all of the testing was done with keyboard macros and screen comparisons. It also tests the macro feature, and it already helped me fix a hidden bug in the macro feature where it was accumulating null characters when no key was being pressed. If one left macro record on it would have eventually sucked up all of the heap memory, though I guess it would have taken a while on modern machines. I may have to refactor the main test function since it is getting pretty long, I’ll see where it’s at when I finish.

Oh and running pytest inside VSCode is wonderful for debugging both the code under test and the test automation at the same time.

Here’s the simple launch config that I use:

{
    // Use IntelliSense to learn about possible attributes.    
    // Hover to view descriptions of existing attributes.    
    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Current File",
            "type": "python",
            "request": "launch",
            "program": "${file}",
            "console": "externalTerminal"
        },
        {
            "name": "Python: Pytest",
            "type": "python",
            "request": "launch",
            "module": "pytest",
            "console": "externalTerminal"
        }
    ]
}

If you want to watch the tests run I have this handy screen recording:

Software Project Python 2 to 3 continued... so many bugs...

Well, it has been good writing tests for the converted editor code ( wip is here: https://github.com/jpfxgood/ped/tree/master/2to3out ) as it has also shown up a bunch of existing bugs and nits and stuff that was just appearing to work but wasn’t… Plus I got to figure out a nice way to test both the internal editor logic and the rendering at the same time. I’ve been using Microsoft’s VSCode to debug things because it can do it in a detached terminal, very handy for a curses based application like the editor.

I’ve also made some decisions about removing some features that I don’t use anymore ( and I don’t think anyone else would use either, an AIM IM integration, nntp client, svn browser… see… you don’t want those ) and I don’t really want to test them or get them working. So those came out of the code as well, the editor will be much lighter.

pytest is pretty robust I must say, it is handling test code with nested functions without batting an eye, it even shows the nesting in the test failure output… very nice. Anyway onward to hopefully finish the core editor classes in the next couple of days.