Daily Dev Notes 2024/06/01

More testing, testing, testing...

I've been really focusing on the bit of code I've got for getting and setting application-level settings. While the code itself seemed fine, I was running into all sorts of weird bugs when running my tests.

This has really been more of a learning exercise about how pytest works. I would get strange behaviour - e.g. I would run one test in VS Code and it would work, but then I'd run all the tests in a file, and the same test that passed earlier would now fail. And then I'd finally get all the tests to work in VS Code, but then I'd run pytest in the terminal and would get a bunch of failures.

I still don't understand this well enough to explain exactly what's going on, but it had to do with the way I was changing application settings in the tests. It seems like pytest runs all the tests sequentially as part of a single process, and if you change settings in one test, they carry forward into the next test. I'm still not sure if this is due to my code to get and set settings, or if it's just the way pytest works, but I have a way around it now.

At the start of each test, I put some asserts to ensure that the settings I'm interested in for that specific test, are set to the ones I expect them to be according to the test settings file. Then I do my testing, and if I need to change any of the settings during that test, I make sure to set them back to the default at the end of the test. I feel like there's probably an easier way to do this, but it works for now.

The other issue that I discovered (which seems obvious in hindsight), is that the Django URL configuration is only loaded once at the start. This was causing me issues too, because I have certain settings that affect the URL configuration, and while I don't yet have any way to change these settings on the fly in production, I was changing them during my tests and then wondering why the URL configuration wasn't being updated.

So for now, I just removed that code from tests, since it was testing unsupported scenarios right now. At some point in the future, I will add the ability to change settings on the fly and at that point in time, I will need to figure out a way to reload the URLs config. I found a bunch of examples from other people who ran into this same issue, so it shouldn't be a problem.

But this has been a good learning experience.