warning: Found bdb Packages database while attempting sqlite backend

I updated fedora from 31 to 33, skipping 32 because I live on the edge.

When I tried using DNF, fedora spewed this on stderr warning: Found bdb Packages database while attempting sqlite backend: using bdb backend.

Sweet, no worries! A swift search and you find this advice

In some circumstances [*] users may see messages like “warning: Found bdb Packages database while attempting sqlite backend: using bdb backend.” This is a harmless indication that rpm configuration and what’s on disk disagree. It can be silenced either by running rpmdb –rebuilddb to convert the database to match configuration, or by overriding configuration to match what is on disk (see above).


Great! I ran rpmdb --rebuilddb and it returned quickly. Running more dnf commands, though, continued giving the same error.

I checked journalctl and found the following errors

AVC avc: denied { read } for pid=12602 comm="rpmdb" name="resolv.conf" dev="dm-0" ino=1966429 scontext=unconfined_u:unconfined_r:rpmdb_t:s0-s0:c0.c1023 tcontext=system_u:object_r:net_conf_t:s0 tclass=lnk_file permissive=0
AVC avc: denied { open } for pid=12906 comm="rpmdb" path="/var/lib/rpm/.rpm.lock" dev="dm-0" ino=799036 scontext=unconfined_u:unconfined_r:rpmdb_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=0
AVC avc: denied { lock } for pid=13224 comm="rpmdb" path="/var/lib/rpm/.rpm.lock" dev="dm-0" ino=799036 scontext=unconfined_u:unconfined_r:rpmdb_t:s0-s0:c0.c1023 tcontext=unconfined_u:object_r:var_lib_t:s0 tclass=file permissive=0

I noticed that the errors say the command causing the errors is rpmdb, which is exactly what I was hoping to find. Perfect! I searched for “AVC avc: denied” and found a link to this fantastic page on the gentoo wiki explaining what I was reading https://wiki.gentoo.org/wiki/SELinux/Tutorials/Where_to_find_SELinux_permission_denial_details

From there, I knew I needed to allow the rpmdb command the permissions that were being denied. I saw that audit2allow seemed to be a command I needed to allow these permissions. The man page lists the purpose of the command as

   audit2allow - generate SELinux policy allow/dontaudit rules from logs of denied operations

I copy and pasted the errors from journalctl into a temporary file I called temp.pp, using audit2allow to create a module I called rpmdb and then installed the module into selinux, reran the rpmdb command and repeated the process until I’d dealt with all of the errors.

audit2allow -M rpmdb < temp.pp
semodule -i rpmdb.pp

As an experiment, I tried repeatedly adding the rules to the same temp file and re-adding the temp file with the same audit2allow/semodule commands, worked perfectly.

Reading the man page for audit2allow and this page https://danwalsh.livejournal.com/24750.html (linked to from https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/security-enhanced_linux/sect-security-enhanced_linux-fixing_problems-allowing_access_audit2allow), I’m pretty sure I didn’t need to use the temp file.

Once all the errors were dealt with, the rpmdb rebuilt just fine to

Why Oh Why Oh Why

I always keep this blog in the back of my mind, and I keep paying for it, but I’m never 100% sure why.

I went through a lot of blogging as a late-teen/early-twenties person. I may be trying to maintain the illusion of persistence of identity.

A piece of why I maintain this is so that employers have something to look at so they can know before an interview that I have some idea of what technical things actually are. “Oh, yeah, that person wrote a blog post about a terraform thingy, so they may have actually used it.”

But I sort of want to be a bit more than that sometimes. I want to document my own changing changes again. I’ve been writing a lot of that on paper, but it’d be cool to do it in text.

Honestly, I wonder if what I’m really trying to do is persuade myself to write my own blogging software, let myself get lost in building technical details to better express myself without having to go through the hassle of actually expressing myself.

Time to begin.

HomeBrew Always Uses The System Ruby

I got back to looking at HomeBrew today. Something I noticed last week, though, was that HomeBrew was always using the system ruby, even when I’m using rbenv to set a global ruby. I decided to take an hour or two out to figure out why that was happening.

Last week, I spent some time figuring out how HomeBrew does command loading. It’s pretty simple: there’s the brew command, which is a shell script, and mostly it sets some variables and dispatches to Library/brew.sh.

brew.sh sets up a whole bunch of the HomeBrew environment stuff, then it attempts to find a shell version of the command you’ve asked for. If a shell implementation can’t be found, it dispatches to this:

{ update-preinstall; exec "$HOMEBREW_RUBY_PATH" -W0 "$HOMEBREW_LIBRARY/brew.rb" "$@"; }

brew.rb attempt to find a ruby implementation of the command you’re looking for.

So, it looked like $HOMEBREW_RUBY_PATH is probably the thing that I need to research. How does it get set? In brew.sh, we can see the following code:

if [[ -z "$HOMEBREW_RUBY_PATH" ]]
 if [[ -n "$HOMEBREW_OSX" ]]
 HOMEBREW_RUBY_PATH="$(which ruby)"
 if [[ -z "$HOMEBREW_RUBY_PATH" ]]
 odie "No Ruby found, cannot proceed."

What’s super cool here is that we can see that on Max OS X, there’s an explicit requirement to use the system ruby. Rbenv and the like aren’t even _asked_ about what ruby they’re providing. As an aside, any budding hackers who’re interested may want to inject support for rbenv here.

What problem is this addressing? I suspect that the homebrew devs don’t want to have to support multiple versions of ruby on OS X. In fact, the commit that implements this reads:

Replace /usr/bin/ruby with full Framework path

Rationale: some users insist on replacing the /usr/bin/ruby symlink
 to point to another ruby on their system, which may break homebrew.
 Use the full Framework path instead, which is less likely to be tampered with.

This also reorganizes the brew --config checks to reflect the different path.

Fixes Homebrew/homebrew#12009.

Closes Homebrew/homebrew#12333.

To see why this was implemented, I checked the issues noted in the git log message.


That all seems reasonable enough. I had wondered if this was an issue of trying to reduce maintenance overhead, and it appears that it was.

Hacking on gist-logs. How do we get there?

Just through exploring the codebase a little, I can see that homebrew behaves a little differently if you have the environment variable HOMEBREW_DEVELOPER set. So that’s a neat little win.

I wanted to understand how brew dispatches all the way to gist-logs, so I could be sure I’m implementing a feature in the right place. I started out by just calling gist-logs with no arguments and received a handy error message:

usage: brew gist-logs [--new-issue|-n] <formula>

I searched for the string “–new-issue” in the brew codebase and found it’s only in the gist-logs.rb file. So that’s useful! I then got sidetracked trying to figure out why these two behaved differently:

[james@greentreeredsky brew (issue44706)]$ brew gist-logs
usage: brew gist-logs [--new-issue|-n] <formula>
b[james@greentreeredsky brew (issue44706)]$ brew gist-logs --help
Warning: No help text in: /usr/local/Library/Homebrew/cmd/gist-logs.rb
Example usage:
  brew search [TEXT|/REGEX/]
  brew (info|home|options) [FORMULA...]
  brew install FORMULA...
  brew update
  brew upgrade [FORMULA...]
  brew uninstall FORMULA...
  brew list [FORMULA...]

  brew config
  brew doctor
  brew install -vd FORMULA

  brew create [URL [--no-fetch]]
  brew edit [FORMULA...]

Further help:
  man brew
  brew help [COMMAND]
  brew home

It turns out homebrew has an internal help generation system! If you are writing a new tool, and you use a magic comment beginning with #:, then homebrew will automatically find the help.

Which means that the failure of the –help command is a defect! I’m an hour in and finding something to fix! Yay! I’m going to let myself get sidetracked by this, as it seems like an easy fix.

First, I added the following magic comment to the top of the file:

#: usage: brew gist-logs [--new-issue|-n] <formula>

Now both –help and no arguments behave the same, but at the cost of introducing duplication. I don’t want to introduce duplication, so I want to figure out how a different command handles both the case that someone enters the –help argument and that ARGV is empty.

Fortunately, I’m half way there. –help is handled by the magic comment and a little support in brew.rb. But the way that the empty ARGV is handled internally to gist-logs is with this little piece of logic:

 if ARGV.resolved_formulae.length != 1
  puts "usage: brew gist-logs [--new-issue|-n] <formula>"
  Homebrew.failed = true

So, I need to figure out how to remove that logic, and invoke the help system when there’s no arguments.

I checked the install.rb command to see how it handles this situation. It raises a FormulaUnspecifiedError exception, which is caught in brew.rb (FormulaUnspecifiedError inherits from UsageError). This dispatches to Homebrew#help, which displays the error message and the text from the FormulaUnspecifiedError exception and exits with 1. So changing the code to this seems reasonable:

raise FormulaUnspecifiedError if ARGV.resolved_formulae.length != 1

The problem here’s simple enough, though. The old implementation set Homebrew.failed = 1, but the new code does not set that. The question is, does that change matter?

From what I can tell, Homebrew.failed is used in brew.rb to simply do an “exit 1”. The help implementation also does an “exit 1”, so the command line contract is maintained, and because Homebrew.failed isn’t actually used anywhere in the new call chain, and it wasn’t _doing_ anything in the old call chain, I think the refactor is a win.

I’ll create a branch for this change and submit it to the dev team and see what they think.

And there’s the pull request: https://github.com/Homebrew/brew/pull/217

Hacking On Homebrew

I decided to work a little more independently on my ruby programming chops, and I wanted to be giving a little more back to the free software world, so I signed up for CodeTriage and picked Homebrew as the project I’ll look at. So far:

  • Code’s hosted on github, so that’s easy enough.
  • There’s a feature request against gist-logs that sounds simple enough to start with (https://github.com/Homebrew/legacy-homebrew/issues/44706), and it’s what codetriage recommended anyway.
  • It looks like cloning the remote repo and then just using the brew command in $HOMEBREW_DIR/bin/brew is enough that you can start hacking away on a local copy of the codebase.
  • I haven’t reached out to the devs on the google groups channel yet, but I’ll probably try that later today.
  • The dispatch chain when looking for a script is $HOMEBREW_DIR/bin/brew –> $HOMEBREW_DIR/Library/brew.sh –> $HOMEBREW_DIR/Library/brew.rb
  • rubocop fails the project on its own style guidelines (I think inappropriately, because most of the violations are in formulae not in core, but I’ll ask the dev team)

So, this is kind of fun!

has_many association causing FactoryGirl beta to throw undefined local variable or method for FactoryGirl::SyntaxRunner

Factory Girl is powerful as heck, but it seems like some of the errors for has_many associations just come up as an error from FactoryGirl’s syntax runner. There are a bunch of different causes that come up from a few google searches, but here’s one that I hadn’t seen answered anywhere else. This could be caused by ruby 2.1, or by using the FactoryGirl beta 3.0.0, I’m not too sure.

I have an association:

to_do_item belongs_to to_do_list
to_do_list has_many to_do_items

I wanted a to_do_list factory that contained several to_do_items. You set this up with the after(:create)callback. For more information, see this excellent blog post from thoughtbot. My factory read like this:

FactoryGirl.define do
  factory :to_do_list do

  factory :three_item_list, parent: :to_do_list do
    after(:create) do |list|
      to_do_items << FactoryGirl.create(:to_do_item, {content: "item one", to_do_list: list, created_at: (DateTime.now - 1.hour)})
      to_do_items << FactoryGirl.create(:to_do_item, {content: "item two", to_do_list: list, created_at: (DateTime.now - 1.day)})
      to_do_items << FactoryGirl.create(:to_do_item, {content: "item three", to_do_list: list, created_at: (DateTime.now)})

The factory itself passes a .valid? call, but trying to use it threw the SyntaxRunner exception. After reading and re-reading the FactoryGirl readme a bunch, on a whim I tried out:

FactoryGirl.define do
  factory :to_do_list do

  factory :three_item_list, parent: :to_do_list do
    after(:create) do |list|
      list.to_do_items << FactoryGirl.create(:to_do_item, {content: "item one", to_do_list: list, created_at: (DateTime.now - 1.hour)})
      list.to_do_items << FactoryGirl.create(:to_do_item, {content: "item two", to_do_list: list, created_at: (DateTime.now - 1.day)})
      list.to_do_items << FactoryGirl.create(:to_do_item, {content: "item three", to_do_list: list, created_at: (DateTime.now)})

and my specs stopped throwing a SyntaxRunner exception and started failing as I’d expect them to. So there you go, it might be necessary in a has_many association to explicitly name the receiving object.

Postgres, Homebrew and Rails on Mac OS X Mavericks

I’ve normally been a mysql user, and know just enough admin stuff with mysql to get rails started up. Today, I taught mysql just enough postgres admin stuff to get up and running with postgres. Using homebrew complicates the process slightly.

Here’s what you expect from reading stackoverflow:

brew install postgresql
psql -U postgres

This won’t work. Neither will any solution like:

sudo su - postgres psql

The reason is that a lot of people are writing stack overflow from linux. Most linux package managers perform the following set of operations to install postgres:

  • create a postgres user
  • su to the postgres user
  • install postgres

The documentation for postgres tells you that You will need to become the operating system user under which PostgreSQL was installed (usually postgres) to create the first user account in order to access the default admin account.

Homebrew installs run as the a normal user account, by default. My default account name on my mac os box is james. As such, the default postgres admin login is james:

psql -U james

and that gets me logged in. The prompt shows a # sign when you’re logged in as the superuser role:

james$ psql
psql (9.3.4)
Type "help" for help.


What do you need to do now to get postgres set up for rails? Let’s assume you’re create a rails app called unimportant. Here’s what you need to do:

rails new unimportant --database=postgresql
cd unimportant
cat config/database.yml

Note that in database.yml, you’re specifically told that the database names you need to set up are:

<<: *default
database: unimportant_development

<<: *default
database: unimportant_test

<<: *default
database: unimportant_production
username: unimportant

So you need to make 3 new databases, and a user named “unimportant” who can access the production database. Assuming you’re running everything locally, here’s what you need to know. Postgres comes with some command line tools for making databases a little easier; it’s equivalent to writing the sql yourself:

createdb unimportant_development
createdb unimportant_test
createdb unimportant_production

Postgres also comes with a tool for making new users. We need an unimportant user:

createuser unimportant

You’re not done yet, though, because the unimportant user doesn’t have permission to write to the unimportant production database. First, let’s look at what postgres sees as permissions for our users. From the postgres pdf, chapter 20:
The concept of roles subsumes the concepts of “users” and “groups”. In PostgreSQL versions before 8.1, users and groups were distinct kinds of entities, but now there are only roles. Any role can act as a user, a group, or both.

So we’re actually looking for how to view roles. From the same chapter: To determine the set of existing roles, examine the pg_roles system catalog, for example SELECT rolname FROM pg_roles;
The psql program’s \du meta-command is also useful for listing the existing roles.

Let’s try it out:

james$ psql
psql (9.3.4)
Type "help" for help.

james=# \q
greentreeredsky:unimportant james$ psql -U james
psql (9.3.4)
Type "help" for help.

james=# \du
List of roles
Role name | Attributes | Member of
james | Superuser, Create role, Create DB, Replication | {}
unimportant | Create DB | {}

james=# SELECT rolname FROM pg_roles;
(2 rows)

james=# SELECT * FROM pg_roles;
rolname | rolsuper | rolinherit | rolcreaterole | rolcreatedb | rolcatupdate | rolcanlogin | rolreplication | rolconnlimit | rolpassword | rolvaliduntil | rolconfig | oid
james | t | t | t | t | t | t | t | -1 | ******** | | | 10
unimportant | f | t | f | f | f | f | f | -1 | ******** | | | 16389
(2 rows)

This is a bit difficult to read, sorry, but the last piece is telling us all the permissions that the admin user and the unimportant user have. In order for the unimportant user to actually be capable of managing the rails production db, it needs the rolcreatedb permission. To be able to log in to the psql command console you’ll need the rolcanlogin permission too. The postgresql documentation tells us to use the ALTER sql command to grant this, rather than the mysql GRANT command. The docs for ALTER tell us what roles we can add.

james=# alter role unimportant login;
james=# alter role unimportant createdb;

All done!

getopts and argv in bash

Let’s say you want to use getopts to parse ARGV in bash. Let’s say you also want a non-getopts argument. For example:

./foo.sh argument
./foo.sh -x argument
./foo.sh -abx argument

The non-optional argument is here always given as the last argument.
So, how do you retrieve that last argument after getopts is finished doing its work? Check it out, yo:

while getopts "ghr" opt; do
case $opt in
echo "Invalid option handed in. Usage:"


That last line is the magic. getopts sets $OPTIND to be the index of the last valid option. Index bash’s ARGV, fondly known as $@, in the position of $OPTIND and you’ll get the last item from ARGV.