Create a search…then speed it up by a factor of 50!

One great way to implement a basic search is to put all your data fields into one big array and then perform the search on this one field. Let’s say, you’ve got a Customer model. Open up customer.rb (in app/models/) and write this few lines of code:

To pick only the searched items we’re creating a scope to which we’re passing the search argument. You can add a function, which will turn all separate fields into one big keywords array.

Now, open up your Controller (CustomersController.rb)

If desired, you can add some validation for cases if no search parameter is provided.

This approach won’t yield you amazon like results, but can bring one pretty far.

Step up the game

If you want to make one little step further and do not want to bother with “heavy industry” tools like ElasticSearch, then you can utilize PostgreSQL’s awesome built in indexing features. This approach requires a bit more setup, than the first one, but once you follow those steps, it will appear straight forward.

The main goal is to write our custom SQL query. In the end what we want to achieve is something like this:

Using rails syntax this would mean we need a query like this:

To create this query and keep things clean I’m using an extra class (models/customer_search_term.rb):

Now we need this helper function, “build_for_name_search” (I’ll omit the first one for now).

This function does nothing else rather than building the string for our SQL query. You can verify it by examining the variables – @where_clause is a string, while @where_args is a hash.

Finally, let’s build our controller:

We’re almost there. This search works, but it’s still a bit slow (see below for speed results).

Now we need to create custom indexes on those tables. We will stick to old up and down migration methods, since rails won’t understand our custom SQL migration if we’d stick to “change”.

It’s important to use “varchar_pattern_ops”, since we are not using an exact match, but instead using a like operator. Now, if we fire up rails dbconsole and perform  EXPLAIN ANALYZE on both methods, we can see the difference.Ama

This was the original result:

After spicing things up:

This is an increase by a factor of 243!

Connect to remote postgreSQL database using Ruby

Ever wrote a script, meaning you needed to connect to a remote PG database using PORO (plain old ruby object)?

This can be quite tricky as I found out. Here is a how to.

Let’s say you are parsing some page on the interwebs.

Now you need to prepare you postgres config on the remote machine to allow connections from your current local machine (per default postgres allows only localhost connections, meaning from remote machine to itself).

There is an amazing tutorial on this page – http://www.cyberciti.biz/faq/postgresql-remote-access-or-connection/

I will sum it up:

Edit  $ nano /var/lib/pgsql/data/pg_hba.conf It might be in some other different folder. If that is the case, just search for “pg_hba.conf” –  find / -name pg_hba.conf

Now search for your own ip and paste it in pg_nba.conf

(Replace 192.168.1.0 with your obtained ip. I left DNS mask unchanged.

Now find and edit  $ nano /var/lib/pgsql/data/postgresql.conf

Changing this value –  tcpip_socket = true . Now comes the important part – if you don’t find any tcpip_socket value, then DON’T add it! Instead search for

And change listen_addresses = 'localhost'  to listen_addresses = '*' .

That’s it! Restart # /etc/init.d/postgresql restart postgreSQL and try connecting  $ psql -h 82.102.42.56 -U username -d database where:

Now we’re all ready to rock and roll!

 

 

Properly create and restore your development postgreSQL on production

So maybe you are like me, who is parsing a lot of static data into a development database, make an app around this data and then you don’t want to have an empty database on production, but instead you want to have a full copy of your development database.

After spending a few hours trying to make it work, there is one solution, that yielded exactly 0 errors (yay!)

On your local machine (in the shell, wherever you currently are) :

Then put it in some safe place on your remote machine:

Make a backup of your production database for safety sake (on your production server, of course)

If there is no production database yet on the server, proceed with creating it, elsewise you might need (if the app has users) to cut the current sessions, so check these out:

http://stackoverflow.com/questions/12924466/capistrano-with-postgresql-error-database-is-being-accessed-by-other-users

or

http://stackoverflow.com/questions/1237725/copying-postgresql-database-to-another-server?rq=1

Ok, now we have no database on production machine at all. Let’s create new production database:

Make sure to populate the password with the same password, as set up in database.yml for production user.

That’s it! No need to change the owner. Now let’s restore the backup file:

As of now I have no issues regarding the one error above.