Properly create and restore your development postgreSQL on production

So maybe you are like me, who is parsing a lot of static data into a development database, make an app around this data and then you don’t want to have an empty database on production, but instead you want to have a full copy of your development database.

After spending a few hours trying to make it work, there is one solution, that yielded exactly 0 errors (yay!)

On your local machine (in the shell, wherever you currently are) :

pg_dump --no-owner your_db_develoment > backup.bak

Then put it in some safe place on your remote machine:

scp backup.bak deploy@192.168.111.222:/home/restore/backup.bak

Make a backup of your production database for safety sake (on your production server, of course)

pg_dump --no-owner your_db_production > /home/backups/04-03-2016/backup.bak

If there is no production database yet on the server, proceed with creating it, elsewise you might need (if the app has users) to cut the current sessions, so check these out:

http://stackoverflow.com/questions/12924466/capistrano-with-postgresql-error-database-is-being-accessed-by-other-users

or

http://stackoverflow.com/questions/1237725/copying-postgresql-database-to-another-server?rq=1

Ok, now we have no database on production machine at all. Let’s create new production database:

postgres=# CREATE DATABASE database_production;
CREATE DATABASE

postgres=# CREATE USER production_db_user WITH password 'qwerty';
CREATE ROLE

postgres=# GRANT ALL privileges ON DATABASE database_production TO production_db_user;
GRANT

Make sure to populate the password with the same password, as set up in database.yml for production user.

That’s it! No need to change the owner. Now let’s restore the backup file:

sudo psql -d mydb_production -U production_user < /home/deploy/restore/backup.bak;

Password for user production_user: 
SET
SET
SET
SET
SET
SET
CREATE EXTENSION
ERROR:  must be owner of extension plpgsql
SET
SET
SET
CREATE TABLE
CREATE SEQUENCE
ALTER SEQUENCE
CREATE TABLE
CREATE SEQUENCE
ALTER SEQUENCE
CREATE TABLE
CREATE SEQUENCE
ALTER SEQUENCE
CREATE TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
COPY 14964
 setval 
--------
 114041
(1 row)

COPY 3151
 setval 
--------
   3236
(1 row)

COPY 28011
 setval 
--------
   7081
(1 row)

COPY 8
ALTER TABLE
ALTER TABLE
ALTER TABLE
CREATE INDEX
CREATE INDEX
CREATE INDEX
ALTER TABLE
ALTER TABLE

As of now I have no issues regarding the one error above.

All hail Gaben… ehm optimization!

Before:

Benchmarking parseQuickLook:
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.534104)
Benchmarking parseQuickLook:
       user     system      total        real
   0.020000   0.000000   0.020000 (  0.610587)
Benchmarking parseQuickLook:
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.422812)
Benchmarking parseQuickLook:
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.405687)
Benchmarking parseQuickLook:
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.396164)
Benchmarking parseQuickLook:
       user     system      total        real
   0.010000   0.000000   0.010000 (  0.371604)
Benchmarking parseQuickLook:
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.411998)
Benchmarking parseQuickLook:
       user     system      total        real
   0.000000   0.010000   0.010000 (  0.387384)
Benchmarking parseQuickLook:
       user     system      total        real
   0.020000   0.000000   0.020000 (  0.673446)
Recommendations report: 
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.000076)
Production cost report: 
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.007484)
Benchmarking parseQuickLook:
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.418086)
Benchmarking parseQuickLook:
       user     system      total        real
   0.020000   0.000000   0.020000 (  0.446162)
Benchmarking parseQuickLook:
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.392303)
Benchmarking parseQuickLook:
       user     system      total        real
   0.010000   0.000000   0.010000 (  0.390188)
Benchmarking parseQuickLook:
       user     system      total        real
   0.010000   0.000000   0.010000 (  0.412362)
Benchmarking parseQuickLook:
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.427071)
Benchmarking parseQuickLook:
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.627583)
Benchmarking parseQuickLook:
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.402546)
Benchmarking parseQuickLook:
       user     system      total        real
   0.010000   0.000000   0.010000 (  0.434892)
Recommendations report: 
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.000102)
Production cost report: 
       user     system      total        real
   0.010000   0.000000   0.010000 (  0.001292)
Benchmarking parseQuickLook:
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.418595)
Benchmarking parseQuickLook:
       user     system      total        real
   0.020000   0.000000   0.020000 (  0.413026)
Benchmarking parseQuickLook:
       user     system      total        real
   0.010000   0.010000   0.020000 (  0.400815)
Benchmarking parseQuickLook:
       user     system      total        real
   0.010000   0.000000   0.010000 (  0.409537)
Benchmarking parseQuickLook:
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.464621)
Benchmarking parseQuickLook:
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.410090)
Benchmarking parseQuickLook:
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.393017)
Benchmarking parseQuickLook:
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.442389)
Benchmarking parseQuickLook:
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.412018)
Recommendations report: 
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.000064)
Production cost report: 
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.004622)
Benchmarking parseQuickLook:
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.408888)
Benchmarking parseQuickLook:
       user     system      total        real
   0.010000   0.010000   0.020000 (  0.417567)
Benchmarking parseQuickLook:
       user     system      total        real
   0.010000   0.000000   0.010000 (  0.461256)
Benchmarking parseQuickLook:
       user     system      total        real
   0.010000   0.000000   0.010000 (  0.410171)
Benchmarking parseQuickLook:
       user     system      total        real
   0.010000   0.000000   0.010000 (  0.405976)
Benchmarking parseQuickLook:
       user     system      total        real
   0.010000   0.000000   0.010000 (  0.396160)
Benchmarking parseQuickLook:
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.436136)
Benchmarking parseQuickLook:
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.400430)
Benchmarking parseQuickLook:
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.394962)
Recommendations report: 
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.000034)
Production cost report: 
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.000094)

After optimization:

Recommendations report: 
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.000046)
Production cost report: 
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.010514)
Recommendations report: 
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.000043)
Production cost report: 
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.000584)
Recommendations report: 
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.000030)
Production cost report: 
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.000774)
Recommendations report: 
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.000064)
Production cost report: 
       user     system      total        real
   0.000000   0.000000   0.000000 (  0.000251)

Hashtag “inlovewithruby” ūüôā

From 14.727 seconds to 0.012306 seconds, that’s 1196 times increase.

Rails eShop – my first big project using Ruby on Rails

[readolog_dropcap ]H[/readolog_dropcap]appy new Rails year, fellow riders!

 

I finished my very first “serious” project – yay! It looks and behaves like a rails eShop (you can find link further down below). Before I go into technical details some intro – this website was created for friends of mine and although it was kind of experimental for me (first real app), it turned out to be quite decent as one might say so.

As with every customer-oriented project, there was a lot of JavaScript involved, which is by far not my strongest language. It started calm and nicely with pure ruby on rails, I could use a lot of stuff I learned from this book – Agile Web Development With Rails. But then… as more and more wished appeared I realized that there is no way around some front-end development.

Enough smalltalk, let’s get straight to the data:

  • Bootstrap Theme used – “Minimal
  • Notable gems: figaro (managing global variables), devise (users), carrierwave (images uploads), sendgrid (for sending any kinds of mails)
  • Database: postgresql (I think it’s pretty a must-have for shops because of hstore¬†– which, as far as I understand makes postgres almost non-relational, because hstore columns allow to store hash/value pairs)
  • Deployment: capistrano, puma, nginx (you can check out my article about deployment here)
  • Other features: protected admin namespace

[readolog_dropcap ]M[/readolog_dropcap]ost notable code

 

JavaScript Рa calculator for end value of a product based on amount, delivery method and size (plus additional services, which are to be differentiated between one for each product and one for a full order). There is some hard coded stuff and of course this piece of code might not be the state of the art, however it works pretty well.

$(document).ready ->

    if $('a.size_input').length != 0
        $('a.size_input')[0].click()

jQuery ->
    $('li a').click (event) ->
        # event.preventDefault()
        $('a').removeClass('active')
        $(this).addClass('active')

    $('form').on 'click', '.add_fields', (event) ->
    	time = new Date().getTime()
    	regexp = new RegExp($(this).data('id'), 'g')
    	$(this).before($(this).data('fields').replace(regexp, time))
    	event.preventDefault()

    get_multiplier = ->
        amount = $('#amount').val()
        multiplier = 1.5
        multiplier = switch
            when amount <= 99 then multiplier = 1.5
            when amount >= 100 and amount < 300 then multiplier = 1.4
            when amount >= 300 and amount < 500 then multiplier = 1.3
            when amount >= 500 and amount < 1000 then multiplier = 1.25
            when amount >= 1000 and amount < 3000 then multiplier = 1.23
            when amount >= 3000 and amount < 7000 then multiplier = 1.2
        return multiplier

    get_delivery = ->
        delivery = $('#delivery').val()
        return parseFloat(delivery).toFixed(2)

    get_size = ->
        # in this method we either select the price value based on the flash
        # drive size, or, if it has a basic price, just simply the basic price
        # from the hidden #basicprice id

        selected_size = $('input[name=volume]:checked').val()
        if (selected_size == undefined)
            return parseFloat($('#basicprice')[0].innerText)
        else
            array = selected_size.split(",")[1].replace(']','')
            return parseFloat(array).toFixed(2)

    get_all_addservices = ->
        # This method delivers all checked addservice price
        # so we can first substract it before applying any multiplications
        addservices = new Array()
        addservicesParty = new Array()

        console.log('inside get all addservices:')
        checkboxes = document.getElementsByName("addservices[]")
        
        for checkbox in checkboxes

            addserviceprice_splitted = (checkbox.value).split(" ")
            # console.log(addserviceprice_splitted[2])
            
            # Here we need to check if the 'apply to whole party is true or false'
            party = addserviceprice_splitted[2]
                

            addserviceprice = parseFloat(addserviceprice_splitted[1]).toFixed(2)
            if checkbox.checked
                parsedPrice = parseFloat(addserviceprice)

                if party == 'true'
                    addservicesParty.push(parsedPrice)
                else
                    addservices.push(parsedPrice)

        if addservices.length > 0
            sum = addservices.reduce((a,b) => a+b)
        else
            sum = 0

        if addservicesParty.length > 0
            sumParty = addservicesParty.reduce((a,b) => a+b)
        else
            sumParty = 0

        console.log('Addservices combined: ' + addservices + '| Addservice to whole party: '+ addservicesParty)
        return [parseFloat(sum), parseFloat(sumParty)]

    recalculate = ->
        # This function recalculates the price
        base = get_size()
        deliveryCoefficient = get_delivery()
        amountCoefficient = get_multiplier()
        addservices = get_all_addservices()

        addserviceOne = addservices[0]
        addserviceParty = addservices[1]

        amount = $('#amount').val()

        console.log('base - ' + base + '| delivery - ' + deliveryCoefficient + ' | add service sum - ' + addservices + ' | amount coefficient - ' + amountCoefficient)

        endPriceOne = (base*deliveryCoefficient*amountCoefficient) + addserviceOne
        endPriceParty = (base * deliveryCoefficient * amountCoefficient) * amount
        endPriceAll = endPriceOne * amount + addserviceParty
        endPriceServices = (addserviceOne * amount) + addserviceParty

        $('#priceValue').text(endPriceAll.toFixed(2))
        $('#priceForOne').text(endPriceOne.toFixed(2))
        $('#priceForAll').text(endPriceParty.toFixed(2))
        $('#priceForAddservices').text(endPriceServices.toFixed(2))

    $('.size_input').click (event) ->
        recalculate()
    
    $('.addservice_checkbox').change (event) ->
        recalculate()
        

    $('#amount').on('input', (event) ->
        recalculate()
        )

    $('#delivery').change (event) ->
        recalculate()

    $("#thediv").click (event) ->
        $("#addservices").toggleClass("reveal-closed").toggleClass("reveal-open")

    $('#load_more_btn').click (event) ->
        event.preventDefault()

Here we have some fancy helper methods which I grabbed from Ryan Bates screencasts. They allow you to add fields on the go:

module ApplicationHelper


	def hidden_div_if(condition, attributes = {}, &block)
		if condition
			attributes["style"] = "display: none"
		end
		content_tag('div', attributes, &block)
	end

	def link_to_add_fields(name, f, association)
		new_object = f.object.send(association).klass.new
		id = new_object.object_id
		fields = f.fields_for(association, new_object, child_index: id) do |builder|
			render(association.to_s.singularize + "_fields", f: builder)
		end
		link_to(name, '#', class: 'add_fields', data: {id: id, fields: fields.gsub("\n", "")})
	end

[readolog_dropcap ]L[/readolog_dropcap]ast, but not least

 

we have of course a fully functional rails eShop with a working cart, order placement, sending confirmation letters to customers and admins, an administration panel for managing products etc. etc.

Unfortunately the owners decided to put this page on ice, therefore it’s unlikely that you will receive some flash drives. However you can check the page out all by yourself –¬†http://souvenirnya-produktsiya.ru/

 

Deploying your Rails app on a VDS

Recently I finally finished¬†development of my very first serious app, which was ready to be deployed on a production machine. This, however, turned out to be trickier than anticipated. I’ve read myself into various tutorials, decided to spin up a droplet on digitalocean with preinstalled Unicorn + Nginx bundle. While this droplet works well with the initial app I had a hard time trying to make

a) multiple sites work

b) any site but the default work

I surely don’t want to assume that unicorn is somewhat hard to handle, it may have been just me, who is unable to accomplish such a task. On the other hand I found that using puma instead of unicorn is far more profitable.

I grabbed this outstanding tutorial –¬†https://www.digitalocean.com/community/tutorials/deploying-a-rails-app-on-ubuntu-14-04-with-capistrano-nginx-and-puma and was able to get the whole app running on production within few hours.

Don’t forget to install ImageMagick¬†sudo apt-get install imagemagick¬†if you suddenly discover, that there are no image uploads on production. Check out this article too –¬†https://chuanhesmile.wordpress.com/2014/12/13/issue-fixed-rollback-transaction-when-uploading-images-using-carrierwave/

This is important because there might be no errors at all and it did cost me a lot of time to figure that out on my own.

My last advise – check out how to create a memory swap because that’s one of the first issues I ran into while deploying my application (I’m using the 5$ droplets with 512 MB RAM for all my projects).