Featured

Things I like about PostgreSQL

I want to write an appreciation post for PostgreSQL! why?! A few days ago, an article Things I hate about PostgreSQL was trending. I think the author is right about the limitations of PostgreSQL in some of the cases. Personally, I have also faced issues because of vacuum and XID-wraparound at Gojek[1]. But I feel there is a need to mention(repeatedly) how awesome Postgres is. Personally, I have been using PostgreSQL for over a decade and I really(really) enjoy working with it. In this appreciation post for PostgreSQL, I am going to talk about few things that I like about it. I am going to focus both on technical and non-technical aspects.

PostgreSQL is 30 years old database

Originated at the University of California, Berkeley in 1992, PostgreSQL is currently one of the most feature rich relational database systems on the planet. It supports numerous data types ranging from arrays to json to user-defined objects. It has a very rich query language that supports most of the major features of SQL:2016. Its newer versions support declarative table partitioning. It works on the majority of operating systems. Currently, at 13 version, postgres has more than 400 releases.

PostgreSQL is a batteries included database

PostgreSQL comes in a single package with everything you need to run a database system. Its defaults work for almost all use-cases if you are starting a project. If defaults don’t work, you can modify the settings/configs through 100s of parameters exposed by PostgreSQL. It is also bundled with utilities like psql, pg_dump. You can use psql to interact with your database or use pg_dump to take backups. Also, there is a client library for all major programming languages, so you can start using the database in your application in few minutes. Also, PostgreSQL is available as managed service on all major cloud providers so you can start using it with a click of a button.

PostgreSQL has a solid open source community

PostgreSQL has been in active development for almost 3 decades. So far there have been over 50K commits on the postgresql repository. It has a very active community on slack and irc. Also, there is an active mailing list where you can get answers to your questions and clarify your doubts. Also, there are User Groups all over the world. Apart from this, there are various events organized around the world every month. Also the community is backed by big companies. Also Postgres license is no nonsense license.

PostgreSQL is being used by almost every giant on the planet

We loved PostgreSQL at Gojek. We had hundreds of PostgreSQL production instances spanning across 18+ products at Gojek. According to stackshare 4000+ companies & 40,000+ developers reported that they use PostgreSQL. The giants like Uber, Netflix, Instagram, Stackshare are using or has used PostgreSQL to server millions of their customers. Not just this, hundreds of big companies have built sufficiently big businesses by providing services/products on top of PostgreSQL. Also according to the Stackoverflow 2020 Survey, 36% of the developers said they are using postgresql for their database needs.

PostgreSQL has very mature transaction support

Transactions are one of the most important features of any database. Transaction helps you reduce your application code by providing guarantees(ACID) at database level. Snapshot Isolation(I in ACID) of PostgreSQL prevents applications from concurrency problems like dirty/non-repeatable/phantom reads. The application developer can trust the postgresql mature transaction capabilities which have evolved over decades.

PostgreSQL is highly extensible

PostgreSQL by design is pretty extendible with extensions. There are hundreds of open source extensions built by the community which are already available for production use cases. The extensions range from monitoring to adding new data types to modifying storage mechanism. PostGIS is one of the widely used extensions which “spatially enables” the postgres server. The extension ecosystem is so mature that you can add functionalities at almost any level of the database – be it at network, sql, storage or even at transaction level.

PostgreSQL has awesome documentation

If there was a prize for best documentation for a sufficiently large project, I think PostgreSQL would be the winner. The documentation is so good and easily navigable. Also the documentation is SEO friendly so you just need to google something like server configuration postgresql and you would certainly get the official documentation link as one of the top results. The documentation is verbose and is tagged with the postgresql version. For example this is the documentation for server configuration of PostgreSQL v13.

PostgreSQL supports non-relational data types

PostgreSQL supports json datatype natively. It supports many operators and functions to query/manipulate the nested json structures. Most of the power of PostgreSQL comes with its relational data modeling but in some special cases storing json structures in a column comes in handy.

Shameless plug

I am currently working on an interactive exercise based PostgreSQL course. The course is aimed at application developers who want to level up their PostgreSQL skills. It is purely about solving real-world challenges we face when working with PostgreSQL. There are more than 100+ real-world scenarios. The scenarios are of moderate to high difficulty ranging from querying huge tables to fixing indexed of a high throughput application to tuning vacuum. You would solve the challenges from your browser in the Interactive Learning Environment(ILE) designed specifically for the course. ILE will have live PostgreSQL databases with programmatically generated traffic to give you a taste of high throughput applications. Feeling interested? Do signup below for early access(no spam guaranteed).


[1] – I no longer work at Gojek

Featured

why should I do production support?

I hear this question almost every month from at-least one product engineer(specially new hires) in the team. In this blog post, I will to try to answer this question with my experience of doing production support at Gojek, an Indonesian superapp with 18+ products.

Context

Currently, I am leading a team called IronBank[1] which is responsible for profitability of Transport product. Transport product is responsible for moving people from Point A to Point B in 4 countries – Indonesia, Singapore, Thailand and Vietnam.

We don’t have dedicated production support folks (yet). From Transport group, we have 3 product engineers on production support. These people are rotated weekly. So this blog post is within the context of a company with decent scale and no dedicated SRE folks.

My initial days at Transport

On my first day, I saw Gojek being down for ~40 minutes because google maps directions api started returning html code in the response ūüôā . It was an intense day. Almost everyone in the company was on production support call. I saw few folks handling the communication with customer support folks, keeping them upto date with the issue. Few folks were trying to reach out to Google support to get the eta on the fix. Two engineers from Transport were building the fallback mechanism of google maps directions api. At that time, Transport was quite lean. The product team had around ~7 product engineers(including frontend) with two PMs. After 2 days of “orientation” my mentor asked me to do production support ūüôā

My production support experience

Initially, I struggled a lot. That was mainly because of three reasons – 1) Gojek didn’t have any process around production support 2) Gojek architecture and scale were huge 3) I didn’t have any real experience of doing production support. Long story short, I was on production support for almost 2 months and I would say those days helped me grow as an engineer. I felt a steep learning curve.

Below are the list of things I learned during production support

  • I developed customer empathy
    • It was very emotional for me to see when thousands of drivers were not getting bids/bookings because one of our database vm restarted ūüė¶
    • One day at around 5pm Jakarta time, thousands of customers were stuck in their office because we forgot to change the data type of one column from integer to big int(integer overflow)
  • I learned about the Alerting & monitoring infrastructure of Gojek
  • I learned how to do write good RCAs
  • Production support helped me figure out the architecture issues in our systems. This also helped me in figuring out the tech debt items
  • I met a lot of (good) product engineers in the company during production support and learned a lot from them
  • I made tonnes of memories while doing production support. It was a fun time

[1] I no longer work at Gojek

why should I commit frequently?

Frustration

Every now and then I see developers making commits of huge size – sometimes 1000s of lines(the above image is from a real incident where one developer added 1000+ lines and deleted 450+ lines in a single commit). Even I have done this mistake multiple times. This small post is my take on why should we commit frequently. At the end of the post, I am going to show a small tool which can help you in building the habit of committing frequently.

Importance of committing frequently ?

  • Having periodic checkpoints because of small commits helps in debugging and makes the history cleaner
  • Small commits make collaboration better. If you commit and push the changes frequently, your team-mates will be able to integrate with your codebase frequently(sometimes even prevent merge conflicts)
  • Small commits encourages us to think iteratively

A plugin to build your habit

If you use zsh, you can use this plugin to encourage the best practises of git. The plugin displays a message in the command prompt when it sees that there are some uncommitted changes. See the image below – the plugin is showing Don’t forget to commit frequently in the command prompt.

Create a chrome extension using vue-js

I found an interesting boilerplate code for quickly building a web extension with Vue, webpack 4, ESLint – vue-web-extension . Follow the steps here to create a boilerplate code for the extension.

This also comes with auto extension reloading on code change using webpack-extension-reloader

I will write a blog post in few days on how to create an overlay on the github profile page using this boilerplate extension code.

Telegram’s new Inline Bots

Telegram today announced inline bots in their blog post.

[Why I am writing this blog post ? I was eagerly waiting for this feature]

From Telegram blog post –

With the new inline mode, bots become omnipresent and can be used as a tool in any of your chats, groups or channels ‚Äď it doesn’t matter, whether the bot is a member or not. Inline bots can help you with dozens of different tasks, like quickly sending relevant GIFs, pictures from the Web, YouTube videos, Wikipedia articles, etc.

So now you can use any bot in the group even without adding it in the groups.

If you are a developer, you can directly head to https://core.telegram.org/bots/inline .

Screen Shot 2016-01-06 at 8.28.09 am

Earlier I created a bot  (@Chuck_Norris_Jokes_Bot ) to get random chuck Norris Joke. Now I will send these jokes to some groups(telegram) to create awkward silences :p

< Happy Coding />

 

Frenzzy.in – First Online Video Challenges Hosting Platform

Valentines Day we launched http://frenzzy.in¬†– Video challenges hosting platform. It is one of its kind place where you can participate in challenges by uploading video links and win¬†exciting prizes. Challenges are of various categories – Fun , Love, Music etc . You can see current challenges at¬†http://frenzzy.in/challenges/¬†. Platform got¬†450+ Users with 2.5k+ ¬†Pageviews withing 2 days of the launch. Challenge – “Best Song Cover” attracted 100+ authentic votes. The platform uniqueness is its simplicity. You just have to follow four steps to win cash prizes worth INR 10k .

  • Step 1 –¬†Upload your video on Youtube.
  • Step 2 – Copy Youtube video link and paste it on challenge page at Frenzzy.in
  • Step 3 –¬†Get the dedicated link of your submission and share it with your friends, bug them to upvote it.
  • Step 4 –¬†And emerge as a winner by getting maximum votes.

Do visit the website and give your feedback.

Graph.js – a Graph Algorithms Library in JavaScript

Hello World!
I am writing a blog post after long time. I was quite a busy these days. One day I thought of developing a game in JavaScript. The game design was such that it required graph algorithms implementation in JS. I googled but found nothing suitable for my work. So I though of implementing my own Graph Algorithms Library in JavaScript.This post is in regard to Graph.js ( Github Repo ) Рa lite weight JavaScript Graph Algorithms Library. Graph.js is easy to use , change and debug. All the algorithms present in it are efficient in terms of space and time complexity.

Algorithms implemented

  • Depth First Search
  • Breadth First Search
  • Dijkstra – Shortest¬†Distance between two nodes
  • Bellman Ford – Shortest Distance between two nodes.
  • Johnson’s Algorithm for all pair shortest path.
  • Prim’s Algorithm for minimum spanning tree of Undirected complete graph.

How to use

Just include graph.js file in your application .
If you want to build the file from the source code you can just run the make command from the top level directory of the project.

Future Additions to the project

  • Implement more algorithms like clustering , max flow , bipartite matching , random graph generation.
  • Add some test suite ( Preferably QUnit )
  • Add¬†a graph visualizer ( Big Enhancement)
  • Some very good code examples of¬†the usage of library.

Note : You can visit Github Repo for example code and other information. As the library is in nascent stage you can find errors . Feel free to fork , make pull requests and report errors.

KillFlash :HTML5

What a newbie thinks about HTML5 ?

HTML5 rumoured as the flash killer is one of the hot topics among the developers. Saying that it is  just a revision(5th) of  the  HTML standard will not serve the power it provided to the developers. I myself being new to web technologies have always thought of making a game . [Disclaimer: I am newbie.]

So we will be making a car racing game.Demo Sourcecode

Language used:

HTML5, JavaScript.

Step 1:

Create a file named index.html and copy the following code into it.


<html>
		<head>
        	<script src="jquery.js"></script>
			<script src="keys1.js"></script>
			<script src="keys2.js"></script>
                        <style>#canvas{border:dashed}</style>
		</head>
		<body>

		</body>
	</html>

In this step we have just created out html file and included the external javascript.

Step 2:
Download and unzip all the files from this folder.

This folder will have three files:

  • jquery.js: Write less do more JavaScript library.
  • keys1.js,keys2.js: These two files are used to handle the key events as handling the key events until now is very dirty.

We are not going into the details of the files keys1.js and keys2.js as it is beyond the scope of this post.If you donot know about jquery you can see the official jquery website or if you want to learn about the jquery you can see the documentation of the jquery or visit w3schools website.

Step 3:

Add the following code into your body tag.

<script>
var CANVAS_WIDTH = 480*2;
var CANVAS_HEIGHT = 280*2;

var canvasElement = $("<canvas width='" + CANVAS_WIDTH +"' height='" + CANVAS_HEIGHT + " '' id='canvas'></canvas>");
var canvas = canvasElement.get(0).getContext("2d");
canvasElement.appendTo('body');
var FPS=30;

var threading=setInterval(function() {

  update();
  draw();
}, 1000/FPS)

function update(){

}

function draw(){

}

</script>

Variables CANVAS_WIDTH CANVAS_HEIGHT are the width and the height of the container where we will be painting everything.
We are creating a canvas with id name =”canvas” and then we are appending it to the body.

The variables FPS represents frames per second.

The setInterval(code,n) is the javascript function used to call the function code after every n milliseconds . So here we are calling the functions draw() and update aftre every 1000/FPS milliseconds. Right now update and draw function are empty.

You can now see your index.html file in the browser. It should look  like this.
pic1
Step:4

Now we will be painting our player on canvas. Just add the following code snippet inside script tag of the body.

var player={

color: "#00A",
  x: 400,
  y: 500,
  width: 40,
  height: 40,
  draw: function() {
    canvas.fillStyle = this.color;
    canvas.fillRect(this.x, this.y, this.width, this.height);
  }

}

Player has the following attributes:

Color: color of the player.

x,y:Coordinates of the current position of the player.

width,height: Height and width of the rectangle.

draw: a function to draw the player on the canvas.

Update the draw function of the step:3.

function draw(){
	 canvas.clearRect(0, 0, CANVAS_WIDTH, CANVAS_HEIGHT);
	 player.draw();
	 }

Again open the index.html file in the browser. It should appear like this. pic2 Step:5 As we have to move our player with the arrow keys so now we will add the event listeners. Just update the function update() of step2.

function update() {

  if (keydown.left) {
    player.x -= 10;
	if(player.x<=0){
		player.x=0;
	}
  }

  if (keydown.right) {
    player.x += 10;
	if(player.x>=CANVAS_WIDTH-32){player.x=CANVAS_WIDTH-32;
	}
  }

}

Whenever the left arrow key is pressed we are decreasing the value of x by 10.And whenever right arrow key is pressed we are increasing the value of x by 10. Now again open your index.html. Your player will now move with your fingers.

Step:6

Now we will be adding the enemy cars in our game. Just add the following code inside the script tag of the body.

var enemy = {

	color: "#000",
	x:480,
	y:10,
	width:32,
	height:32,
	speed:10,
	draw:function(){
	  canvas.fillStyle = this.color;
    canvas.fillRect(this.x, this.y, this.width, this.height);

	},
	 update:	function(){

this.y +=this.speed;
if(this.y>=320*2){
this.x=(Math.random()*960)%960;
		this.y=10;
}
	}
	};

Enemy has the following attributes:

Color: color of the enemy.

x,y:Coordinates of the current position of the enemy.

width,height: Height and width of the rectangle.

draw: a function to draw the enemy on the canvas.

update:a function to update the position of the enemy.

and now update the draw and update functions.

add enemy.update(); in update function.
add enemy.draw(); in draw function.

Open your index.html in the browser.
Now we have to check the collision of the enemy car and the player car. 

Step:7
Just add the following code in the script tag inside the body.
function collision(a, b) {
  return a.x < b.x + b.width &&
         a.x + a.width > b.x &&
         a.y < b.y + b.height &&
         a.y + a.height > b.y;

And then update the draw function.

 
function draw(){
	 canvas.clearRect(0, 0, CANVAS_WIDTH, CANVAS_HEIGHT);
	 if(collision(enemy,player)==true){
clearInterval(threading);
canvas.font='20pt Calibri';
canvas.fillText("Game Over Press F5 ",600,500);
 }
	 player.draw();
	 enemy.draw();
}

Now play

XOR Properties

XOR also known as Exclusive OR  is a logical operation operates on two operands which results in true only if one of the operand has a value true .

In Java,C++,C,C#  it is represented by ^.

In computer Science it is often used for bitwise operations.

               010  XOR 110 =100

               001 XOR 000=001

               111 XOR 111=000

               1010 XOR 1111 = 0101 

There are certain properties of XOR which accounts for its numerous uses.  ( I came to know some of them when I was trying to solve a problem in codechef. )

  • XOR operation is Commutative . A^B=B^A
  • XOR operation is Associative . (A^B)^C=A^(B^C)
  • XOR ing of same number is zero. A^A=0
  • XOR has zero(0) as the identity element. A^0=A

Here are some of the problems where XOR minimizes our effort.

  1. Swapping the numbers.
  2. Finding the number in the list which has odd parity.

1. Swapping

Swapping the numbers is very easy . There are many ways to do this

Method 1: 

int x=10,y=5,temp;
temp=x; // temp=10
x=y; // x=5
y=temp;  // y=10

Method 2:

int x=10,y=5;
x=x+y;   // x=15
y=x-y;   // y=10
x=x-y;  // x=5;

Method 3( Using XOR):

int x=10,y=5;
y=x^y; // 15
x=x^y; // 5(=y)
y=y^x;  // 10(=x)

How is it happening? and Why?

Recall the properties of XOR-

Merge  steps : y=y^x and x=x^y  as x=x^(y^x)

OR


x=x^(x^y) ; // ( As XOR is commutative)

OR


x=(x^x)^y; //( As XOR is associative)

OR


x=0^y=y; //(As A XOR 0=A)

Similarly


y=0^x=x;

2.  Finding the numbers having odd parity.

Example:

There is an array which contains 2n+1 elements in which there are n numbers having duplicates. Only one element is there which which is not paired. How we can find  that element ?

Method 1:

Count the occurrence of every (distinct ) element . If for any element count is 1 ,that is the required element.

Time Complexity- O(n^2);

Can we do better?

Ya, of course.

Method 2:


int result=array[1];  //assuming  array is 1 based

for(int i=2;i<=array.length;i++)

result=result^array[i];

result contains the required element.

If you follow the Use of XOR in Swapping  numbers  then you can easily understand why it is happening that result contains the required element.

Happy Coding!!! ūüôā

Something about Java

Introduction

With the advent of technical Civilization, where competition is the only survival maxim, JAVA is much ahead than its counterpart technologies.¬†Its flexibility is its power and ‘inheritance of legacy’ is buzzword among techno-savvy.

JAVA and C++ have much in common and indirectly depends on C.Java is a hot property of Sun Microsystem, thanks to its godfathers such as James Gosling,Patrick Naughton, Chris Warth, Ed Frank and Mike Sheridan.?

Advantages of JAVA

JAVA offers a number of advantages to developers.

  • Java is simple: Java was designed to be easy to use and is therefore easy to write, compile, debug, and learn than other programming languages.The reason that why Java is much simpler than C++ is because Java uses automatic memory allocation and garbage collection where else C++ requires¬†the programmer to allocate memory and to collect garbage.
  • Java is object-oriented: Java is object-oriented because programming in Java is centered on creating objects, manipulating objects,and making objects work together. This allows you to create modular programs and reusable code.
  • Java is platform-independent: One of the most significant advantages of Java is its ability to move easily from one computer system to another.The ability to run the same program on many different systems is crucial to World Wide Web software, and Java succeeds at this by¬†being platform-independent at both the source and binary levels.
  • Java is distributed: Distributed computing involves several computers on a network working together. Java is designed to make¬†distributed computing easy with the networking capability that is inherently integrated into it.¬†Writing network programs in Java is like sending and receiving data to and from a file.
  • Java is interpreted: An interpreter is needed in order to run Java programs. The programs are compiled into Java Virtual Machine code called bytecode.¬†The bytecode is machine independent and is able to run on any machine that has a Java interpreter. With Java, the program need only be compiled once,¬†and the bytecode generated by the Java compiler can run on any platform.
  • Java is secure: Java is one of the first programming languages to consider security as part of its design. The Java language, compiler, interpreter,¬†and runtime environment were each developed with security in mind.
  • Java is robust: Robust means reliable and no programming language can really assure reliability. Java puts a lot of emphasis on early¬†checking for possible errors, as Java compilers are able to detect many problems that would first show up during execution time in other languages.
  • Java is multithreaded: Multithreaded is the capability for a program to perform several tasks simultaneously within a program. In Java,¬†multithreaded programming has been smoothly integrated into it, while in other languages, operating system-specific procedures¬†have to be called in order to enable multithreading. Multithreading is a necessity in visual and network programming.

Disadvantages of JAVA

  • Performance: Java can be perceived as significantly slower and more memory-consuming than natively compiled languages such as C or C++.
  • Look and feel: The default look and feel of GUI applications written in Java using the Swing toolkit is very different from native applications.¬†It is possible to specify a different look and feel through the pluggable look and feel system of Swing.
  • Single-paradigm language: Java is predominantly a single-paradigm language. However, with the addition of static imports in Java 5.0¬†the procedural paradigm is better accommodated than in earlier versions of Java.


Difference between Java And C++