Archive for the ‘Work’ Category

Useful command to test speed of a container, vm, or system

Monday, March 6th, 2017

I’ll be breaking down the following command part by part:

time dd if=/dev/zero of=test.dat bs=1024 count=100000

 

What does time do? It runs a process and then captures how long it took to execute.

What about DD? Well, it’s a command that copies data from a standard input to a standard output.

What about the params if, of, bs, and count?

“if”: It’s decently obvious, but “if” specifies the input, in this case we’re taking input from a special file that provides as many null characters as there are read from it; an infinity file of sorts.

“of”: It’s the output file.

“bs”: Byte size

“count”: the number of blocks

So all together, the command writes 100,000 blocks of 1,024 bytes of binary zeroes into the file of “test.dat”. In other words, generates a 100 MB file. This command allows you to generate a 100 MB file and test the  IO performance of a system. As we move towards a world we’re optimizing the crap out of everything, this is a very useful command to know.

Amazon S3 Outage

Tuesday, February 28th, 2017

Today’s post is regarding https://techcrunch.com/2017/02/28/amazon-aws-s3-outage-is-breaking-things-for-a-lot-of-websites-and-apps/

These type of occurrences are becoming more and more common. Tons of company has placed a ton of faith into the Amazon ecosystem, and time and time again, it looks like Amazon has let them down. When these things broke, it broke at a MASSIVE scale (AWS outage knocks Amazon, Netflix, Tinder and IMDb in MEGA data collapse, https://www.theregister.co.uk/2015/09/20/aws_database_outage/ )

http://research.omicsgroup.org/index.php/Amazon_Web_Services

There were other outages in 2012, 2013, and probably more unlisted. I think it’s an interesting challenge that Amazon is tackling, and I feel like more and more of the web is putting all of their eggs into one giant basket.

I wonder, if we were to build a truly scalable, and unlikely to be impacted system, maybe it might make sense to diversify the system’s infrastructure to utilize multiple services. Maybe some redundancy at the DNS layer, then some more at the LB, some more at how things are replicated, localized and so on… Just something to reflect on due today’s outage, “How can I prevent my organization from being impacted by this?”

Hybris vs Magento

Friday, September 26th, 2014

“We’re on Magento, but we need to upgrade to Hybris!”

“Nothing is true, everything is permitted

I went to magento.com and hybris.com and I took a look at two companies, and then did a benchmark on the two companies. Which of the following do you think is the “better” version?

bloom Oakley

 

The slower loading one is actually Hybris. The faster one is Magento. People are often quick to dismiss languages, technologies, and softwares. I say nay! Try to figure out things first before you throw all those “extra screws”. It’s important to do a cost-benefit analysis on MANY fronts.

Don’t buy into hype. Too much of this world is built upon inefficiencies. Do understand that often times, interests conflict, what is in your best interest isn’t in their best interest.

Hybris is built in JAVA, JAVA has many pros, but one of the cons is that developers are hard to find, and it’s not exactly the fastest to code on either. Magento is built on PHP, many cons, but one of the pros is PHP developers are plentiful and projects can be built quickly and often times, very cost-effective.

Just understand that the more complex and inaccessible your environment, the harder it is to scale it. You’ll run into issues into many forms of scaling issues, whether it’s code, load, or human-capital. Switch to a solution only after carefully assessing the pros and cons of it, this choice MUST be made extremely carefully because the impact of this decision is extremely far reaching. Also understand that simply because certain things are “best practices” doesn’t necessarily mean that it’s the “best practice” for your company and situation.

Only one way to do things? The cat would disagree

Friday, September 26th, 2014

I had a discussion with an industry peer today, regarding databases. Two conclusions he arrived at, which are right, but also wrong. One, “strings have no business being in a SQL statement”, two, “IDs have no basis being in a mapping table”. From a peer data storage and efficiency perspective, you’re correct, but from a practically perspective, you’re wrong. The statement about IDs being in a mapping table, from a peer database perspective, you’re correct, and from a real-world perspective, you’re wrong.

Strings have no business being in a SQL statement

The point of readability is to provide the ability to deduce, at a glance, as much information as reasonably possible. So lets say we have the following database table structure:

graph

 

How would you query all the articles of a section? My response is:

SELECT * FROM Section S
JOIN SectionArticleMap SAM ON S.idSection = SAM.Section_idSelection
JOIN Article A ON A.idArticle = SAM.Artcile_idArticle
WHERE Section.Title = ‘Name’

The only response he thinks is acceptable is:

SELECT * FROM SectionArticleMap SAM
WHERE Section_idSection = 1
AND Article_idArticle = 1

He claimed a string has no place being in a SQL statement, he believes there’s only one correct way, and I’m sorry, but he’s wrong. He favors IDs because it’s immutable, and he believes they will remain longer, which is true, but if you look at categories, they’re represented in names, and not IDs. In a sea of SQL statements, I would have to do a lot of grunt work to figure out exactly which section the statement is tied to, if I wanted to re-use, I’d have to figure how which ID to replace it with. The prior allows me to easily figure out the section and re-use the query. The section is called “Name”, and if I need to re-use the statement for another section, I simply change the name.

I’m not saying the prior is THE CORRECT way of doing things, nor am I claiming the later is the INCORRECT way of doing things. What I’m claiming is the strong statement that ‘such things have no business being in a SQL query’ is wrong. The prior is clearly easier to understand than the later. I know at a glance that I’m fetching articles for a section titled “Name”, the later, I’ll have to do some additional queries, and if the titles aren’t maintained in the DB, but in the code, then some code diving, and if the DB structure somehow became unsynced with the code, then some nightmares are due to follow. There are pros and cons for every approach.

IDs have no basis being in a mapping table

I basically add an ID to all tables now and days for cross-platform compatibility. I informed him during my time as a professional developer, I’ve come across scenarios that merited an ID being in a mapping table, in which he countered, that he’s been working professional for 25 years and there is never a case for an ID column in a mapping table, and anything requiring it is just crap code. It appears that during his time he might not have dealt with the need for many different codebases to interface with the same database, or at the very least, not CakePHP. “By convention the ORM also expects each table to have a primary key with the name of id" (http://book.cakephp.org/3.0/en/orm/table-objects.html)

From a database perspective, it’s very easy to say that the ID as a primary key takes up unnecessary space, and is bad practice, but once you factor CakePHP into the picture, then having an ID IS the best practice.

Is CakePHP crap code? I personally don’t think either CakePHP or any software built on-top of it is crap code, there are always room for improvement, but without understanding the rhyme or reason of why things are the way they are, I’m hesitant to claim things as broken.

I’m not a big fan of people with high-technical responsibility being extremely closed minded. Certain solutions aren’t ideal for one-case, but might be ideal for another, which is why in academia, you’re going to hear a lot of “it depends”. People whose lives involve wisdom and learning, often time know that there’s never a clear-cut answer for everything, and everything depends on other factors, why then, is the world so littered with single solution answers?

Managerial Assessment

Thursday, July 3rd, 2014

It’s that time of the project again, something went wrong, and a goat needs to be sacrificed. As a person who is often found to be in charge of projects, I hold my bosses to the same standards as I hold myself, and underlings. If something goes wrong, the problem goes from the bottom, all the way to the top of the chain.

In a simplistic example, assuming there is a dev team, a team lead, a CTO, and then the CEO. If the project fails apart, and there is a firing decision, the CEO MUST have a team debrief. Every single member involved needs to write in their own opinion, what happened. Sure, a project could’ve failed because someone on the bottom didn’t know what they were doing, but at the same time, isn’t it the team lead’s job to make sure they knew? Then isn’t it the CTO’s job to make sure the team lead’s on task? Isn’t it the CEO’s job to make sure the CTO is capable of such actions?

Fact of the matter, incompetence happens at all levels of a corporation and company. Just because there is a scapegoat doesn’t mean the issue has been taken care of. You have a termite infestation, you’ve killed a termite, but the infestation still exists.

As a CEO, you should gather data on various people’s perspective on what the issue is, and formulate  your own decision. You have to get a perspective of how things are looking down, and then another perspective of how things look like from below. Just like a game of “communication”, if you don’t know what message the very end received, then you don’t know the message was corrupted along the way, in fact, unless you investigate the “nodes”, you won’t even know where and when things got corrupted. Not debriefing is like allowing your ship be sailing through iceberg ridden waters, without checking for icebergs.

Scapegoating will buy bad management time between the current SNAPFU and the next, but if you were to catch the manager in the act of scapegoating, you can prevent yourself from losing some very talented individuals (human capital), at the same time, preventing the bad manager from gaining power. Think about it this way, once the bad manager sets the tone that anyone who disagrees with his horrible management style will get fired, who will correct his actions? A strong IT company needs to be built on allowing talent, innovation, and best practices to flourish. Allow bad managerial nodes will create a chilling effect, which will ultimate hamper your IT team, and ultimately your business.

As a CEO, debriefing, exit interviews, and what not, are the least you can do. As a board member or investor, I’d expect them to do at least this much. Even the highest level, there is such an assessment, so why wouldn’t you think that as a CEO, you can afford to simply take management’s word for things? Even auditors are brought into the picture from time to time. To improve, you must assess, progress without assessment is most likely just bull excrement.

www subdomain or no www subdomain

Monday, May 19th, 2014

This topic is a very old an ancient topic, but I’ve arrive definitively at whether or not the main domain should have www, or not. The answer is “it should”.

The reason being, is that a cookie set at the domain level, exists for all subdomains. If you have subdomains, or ever plan to have subdomains in the future, it’s best to use “www” subdomain for your main site. It’ll pay off by saving you some headaches down the line when you have specialized subdomains in the future (blog, beta, members, etc.)

Project Constraints and Project Selling

Tuesday, March 4th, 2014

There are 3 things you can control about a project, time, resources, and features. Of the three, you at best can control 2.

Which is why I propose for projects to have the following creation and definition flow:

  1. Feature gathering
  2. Resources / Budget constraints
  3. Time / Delivery constraints
  4. Project planning, project options, packaging, pricing
  5. Investigation
  6. Execution

 

  1. We want to under to undergo feature gathering first, because how can you size something with unknown dimensions?
  2. We want to know what are the budgetary constraints to the project, since we limited resources, we’ll have limited options, and we’ll have to live with the consequences of having limited resources.
  3. We want to know when the project needs to be delivered by, often times, if it’s something that needs to be rushed, then over time might be necessary, or perhaps more resources.
  4. This is where we plan the project and price out the project. I think it makes sense to allow the client to control 2 of the 3 factors of project planning. Once we know what the client is willing to give up, then we can go ahead and structure the deal around it. I’m sure with the resource allocation any venture can be profitable.
  5. During this phase we need to figure out exactly what is entailed with the project, and whether or not we can properly take on the project.
  6. During this phase we basically get it done.

This flow seems to make sense to me, if a person knows a better flow, let me know because this is the flow I’ll stick with for now.

Sell Reputation

Wednesday, February 26th, 2014

The Greek philosopher Aristotle divided the means of persuasion, appeals, into three categories–Ethos, Pathos, Logos. Today, we’ll talk about ethos.

When you’re trying to persuade a customer that your product is worth more than another person’s product, you will invoke one of the three. Substantial investment will be made mostly on the logos and ethos front. In modern day-terms, logos will be data, reports, forecasts and etc, whereas ethos will simply be reputation.

I believe perception and reputation is a good form of investment, because if you want to convince your customers that your product is worth more, you’ll have to employ one of those 3 methods. You can logically convince a customer, and they’ll provide you with logical prices.

The key is to focus on the illogical. Beauty is in the eye of the beholder, and your evaluation will be based on what they perceive. Sometimes, it can be lower, sometimes it can be accurate, and sometimes it can be higher. The fact that it can be higher gives you a great opportunity to capitalize on the differential.

You can sell logic, or you can sell reputation, and even empathy. If you had to choose one, I’d think reputation would have the most potential for irrational profits.

Virtual Test Case Definitions vs Real World Expectation Declarations

Wednesday, February 19th, 2014

In my space, there are a lot of parallels between the virtual space and the physical space. When I see a cup, I see a cup class. Then I think how it’s a unique cup, so it’s an extension of the cup class, perhaps, if we were to think about how cups are manufactured, then it’s a cup factory class, and it’s merely an object of that factory. So on.

Before a test is written, there is a clear understanding of what is expected to happen, and the test is considered incomplete until the test passes the assertion. It’s interesting because in the real world, I believe these are simply listing one’s expectations.

As a leader, I think it’s clear to communicate the things you TRULY care about in regards to your company. Just like in test driven development, you don’t care about the how things are defined, or constructed, but there’s a definite end result you care about. Your job as a leader is to communicate that end result. To illustrate this concept and the underlying reactions to such declarations, for example, lets say you said you care about the following things: Conversion rates, sales, traffic, site speed. The people who are responsible for those things might react as follows:

Conversion rates:

  • Examine existing conversion rates
  • Examine which pages convert the best, gather lessons
  • Monitor conversion rates across the site
  • Monitor overall conversion rate performance

Sales:

  • Examine existing sales
  • Examine which products sell the best, figure out why it sells the best
  • Examine current audience and existing markets
  • Devise a way to expand audience and market
  • Monitor sales and etc

Etc.

My point is, to do an effective job, clear definition of what needs to be done, will allow the people who are responsible for execution to determine how much resources are required, and if there are insufficient resources, which corners to cut. One of my mentors told me, in a project, you can control 2 of 3 things: Resources, Features, or Time, but never all three. Listing your expectation is the equivalent of listing the features. If you don’t want a person dedicated to these tasks, then that’s a resource constraint, and the execution team will respond with how much longer such constraints will delay the expected results. A leader doesn’t need to execute, but a leader needs to plan, communicate, assess, and make judgment calls based on assessments.

Why Segregate Production and Staging

Sunday, February 16th, 2014

I purposely didn’t specify server and environments. It’s EXTREMELY important that the staging environment isn’t sharing the same box as the production box.

A good staging box perfectly mirrors the production box, you’d imagine, that it’d be two birds with one stone, if you simply create a staging environment on the production machine. This is an extremely risky and bad idea.

The staging environment is for testing, all sorts of crazy things can happen, from installing hacked code, to code that are so inefficiently coded that the web server and the connecting database server’s load goes out of control. When this happens, sales will be lost, data won’t transmit, and there will be a definite cost to business.

So when weighing the convenience versus the cost, having a staging environment on a production environment is almost never worth the cost.