IBM i DevOps TechTalk #21
Git misconception in the context of IBM i

Ray Bernardi – Welcome to IBM i DevOps Tech Talk, where we discuss key topics and questions with our ARCAD experts. I’m Ray Bernardi and I’ll be your host today. Today we’ll be speaking with Jeff Tickner, the CTO of North America, and Alan Ashley, who is a senior solutions analyst here at ARCAD Software. We’ll be discussing Git, the powerful version control system.
More specifically, we’ll be talking about Git misconceptions in the context of the IBM i. We’ll see if we can dispel some myths and shed some light on best practices. Let’s get right into it.

Alan Ashley – So, Jeff, as more and more companies are moving to use Git or a version of Git as their source control, what are some of the things that you’ve seen through implementations that are impacting the IBM i developer such as do they need to be a Git expert going into this?

Jeff tickner – So the customers that I implemented with Git, they’re generally being told that they have to use Git because the other teams are using Git. So they have on staff somebody that knows Git, not an RPG developer, but somebody else. And that’s the starting point, is somebody at the organization has to know something about Git to support the IBM i developers in the long run. Because Git can be complicated and eventually those developers will learn to use Git. When you’re using a package, which is the most common thing, there’s a wrapper around Git. You don’t necessarily have to know all of the Git commands. The command line commands to use Git, but there’s different ways to use it. So there is certainly some knowledge needed.
But normally you already have that expert at the company that understands branching and so on.

A.A – OK. So we’re taking advantage of the expertise already within the company to handle some of that branch aspects of things and the Git knowledge to help transfer that over to maybe the RPG developer may actually start to understand what’s going on.

J.F. – Yep. But ARCAD is the expert also ARCAD provides its own wrapper to interact with Git for the IBM i developers as well. So we try to simplify things, but somebody has to understand Git right.

A.A – And that comes down to some of the things around merge conflicts. When you start merging branches at that Git level.
You need somebody there to go. Which one do we pick. Particularly around concurrent development because that happens a lot on the IBM i where you have two developers maybe working on the same piece of source code, and they actually happen to hit the same several lines within the code.

J.F. – Yes, that’s an interesting point there. That is a really common misconception. If I adopt Git, I won’t have any issue with concurrent development. And the real answer is Git gives you better tools to manage concurrent development, but it’s not magic. And again we go back to, somebody has to understand how to resolve all merge conflicts.
But you have to have a resource that understands the language. So if I have a .Net developer that’s an expert in Git and I have a merge conflict in an RPG member, they’re not going to understand the best way to resolve that merge conflict. They’ll know the tools and the options. So we have to put those two experts together, the RPG guy and the Git guy.

R.B. – And that’s exactly what Git is doing here. It’s allowing those two teams to collaborate much better and more efficiently than before.

J.F. – Really good point. And eventually those this is my experience. The RPG developers are going to get better and better at it and they’ll use those other resources less and less, because they’ll get familiar with it. So but there is a learning curve. And you do have to be prepared for that.
Git isn’t magic. It doesn’t make concurrent development automatically. Simple. But you have the resources already. You have the RPG developer, you have the guy familiar with the mechanics of a merge conflict. You just got to put them together like Ray saying bring those guys together. That’s the intent of the common tools.

A.A – So we’ve got some collaboration going on between the teams, our IBM i guys are starting to understand Git a little bit now some of the customers that we’ve dealt with they have two developers small shop maybe a couple of hundred programs here and there, a couple of file descriptions. And then we have others that are monster that have hundreds of libraries and thousands of source members and things like that can handle things.

J.F. – So that was the interesting thing. When we started this, we thought it was going to be the small companies that had the flexibility to adopt source control, but it was the exact opposite. It was the big companies that were already using Git and saw the power of it that wanted their IBM i developers to use it. So one of our very early source control customers was the biggest IBM i developer in the world.
So scalability is an issue. We’ve spent a lot of time figuring out how to speed up the operations, and I’ve heard that other vendors have done similar kind of optimizations there. We have the most experience at this point. It’s pretty usable even when I have 50,000 members. We still keep it very responsive both on the interacting with a branch and the build.
Those are both problems. We spent a lot of effort on that, so if I start out of the box with Git, like, I install the Yum, I’m going to be right in the middle of that pain. I’m going to say, it took me a half an hour to clone my source. That’s the real world. And then if you use a package, you have options, like ARCAD, to speed that up.
So that might not be an incorrect misconception, but we’ve worked our way around it so that it does scale. And the other part of the scaling, the number of developers. So if you have an operation that takes a really long time, maybe it’s not bad if one guy does it, but if you have 15 guys doing it or 50 guys doing it now, it’s really annoying because they’re all waiting.
And so optimizing all those processes makes it easier for everybody to use. So right now with ARCAD we can say this solution does scale well even for hundreds of developers.

A.A – You brought up an interesting point there. And it’s kind of changing shifting gears so to speak. You brought up installing Git through Yum which kind of implies that you’re using open system activities within your IBM i or with on a windows machine or something to that effect. How secure is Git out of the box? Because that’s a big topic in the IBM my world.

J.F. – That’s a really good point. So really Git is about auditing because it can tell you who changed what line and this clever thing called blame. But they’re optimistic. Let everybody change everything and then we’ll figure it out. That’s literally what they say. Because Git really isn’t all that secure. So this again is the wrong Git, the core Git engine is not that secure.
And then you can apply security on top of it, just like the IBM i ,out of the box IBM public has star. But you can control it. Git is the same way if you go with a Git package like GitHub or Bitbucket or one of the many packages there, they make it easier to apply this security.
If you install RawGit then you have to script all that. There’s little they call them pre-commit hook scripts that can do it, but nobody wants to do that. You want to go to package just like if you try and use Git out of the box with your developers, you get exposed to the most complex use case, and you go with a package that provides a wrapper and makes it easier to use.

A.A – So ARCAD is also providing this wrapper. Correct?

J.F. – Oh, yes. But the interesting thing is one of the complaints I get from my customers is we have a copy of the repository, and IFS like everybody else, and we have to keep that right accessible for all the developers so we don’t trust it. So every time you interact with Git, we make sure that that copy on the IFS reflects what’s on the server, because everybody has to have right access to it, so it can’t be trusted.
And that adds a little bit of overhead. But we’re very efficient about it.

R.B. – Now we’re talking about security. The interaction with the actual Git repository is all controlled with like SSH keys and so on, is it not? It’s not just wide open.

J.F. – It’s wide open for the developers. So if you have 20 developers, any of those 20 developers can change anyone else’s branch. And they’re really not much of a concept of ownership of branch. That’s what we mean by wide open. And Git says it’s okay because I can look at that line, so I can say, hey, you’re working on this code.
And I can see four different developers have changed that line in the code. So that’s the power of Git. I can see successive changes to a line of code and say, oh, Alan changed it, then Ray changed it, then Jeff changed it. That’s the power of Git. It’s not controlling who can change it, but telling you who changed it.

A.A – And so then ARCAD kind of lays over that and allows, because when you go through the checkout process and you get that one member that you’re going to edit, or if you don’t have the rights within ARCAD, it’s not going to let you edit it.

J.F. – Yes. So that’s correct.

A.A – That’s where our security starts to come in and our separation of duties and so forth.

J.F. – Yes, but I can go out to the Git server and change it there if I want. If I haven’t applied controls there, it’s not much different which comes in.

R.B. – But it’s not much different than what IBM i developers are doing now. If they have a common development area, typically they’re all in there together?

J.F. – Yes. So our customers that have a common development library, they all have right access. And you don’t know who changed it last? IBM’s not tracking, that a line was changed by this person just, Oh, the source date changed.

R.B. – So basically, we’re talking about the same thing. Only now you have an audit trail.

J.F. – Yes. Now you have an audit trail.

R.B. – So that sounds like a good thing to me.

J.F. – Yes. And the Git packages can provide more security on top of that. So they can force the developers into the process, so that they have to go through peer review and so on.

R.B. – I mean, you can even do things like encrypting files before you save them to the get repository. If you’re really concerned about something, can’t you?

J.F. – Yes. You could. The Git packages can provide for that. You can provide lots of controls over things. That’s the intent of those. And the caution is if somebody decided they wanted to try out Git and they just install Git for Yum, they don’t see any of this. They see just RawGit.
So they’re like, I’ll try this out. It’s going to be wide open, out of the box. Everybody can do whatever they want. And all we can see is, oh I see this person changed it. So it’s kind of a cautionary tale. Sometimes our customers will say, hey, I install Git and I’m playing around with it.
And I say, well, that’s Git, but that’s not how we’re going to use it, because we know better.

A.A – So, we’ve just covered a few of the concerns that an IBM i person might have on this. And the list really goes on and on and on, because there’s always some kind of issue that comes up. It’s like, oh, can it do this? Can it do or so forth. And we’re just kind of saying, hey, you know, it’s not as big and scary as it may seem.
Yes, there will be some issues. And one of the things you had said was if you’re going to get to fix your workflow, but yet you still use the same workflow, Git is not going to solve your problem. I think that’s right.

J.F. – You said it. So concur a little bit on that. People think it’s magic for concurrent development and it just gives you better tools. The same with your workflow. If you have pain from your workflow and you think going to Git is going to remove that pain without adapting your workflow to leverage Git, you’re going to have the same pain, just have better visibility.

A.A – At least then you can see where your pain points are. Yes, it’s a little bit better.

R.B. – There’s also something else here though. I mean going to Git allows you to integrate with CI CD pipelines a whole lot better.
You possibly can now.

J.F. – Yes. Some of our customers go to Git because they’re using a whole suite of tools, to manage things and putting their source in Git exposes it to that set of tools for all kinds of things security scanning, all kinds of things. DevSecOps So that’s one of the reasons people want to go to Git is because the Git repository has so many integrations, it’s amazing.
So you get a lot of choices then going to Git.

R.B. – Let’s stop there because we could talk about this for hours. We really could. In the end, what it boils down to is Git will improve productivity. It will improve collaboration, and it definitely improves your source code management. ARCAD software has been working with Git now for over a decade, and we understand the issues and we understand how Git can revolutionize IBM i development.