INTRO: Welcome to Apply Filters, the podcast all about WordPress development. Now here’s your hosts, Pippin Williamson and Brad Touesnard.

BRAD: Welcome to Episode 32. Today we’re going to be talking about what we’ve been up to lately: managing the code of the core plugin with add-ons, which can be a challenge, and laying out and planning the code for larger plugins and kind of the structure of a large plugin. Pippin, man, what have you been up to?

PIPPIN: Well, the last week or two have been a little bit busier than normal. We’re ramping up for a large EDD release, which is version 2.2. It’s a release that we’ve been working on since, well, 2.1, which went out in, I think, August, so it’s been about three or so months that we’ve been working on it. It was a pretty large update, not a whole lot of user facing changes. A lot of just background refinement still, few UI things, a couple of new features here and there, like minor stuff.

BRAD: What are some of the major plumbing? I guess I call it plumbing because it’s, like, the background.

PIPPIN: Yeah. One of the biggest ones that we did was we introduced a new product class called EDD Download. Previous, prior to 2.2, if you wanted to interact with a product as a developer, like let’s say in an extension or a theme, and you wanted to go get the sales of that product, the earnings of it, or you wanted to get all the price options or the download files attached to it, et cetera, there was just a whole mishmash of helper functions to use.

And so, we’ve now created a new class that makes all of that data much more accessible. It also makes it much easier to create new products as well. So since a product is a custom post type, you used to be able to just say, like, WP insert post. Set the post type to download. Set all the different stuff. And then update all of the post meta that you need.

BRAD: Right.

PIPPIN: Now you can do it much more simply and you can just say new EDD Download.

BRAD: Right, so you’ve created basically, like, an API that —

PIPPIN: Yep, to interface with the product.

BRAD: Right, and keeps kind of away from the lower level.

PIPPIN: Yep, exactly.

BRAD: Like WP insert post and all that stuff.

PIPPIN: And there’s a couple of big advantages to it. Number one is it just makes it a lot easier for someone who is new to the plugin or to the project to come onboard and figure out how to do something with a product. But, number two; it also makes it a lot easier for us to change things behind the scenes later on.

BRAD: Mm-hmm.

PIPPIN: If we want to change the database schema, if we want to do something like that, maybe we decide to move all of the data to a custom post type. We can do that much easier because we don’t have to worry about people manually calling get_post_meta on something.

BRAD: Right.

PIPPIN: Because instead they’re using our product helper class that we can adjust the methods there and everything will just work.

BRAD: Right, yeah.

PIPPIN: So it’s something I wish we had done a long time ago, but we just now got around to it. And there were a bunch of other little changes. We found a few security flaws that we patched up. We made a couple of small UI changes. Overall, it was a pretty solid release.

BRAD: Cool.

PIPPIN: There were no database schema changes. There was no data migration or anything like that, so it wasn’t as risky of a release. Put it that way.

BRAD: So that’s out, is it, 2.2?

PIPPIN: Yes, that is out. That went out on Tuesday.

BRAD: Cool. And is that–?

PIPPIN: Two thousand updates a day since release, so it’s gone out to quite a few people. We’ve had — I think two people had issues with it, maybe a few more than two. We had one issue that kind of bit us that we didn’t catch early on. We were beta testing for four weeks, and we’ve been running it on all of my live sites and a couple other live sites for a while.

But we implemented this new function, so EDD works primarily with ajax. Everything is ajax. When you add something to the cart, when you update the cart, when you do things like that, everything is ajaxed. And so one of the issues that anybody who has done a lot of ajax or JavaScript stuff on the front-end with plugins will know is that a lot of times you have sites that ajax just doesn’t work on, either because there’s a JavaScript error, they have a security rule in their HT access file, they have a security plugin. Something is blocking the admin ajax file or preventing it.

So we implemented a function that’s just called EDD_test_ajax_works, and if it detects that ajax doesn’t work, it displays a notice and tells them, “Hey, heads up. It looks like there’s an issue. Here’s how you can fix it, or you can dismiss this notice.” And anyway, for whatever reason, that function decided to trigger an error on, like, .0% of sites.

BRAD: Oh, the irony.

PIPPIN: And of course it’d be a fatal error.

BRAD: Yeah.

PIPPIN: Never happened to us anywhere. Never saw it anywhere until we pushed it out.

BRAD: Right. Eh, there are just too many variables. You can’t get them all, right, before you push it out.

PIPPIN: Yeah.

BRAD: Yeah. We’ve just recently released Amazon S3 and CloudFront plugin, the first release in over a year.

PIPPIN: That’s an update to the free one, right?

BRAD: Yes. Yeah. We’re just working on the free version right now.

PIPPIN: I remember you mentioning that you were thinking about making a pro version of it too, so this is just the update to the free one.

BRAD: Yeah, so the first step in doing the pro version is to get the free one in good shape, and we’re going to build on top of that, use that as a foundation.

PIPPIN: Very cool.

BRAD: Yeah, and the other part of that is I added a little opt in box so people can submit their email and so I can get an email list going of people that are actually interested in the pro version, so then we have that launch list to launch to. That’s important as well. All that stuff was in the new release that just went out. But like you were just saying — so that went out on, I think, Tuesday or, no, it’s been a week now since it’s been out. But we’ve done two point releases since then and one just today, so you can’t — we tested it as best as we could, but.

PIPPIN: Um, yeah. Something else that we did recently that was — actually, I’m kind of curious about your background update a little more.

BRAD: Okay. Sure. Yes.

PIPPIN: Tell me a little more about that.

BRAD: So one of the features of the Amazon S3 and CloudFront plugin that we added was, so one of the problems was that we weren’t storing the region of the file that was uploaded. So when you upload a file to S3, the bucket, whatever bucket you put into, that bucket has a region associated with it. And that determines what the URL is going to be. For example, if you’re in the standard region, then the URL is going to be S3.AmazonAWS.com, right?

PIPPIN: Right.

BRAD: That’s just your standard one. But if you’re in, like, the western region of the U.S. or whatever, if that’s where your bucket is located, then it’ll be, like, US-, or it will be S3-US-west-1.AmazonAWS.com, something like that, right? But we weren’t doing that or the plugin didn’t do that. Prior to this release, it would just do ‘S3.’ regardless, and that was causing problems for some people that were using buckets in other regions.

PIPPIN: Is that usually a per-bucket or per-file thing? I’ve used the S3 API before and, when I did it, we just gave the option to specify the region, but it would specify it globally.

BRAD: Yes, that’s true. Yes. So it’s associated with the bucket, right? But the problem with the plugin is that you can change your bucket. So if they changed to a bucket in another region, right?

PIPPIN: The other files don’t work anymore.

BRAD: Yeah, exactly.

PIPPIN: So your background updater is just going through and updating those?

BRAD: Yeah, the background updater just goes through all your current files very carefully in the background, and just gets the bucket region and stores that in the metadata for that file. So it does that for every file so that we have that information and we can construct that URL without having to ask S3 every time what’s the region for this file.

PIPPIN: Got it. How did you scale that?

BRAD: Yeah.

PIPPIN: Because obviously there’s going to be some people that are going to have hundreds of files, some with thousands, and some with hundreds of thousands.

BRAD: Yeah, so we just batched it. I think we chose 500, so it queries. It queries the database.

PIPPIN: Is it done, like, in a cron job, or do they have to sit there on that screen and watch the screen reload over and over again?

BRAD: When you upgrade and you go to the settings panel, you’ll see a little warning at the top that’ll say, “We’ve got this background updater running, and here’s what it does.” And that’s all it is. It runs on a cron, and it runs; it picks 500 files and looks at them and gets the S3 bucket for that file, gets the region, inserts that meta, and it just updates it. And it does that for 500 files at a time. And it’s got timeout protection as well, so if during that time it times out or, sorry, if during that batch it times out, it’ll fail gracefully, and it’ll just, the next time, it’ll pick up.

If it fails — I mean we put so much into this stupid, little thing. If it fails a certain amount of times, there’s a threshold. If it hits that, I think we chose 20 failures, like if it fails on 20 files, for example.

PIPPIN: Does it just skip them?

BRAD: No, it’ll stop. It’ll pause it, and it’ll — the message will change in the settings panel. It’ll say that it’s been paused because of this reason. And you can also pause and resume it. It’s pause and resumable. We kind of went crazy with it, to be honest.

PIPPIN: That’s awesome. But I think that’s a nice testament to how much thought you put into the plugin. And I think that’s a very good sign for the pro plugin to come for anybody who is using it.

BRAD: Yeah, that’s what I felt. I didn’t want to compromise on that, and there’s nothing more frustrating than something like that failing on somebody, I think.

PIPPIN: Nah, especially when we’re talking about a large-scale.

BRAD: Yes, exactly.

PIPPIN: If they’ve got three or four files, you know, big deal. But if you’ve got hundreds or thousands, that’s a little different story.

BRAD: Yeah, exactly. So, yeah, it’s funny. The issues that we’ve had to fix had nothing to do with the updater. So far, the updater seems to be working okay.

PIPPIN: Great!

BRAD: One of the issues, I’ll tell you. It’s just the stupidest things. S3 has this weird thing. If your bucket is in the EU, so it’ll use EU as the region, which we weren’t — we didn’t even know that that was possible, right? We thought that all the EU regions were, like, EU-west-1, you know, in that form.

PIPPIN: They’re not?

BRAD: In that format, no. There’s one of them that is EU, just EU. And, if you use it to construct the URL, it doesn’t work.

PIPPIN: That’s awesome.

BRAD: Yeah, it’s a really bizarre thing. We still don’t know the full reasoning for it. Like on the Amazon side, we don’t know the history behind it or anything like why it’s like that.

PIPPIN: I’d be curious to know if you find out though.

BRAD: Yeah, for sure. So, man, what about this public repo that you, or this repo that you made public?

PIPPIN: Sure. Earlier this morning, I decided to take one of my larger plugins, which is Restrict Content Pro–it’s my membership plugin–and I made the repository on GitHub public. So anyone can access it. Anyone can view the code base. Anyone can submit an issue or pull request. And it’s something I’ve been toying with for a while.

Some people may be aware of this, but when we did Affiliate WP back in April, we launched it on a public GitHub repo from day one kind of as an experiment. I’ve never played with a commercial plugin being on a public GitHub repo. I didn’t really know how it would go. Maybe we’d have lots of people downloading it for free and not paying for it. Maybe we’d get lots of support questions from people that hadn’t actually purchased a license.

And, in short, we didn’t see any of that. But what we did see is we actually saw real users who had paid for the plugin and other real developers submitting bug fixes, and submitting patches, and submitting suggestions. And it helped provide that same kind of like beautiful collaboration that we can see sometimes in open source on a commercial product, which doesn’t usually happen because usually everything is behind closed doors.

And so it was really cool to see that, and so today I decided to do the same thing again with my membership plugin, and so it is public as of today. Anyone can view it. You can find it from my GitHub page, and we’ll see how it goes. Restrict Content Pro is one of my older plugins. It’s the second or third really large plugin that I wrote. And so it’s got some cobwebs in it, and it’s got some things that are less than stellar, but it’s one that I’m really excited to kind of reinvigorate and push forward into a lot of cool new things with it. And, I think, making it public so that other people can either help or see or provide feedback is going to be a really important part of that.

BRAD: Interesting. With Affiliate WP, you’ve already been running this experiment for, it’s almost a year now, isn’t it, Affiliate WP?

PIPPIN: It’s about eight, nine months.

BRAD: Yeah, so that’s long enough. So you don’t get people coming to you and expecting support at all?

PIPPIN: We’ve had one person.

BRAD: Okay.

PIPPIN: And that one particular person even said, “Hey, I don’t have a license key. Could you help me?”

BRAD: Right, right, so they knew what they were doing.

PIPPIN: They knew what they were doing, and we just said, “No, I’m sorry. We can’t.”

BRAD: Right.

PIPPIN: You need a license key. And that’s it. I have no idea how many people have downloaded it for free from the repo and are using it. But to be honest, I don’t really care.

BRAD: Right.

PIPPIN: I don’t really care because it’s not worth it. Piracy is something that’s going to happen. And if you let it affect you, and you let it — and you try to actively fight it, it’s just going to slow you down. It’s going to hurt you more, and it’s going to hurt your actual customers more. So, look; if you’re going to pirate it, you’re going to pirate it.

BRAD: Yeah.

PIPPIN: I’m making it really easy for you. At least this way we don’t have a problem with pirates distributing bad copies that have malicious code in them.

BRAD: That’s a good point, yeah.

PIPPIN: And, to be honest, I think the benefit of it, the first pull request we got that fixed a bug or the first bug report we got that was important made all of it worthwhile because that would not have happened if it had been private.

BRAD: Hmm, interesting!

PIPPIN: Most likely.

BRAD: Very interesting.

PIPPIN: And I can tell you that, like, Affiliate WP has been pretty successful so far as a commercial plugin.

BRAD: Right. Do you think part of the reason is maybe that people just don’t know about the repo and they’re not really looking for it?

PIPPIN: Certainly. I think that’s probably a lot of it. I think there are a lot of people that are perfectly fine purchasing, but they would also probably download it if they saw it. But the thing about GitHub that’s actually really great for this is it really doesn’t cater very well to non-developers.

BRAD: Right.

PIPPIN: It doesn’t cater to people that are looking for a free download because, when someone lands on GitHub, they see a bunch of PHP files and source code and jargon like that.

BRAD: Hmm.

PIPPIN: Even though, yeah, there’s a down (indiscernible) bar, there’s nothing initiative about what that means to someone who is not a developer and who is not familiar with GitHub. And so, yeah, maybe people are looking for it, but they’re not usually finding it that way.

BRAD: Right.

PIPPIN: And it’s not like with the public GitHub repos, we’re not going on our main sales page that says, “Hey, you can go download it from GitHub.” No. We will happily tell anyone who asks if they can view the source code, and we’ll just send them straight there. If someone says, “Hey, I have a bug that I’d like to report,” we send them straight to GitHub.

BRAD: Right. Yeah, that makes total sense. It’s funny because I just searched. I just did a Google search for Affiliate WP, and your GitHub repo is the second hit. It’s right underneath your site. It’s not like it’s — it’s right out there in the open. You know, it’s not hard to find. That’s for sure.

PIPPIN: Nope.

BRAD: Huh. It’s an interesting experiment for sure.

PIPPIN: It’s been fun to kind of see how it goes, and so far I’m really happy with it.

BRAD: I think if — how many of your customers do you think are developers, like what percentage would be developers?

PIPPIN: Maybe 5%.

BRAD: Oh, really? Okay, so that’s probably why it works well.

PIPPIN: Really low.

BRAD: Do you think if you had a plugin, say, that was a developer tool, maybe one that migrated database or something like that, probably —

PIPPIN: It might be.

BRAD: Probably not, won’t work as well.

PIPPIN: I don’t know. I mean it’s definitely possible, but I don’t really think it would make much difference.

BRAD: Yeah? Really? Hmm.

PIPPIN: The reason being is simply that people that actually are interested in your product, we like to say that piracy is a major problem and all these things, and no one will pay for things. I think that’s a bunch of malarkey because, in reality, I think the vast majority of people are actually perfectly fine paying for something.

BRAD: Yeah. Hmm. Yeah, it’s interesting. It’s a scary experiment for me. [Laughter] I might do it in the future, but just not ready yet.

PIPPIN: Sure.

BRAD: I wonder. Yeah, yeah, it’s an interesting —

PIPPIN: I do know now that I’m not the — I know a couple other people that have now done this as well.

BRAD: Oh, yeah?

PIPPIN: Justin Sainton did it recently with the WP eCommerce plugin. And, as far as I know, he hasn’t had any fallback on it.

BRAD: Huh, interesting.

PIPPIN: Yeah.

BRAD: Cool.

PIPPIN: What else have you got? You were mentioning something in our pre-chat about doing something with your add-ons and with for WP Migrate DB Pro, something you changed within. What are you doing?

BRAD: Well, first of all, I’ll just give you a quick update on what we’re working on there. We’re working on the next release. One thing is migration state. Well, that’s what we’re calling it. The way it works now is that we’re just kind of passing data. The data that defines the migration, like the settings basically of the migration you’re running, we just pass them through, through each ajax request. It doesn’t just do one ajax request and then store the state on the server. We keep passing it back and forth through.

Anyway, we’re going to make that a little bit cleaner so we’re not — so we’re just passing around a state, like an ID that defines the data and store the data somewhere else. So we’re working on that. That actually sounds like a little thing, but it’s actually a lot of work with the plumbing. And we’ve completely rewritten the media files add-on.

PIPPIN: Why? What was the reasoning behind deciding to rewrite it?

BRAD: Timeouts. Servers that are configured with PHP-FPM do not respect the PHP set timeout function or set time limit function. And so what happens is sometimes you get a timeout if the request doesn’t finish in whatever the PHP-FPM setting is on your server, which it’s typically around 30 seconds, I think. So on any hosts that are using PHP-FPM, which is a lot of them right now, you’d get a timeout if you were pushing media files because when it was determining which media files to migrate, that process can take quite a while. So sometimes you’d get a timeout there.

PIPPIN: And there could be a lot of files to scan.

BRAD: Exactly. If you had — basically, if you have over 10,000 files to compare, it would timeout with that server configuration.

PIPPIN: Now was that when pushing the files to a server or pulling them from a server?

BRAD: I don’t think it mattered, actually.

PIPPIN: Okay.

BRAD: I think, as long as you —

PIPPIN: I guess it’s pretty much the same process on either end.

BRAD: Yeah, exactly. And then — so basically we had to rewrite everything though because of that, unfortunately. It sounds like you wouldn’t have to rewrite everything, but it’s just the way that the plugin was structured and the requests were set up, we had to just do it all over, so we did that. Well, we’re still working on that, actually.

Then CLI add-on, right now it’s a super simple command, and we’ve opened it up to have all of the migration options available so you can define your entire migration from the command line.

PIPPIN: That’s great.

BRAD: That’ll be the next release, and we’re pretty excited about that.

PIPPIN: I’m excited to see more and more people doing stuff with WP-CLI.

BRAD: Yeah, yeah.

PIPPIN: We actually just put EDD 2.2 included a new command in it, so in 2.1 we added basic CLI commands, which was, basically just allows you to view data, so you could list recent payments. You could list your earnings or your sales stats.

BRAD: Isn’t that neat?

PIPPIN: List your discount codes.

BRAD: Yeah.

PIPPIN: Oh, it’s super cool stuff.

BRAD: Yeah.

PIPPIN: But we want to go a little bit further, and so we’re going to add in abilities to — I mean, you can already create products because they’re just a post type, and discount codes and stuff like that, but we’re going to go further. But one of the things that we added this time, which because CLI is so much of a developer tool, we added the ability to create sample purchase data. So let’s say you’re building an extension and it has to do something — maybe it’s reporting, and one of the ways you want to test your reporting stuff, your features, is that it scales. Well, it’s pretty ridiculously time-consuming to go through and create 10,000 or 50,000 payment records. So we just added one CLI command that says EDD payments create 5,000, 10,000.

BRAD: Right.

PIPPIN: And so kind of like how the CLI has a command for generating tons of sample post data, it does the same thing.

BRAD: Right. That’s awesome.

PIPPIN: Super handy.

BRAD: Yeah, for sure.

PIPPIN: So I’m excited to see more and more plugins … that.

BRAD: Yeah, me too. I love it. I love the WP-CLI.

PIPPIN: And now that all my sites are in a position where they can actually use CLI, like live, I’m actually using it.

BRAD: I kind of wish that I used it more. I never really — like I haven’t really incorporated it into my workflow, so I don’t really use it on a daily basis right now. Do you?

PIPPIN: Not on a daily basis, but I would say on a weekly basis.

BRAD: Hmm.

PIPPIN: I use it quite a bit for unit tests.

BRAD: Right, yeah. Oh, yeah, yeah. I mean every time I run a unit test, our scripts do that, but I’m really running the script, which has CLI commands in it, so I don’t know.

PIPPIN: Sure.

BRAD: It’s cheating. Yeah, so one thing that we’re planning to do really soon is to move our add-ons. Right now our add-ons and our core plugin are all in separate Git repos, so what we’re planning to do is bring them all together into one repo. And I mentioned this today on Twitter and people started commenting and stuff and thought it would be cool to discuss it here.

I’ll tell you the main reason that we decided to do this or the main reason that it came up is because we were implementing Travis CI for our add-ons, so we were — you know when you’d push a commit for the media files add-on, for example, it would fire a Travis CI build. But for that to work, the add-on, I mean just running the add-on through Travis CI doesn’t make any sense because it requires the Core plugin, so you have to include Core. But also, the CLI add-on also has some dependencies with the media file. It’s like they’re all interconnected, right? So whenever one of them is — like, whenever you commit to one of them, really the build, the Travis CI build, should be all of them together, right?

PIPPIN: Makes sense.

BRAD: Because that commit can affect the whole thing. You could commit in the CLI add-on and it breaks the media files add-on, for example, right? So you really need to run them all together.

Then Ian brought up why don’t we just bring them all together into the one repo and then it makes it a lot easier. And then we started thinking about all the other benefits of that where, for example, oftentimes we’ll have an issue, and then we realize it’s a Core issue. But then realize, oh, it also affects the media files add-on. Also it affects the CLI add-on.

PIPPIN: Yeah, this happens to us in EDD all the time.

BRAD: So you have a GitHub issue sitting in your Core repo, and you actually have to have links to the —

PIPPIN: Yeah.

BRAD: Maybe you could create sister issues. I don’t know.

PIPPIN: We have an issue in EDD Core right now that I believe is open across two or three extension issue GitHubs as well because it affects all of them, but it’s really the same issue.

BRAD: Right, right.

PIPPIN: It’s kind of obnoxious.

BRAD: Yeah, it is a little messy, right? You create GitHub issues.

PIPPIN: Create separate issues in each repo.

BRAD: Each one and kind of link them together.

PIPPIN: Yeah. Usually we’ll just copy and paste links.

BRAD: Yeah.

PIPPIN: I mean GitHub is really nice about how it can link issues across repos together.

BRAD: Yes, absolutely.

PIPPIN: Which is cool.

BRAD: Yeah.

PIPPIN: At least there’s some sort of weird spider web connecting them.

BRAD: Yeah. I mean this isn’t, like, a huge thing for us because we only have a couple add-ons right now, and we’re not planning on having, like, 100 add-ons ever, right? We’re going to have maybe a half dozen at some point would probably be my guess.

PIPPIN: Sure.

BRAD: So I could understand why, like, it wouldn’t make sense for some other projects. Like EDD, it wouldn’t make sense to have all your add-ons in with your Core, I don’t think.

PIPPIN: I think we could say it would make sense to do, like, the top ten add-ons that tons of people use.

BRAD: Right, yeah. I could see that.

PIPPIN: But you then run into where do you draw the line.

BRAD: Yeah, yeah.

PIPPIN: They’re like, hey, this new add-on, I think it’s going to be super popular. Should we go and throw it in?

BRAD: [Laughter]

PIPPIN: It just gets messy.

BRAD: Yeah, yeah, yeah. And then, I mean, we already have — technically we have two plug-ins in one repo right now, right, because we use a build script to build the free and pro versions from the one repo. And so it’s not a big stretch for us to include the add-ons in there.

It was an interesting comment someone made on Twitter today that started this discussion too. I think it was someone had mentioned– Did you see this conversation?

PIPPIN: I think I did. I’d have to look back through my Twitter stream and see if I can find it again.

BRAD: Yeah. Someone mentioned that, like, plugins have to stop, just have to stop having add-ons for little, stupid features.

PIPPIN: Yes!

BRAD: That’s what it was.

PIPPIN: Yes.

BRAD: Right?

PIPPIN: Yeah.

BRAD: And I can agree with that on just solely on that one very specific statement. But as soon as you start to think about an e-commerce system, for example, it doesn’t really apply, right?

PIPPIN: Right. Well, one of the points that was brought up in that discussion that happened earlier this morning on Twitter is that, as an add-on gets more and more complex, when should it become an add-on as opposed to just built into the core plugin? And there’s a couple ways I think you can look at it. Number one, if it’s a — there are some features that just work fine in the core plugin just because of the fact that they’re not a big deal. Like, if somebody doesn’t use it, it’s okay; it just sits there.

But if a feature gets larger and larger, it might be really beneficial to split it off into an add-on, not just because you have more code sitting there, but because then, as the developer building it, you have a lot more control and ability to fix things related to that feature without pushing out an update all the time for the core software.

BRAD: Right.

PIPPIN: And so you can kind of maintain them separately.

BRAD: Right. Yeah, that’s a good point. I didn’t really think about versioning. I guess the more complex an add-on gets, the more you’re going to want to update it independently.

PIPPIN: Just as an example, software licensing for EDD is one of the larger extensions, and it gets updated about every single week or maybe once every two weeks.

BRAD: Right.

PIPPIN: EDD Core gets updated once a month or so.

BRAD: Right.

PIPPIN: And if the two were coupled together, it would be much more difficult to push out a small security fix or a small bug fix for one or the other because they would then be tied together.

BRAD: Right.

PIPPIN: The flip, the downside, and one of the points that he originally brought up on Twitter, and this is Clifton, who is one of the core developers Shop e-commerce plugin, one of the points that he brought up is that it’s more difficult and it’s a lesser user experience to have someone install five add-ons as opposed to install one plugin and have all of them, which I think is absolutely true.

BRAD: Mm-hmm.

PIPPIN: But I think the counterargument to that, which I think you need to really consider, is that installation happens once.

BRAD: Right.

PIPPIN: It might take a little longer to install, but, in the long run, it might be better.

BRAD: Yeah.

PIPPIN: But it varies for every add-on.

BRAD: Yeah. And, I mean, totally. I mean, well, just think about how many times people install my plugin versus how often they would install EDD, for example. Unless they’re building EDD sites and that’s all they do, they’re not going to be installing EDD every day, right?

PIPPIN: Right.

BRAD: But if they’re working on a bunch of different client sites and they’ve used Migrate DB Pro —

PIPPIN: They’re going to install WP Migrate DP Pro all the time.

BRAD: Right, so it makes sense for me to put a lot of effort into making that as smooth a process as possible. So, yeah, priorities, I guess.

PIPPIN: I think a takeaway is that just because you’re building add-ons doesn’t mean they should be separate plugins.

BRAD: Yeah.

PIPPIN: Every plugin is different.

BRAD: Yeah. Yeah, you really have to see.

PIPPIN: Don’t just make them add-ons because someone else said, “Oh, you should make add-on plugins because they’re better.”

BRAD: Yeah. Yeah.

PIPPIN: I think, before we get too much further and run out of time, I think I’d really like to talk about something that a listener submitted a few weeks ago.

BRAD: Yeah, sure. Do you want me read this?

PIPPIN: Yeah, why don’t you go ahead and read it out?

BRAD: Sure.

PIPPIN: This is something that came from Jonathan Perlman.

BRAD: Yeah.

PIPPIN: This was a question that he asked.

BRAD: Right, so Jonathan says, “I’m writing a plugin that customizes WooCommerce by adding checkout fields, add ad hoc reports from queries, and unique functionality for this specific site. How do you start structuring a big plugin from the get go so that it doesn’t have to get a major refactor later? Thanks.”

PIPPIN: Well, so first of all, I think that this was a good one for us to talk about because this is something that we’ve both done. We’ve both had major plugins that we’ve refactored over time, and we’ve both had major plugins that we’ve worked on planning from the get go to help prevent those kinds of refactoring. So first, I want to give you an example before we jump into anything.

BRAD: Sure.

PIPPIN: So EDD, for example, has been refactored, had major significant portions of it refactored quite a few times over the last two and a half to three years. The reason being is that, early on when it was first built, it wasn’t planned, like very little when the plugin was planned. It was kind of like, hey, let’s just dump this here. It works. Great. Let’s go with it.

BRAD: Mm-hmm.

PIPPIN: And so we put a whole bunch of stuff in post meta. We put a bunch of stuff in various places that it probably didn’t belong. And then, as they plugin grew, we realized that that wasn’t very good for it. We realized that wasn’t efficient, or we need to do something better. So we started creating a custom table here or there, or changing the way that data is stored. Then we had to write migration schemas and do all of those different things. That’s kind of a pain in the rear to do.

BRAD: Mm-hmm.

PIPPIN: And it’s also something that can very easily break if you’re not super careful with it. Then, on the flipside, after doing that, I had Affiliate WP, and that one I planned the entire database schema before I wrote any code. I wrote; I planned the whole thing out so that I could avoid that refactoring later on.

BRAD: How did you go about — did you sketch it out on a piece of paper or just write it out in a GitHub issue?

PIPPIN: I did it in a GitHub issue, and then I also — when I designed my database schemas or the API, I usually actually go in and just write like a shell API, so like I’ll write a class.

BRAD: Right.

PIPPIN: And I’ll write everything….

BRAD: Pseudo code, almost.

PIPPIN: I’ll write pseudo code that basically just — and that then tells me how I need to store.

BRAD: Hmm.

PIPPIN: It’s kind of like a diagram, but in code.

BRAD: Yeah.

PIPPIN: I think pseudo code is exceptionally helpful.

BRAD: Yep.

PIPPIN: It doesn’t work for everybody, but it works for me.

BRAD: We kind of — we have a similar process, but we do it all on GitHub because GitHub has pretty nice code, like the display of code is pretty nice on GitHub, so we usually do little snippets or little pseudo code snippets that kind of show. It’s easier than words sometimes, right?

PIPPIN: Definitely.

BRAD: Yeah. That’s how I go about structuring a plugin, usually, is I start putting, adding issues to a GitHub repo and then start tagging them with milestones.

PIPPIN: I’ll usually do feature. I’ll usually do, like, all of the features as issues.

BRAD: Yeah.

PIPPIN: And so like for Affiliate WP, for example, I had, like general reporting: here is what I want to have reporting features. Referrals database: here is how I want to lay out the referrals database. Here is how I want to lay out the affiliates. Affiliate registration, like how do people become an affiliate account, and kind of lay that out. And sometimes it would be a description. Sometimes it would be pseudo code. But I think, just like going through the process of laying all those out, even not necessarily in great detail, gets you to sit down and think about it on kind of a global scale of the project.

BRAD: Right. It’s funny because, like this question, “How do you start structuring a big plugin so that you don’t end up with a major refactor later?” I think you can get carried away here.

PIPPIN: Yeah.

BRAD: I think you can start, like, “Oh, I’m going to start with I’m going to build a framework.” And before long, you have a framework and nothing actually coded towards the actual end goal of functionality that you need for this thing, right?

PIPPIN: It’s a great example of maybe you should just do it.

BRAD: Yeah! Yeah, just do it. You can refine it later. It’s not the end of the world to refactor too. I mean what you just said about the database schema, I mean that’s a good idea to really put some thought into that when you’re doing it and make sure you’ve got it right. But, I mean, even then it’s not foolproof because I bet when — so one thing that you’ve said recently that you wish you had done is use custom tables for EDD, right, for certain things. Did you say that, or am I making this up?

PIPPIN: Yes, yes, definitely.

BRAD: Right, so if you could — I mean when you made the decision not to use custom tables, it was pretty much in the WordPress. Among WordPress developers at that time, it was probably pretty much what most people would have said, “Don’t use custom tables,” right?

PIPPIN: Yes. I remember myself saying that several times.

BRAD: Yeah, so things change, right?

PIPPIN: Absolutely.

BRAD: People’s ideas change.

PIPPIN: I think you just hit the nail on the head right there is that when you’re planning a large plugin, expect things to change.

BRAD: yeah.

PIPPIN: One of the things I’m really glad I did in Affiliate WP for planning it and I have also done in EDD now is I wrote an upgrade system because I knew that in the future I would have to change things.

BRAD: Yes.

PIPPIN: Maybe you have to add a column to a table. Maybe you have to change a data type on a table. Plan for that to happen. You can’t get your schema right every time. You can’t get your API right every time. But as long as you plan for it to change, you can make that a lot easier.

BRAD: Right. I would actually say don’t write an upgrade system until you need it. That would be my thing because we just added one to the Amazon plugin, and it wasn’t a big deal. We added it in. It didn’t take much work. The background updater thing, that took a lot of work.

PIPPIN: I think it depends on — so I’ll agree and disagree with you there.

BRAD: Okay.

PIPPIN: I think it depends on the plugin and the kind of data you’re storing.

BRAD: Mm-hmm.

PIPPIN: I’m use Affiliate WP as another example of this. If you are creating four different database tables, I think you need to do it from the get-go. If you are storing all of your stuff in a little bit of post meta for a bunch of different posts or attachments, then it’s probably not a big deal.

BRAD: I see, yeah.

PIPPIN: But in Affiliate WP’s case, our upgrade routine is the same as the installation routine, so I can take the same function call that creates the initial tables during install, and I can run them post install and they will process the upgrades as well.

BRAD: Right, right.

PIPPIN: For example, if I change a data type on a table, all I have to do is call the creation routine again and it will upgrade it.

BRAD: Oh, okay. I see.

PIPPIN: That’s because the upgrade and the install routine are the same thing.

BRAD: Right, right. Okay. I was thinking more like what you just mentioned about meta, metadata, like if you need to upgrade that.

PIPPIN: Yeah. No, I don’t think you need to worry about creating a big batch upgrade routine that’s going to be hundreds of thousands of rows of data.

BRAD: That you may or may never need in the future.

PIPPIN: Yeah. Hopefully you think about it enough so that you don’t usually need that.

BRAD: Yeah. I took a design course once, and I remember the instructor saying something about 3-D modeling, like when you’re doing 3-D modeling. Never model what you can’t see or what you’re not showing. Right? Why would you? Why would you ever model something that’s never going to be seen, right? That’s ridiculous.

PIPPIN: Sure.

BRAD: I feel like that kind of applies to programming as well, right? Don’t model something that you may use in the future, but you’re not sure if you’re going to ever use it, unless there’s a good reason to do so.

PIPPIN: Yeah.

BRAD: Like you were just saying with the upgrade routine. That makes sense.

PIPPIN: It might be wasted effort.

BRAD: Well, maybe. But as long as it’s not too much effort, right?

PIPPIN: Yeah.

BRAD: That’s the other thing. You have to balance against that. If it’s very little effort to protect yourself from the future, then definitely do it, right? But if you’re — yeah, if you’re going to spend three days writing a framework that may or may not be useful in the future, I have to question that.

PIPPIN: Yeah. I think custom tables are something I’d like to get into a lot further in a future episode.

BRAD: Mm-hmm.

PIPPIN: If anybody has any feedback on scenarios where they think it’s good or bad or any kind of feedback, let us know. It’d be good for a future episode, maybe next time, maybe the time after that.

BRAD: For sure.

PIPPIN: It’d be cool to see if we can find somebody who has done a bunch of work with and without custom tables and bring them on and see what they say.

BRAD: Yeah, yeah, definitely.

PIPPIN: Somebody beyond just the two of us.

BRAD: Yeah. I wonder — yeah, maybe we could find — yeah. If someone is listening and you’ve done a lot of work with custom tables.

PIPPIN: Hate or love.

BRAD: Hate or love, yeah, I would love to have you on to tell us about them because I don’t have a ton of experience with them.

PIPPIN: I had a little bit of experience with Restrict Content Pro because it’s had custom tables since day one. But then I didn’t really get into doing everything with tables until I got into Affiliate WP.

BRAD: Right. Oh, right, so there are custom tables in Affiliate WP.

PIPPIN: Oh, yeah. It has four custom tables.

BRAD: Right. You know, we should have somebody on from Rocket, Rocket Genius.

PIPPIN: Yeah, that’d be great.

BRAD: Gravity Forms has had tons of custom tables for, I don’t know, since the beginning of time, right?

PIPPIN: As far as I know, since day one of the plugin.

BRAD: Yeah. Yeah, so they would probably know.

PIPPIN: Carl is also very outspoken about don’t drop everything in post meta.

BRAD: Right, yeah.

PIPPIN: Yeah.

BRAD: Cool. Well, should we wrap it up?

PIPPIN: Yeah, let’s do it.

BRAD: All right. Well —

PIPPIN: Anything else you want to throw out there?

BRAD: No. Tomorrow we’re going to be doing our second Core Contributor Day, so we’re going to be working on — actually, Ian is probably starting work right now in New Zealand.

PIPPIN: That’s fantastic! Anything in particular that you guys are going to try to work on?

BRAD: Not really. We just try to pick up small bugs that we can try to finish within the day. I think I worked on, like, three or four tickets last month when I did it, and it was a lot of fun. I really enjoyed it. It was the first time I really spent a significant amount of time.

PIPPIN: Stepping out of your own projects and going to WordPress Core is super refreshing sometimes.

BRAD: Yeah, well, it’s — I mean the level of activity is pretty intense.

PIPPIN: Yeah, it’s huge.

BRAD: Yeah, but it’s just nice to work on something different and pay back, you know.

PIPPIN: Yeah, definitely.

BRAD: Contribute back to the community.

PIPPIN: I would throw out a small note again for our sponsors. The WP Ninjas are some pretty great guys. They wrote a couple of cool plugins, and they’ve been very generous in sponsoring Apply Filters that has allowed us to do some cool things.

BRAD: Awesome.

PIPPIN: Thanks, guys.

BRAD: Thanks, guys.

PIPPIN: Thanks for listening.

BRAD: Yeah. Thanks, everybody.

PIPPIN: Catch you next time.