Code Consolidation

I found some time to do some consolidation work across my plugins and I both wanted to share what's been working for me, but also to get ideas for how to take it further. When I first got started, I developed each of my plugins in somewhat of a bubble and took bits and pieces from my other plugins when starting something new. This not only led to lots of opportunities to consolidate, but also some inconsistencies and if -- over time -- I found a better way to do something, I had to hunt down all of the places where that thing was used. Come up with a better format for a config setting? Have to change them all one by one. Not anymore.

What I have so far:

• Riffing on Rogue's RPFramework, I've started to develop a separate module that contains methods with blocks of common code. Mine is in no way comparable, but RP's approach is a great way to get ideas (that is to say, steal them).
• Using the Indigo API <Template> facility, I've developed XML templates to simplify pluginConfig files for common settings (like plugin update notification and debug settings). I can see these being useful for other things too (like MenuItems).
• I create one instance of each item in a central folder in my development environment and then create hard links in each of the appropriate plugins' source files. Update one, update all.
• When working on any plugin, if I find a bug or enhancement, I can include it then and there and it will "automatically" propagate to my other plugins that use that framework element. It is important to note that you must make the changes to the original version of the file. It seems that if you make a change from one of the links, the change will only be made there and the link will be broken.

I can see a lot more going into this over time. It's been a bit of a lift to refactor and test this approach and fold it into each plugin, but in the end I think the effort will more than worth it. I'm happy to share details of what I've learned along the way. For example, remembering to add the new file to Git (does wonders for silencing tracebacks).

I do realize that to most of the developers here this is like me saying that I just found my elbow. But any hints or tips that others use to help me find my other elbow would be most appreciated. Maybe others could benefit from the discussion too.

Code: Select all

<?xml version="1.0"?>

<PluginConfig>

   <Field id="infoLabel0" type="label">
      <Label>Some Plugin</Label>
   </Field>

   <Field id="pluginRefresh" type="textfield" defaultValue="15" tooltip="Enter the number of seconds between refreshes.">
      <Label>Refresh Frequency</Label>
   </Field>

    <!-- Notifications Template -->
    <Template file="template_notifications.xml" />

    <!-- Debugging Template -->
    <Template file="template_debugging.xml" />

</PluginConfig>


I would suggest putting all the common code and templates in a subdirectory of your plugin's main folder, and using a relative path in the Template file statement. That way the common code is a single drop in to the main plugin directory.

I admit, I haven't done anything like this yet because I haven't figured out how to handle this in Git. I would want the common code to be it's own Git project that gets included by multiple individual plugin projects. Otherwise there's no good way to update them all. I'm sure there's a way to get Git to do that, I just don't know what it is.

Otherwise there's no good way to update them all.

I don't know that it's the best way, but this seems to be working really well for me.

• In my development folders I have a separate folder for my common elements:
  
• I created hard links for each of the shared elements into each plugin package as needed.
  
• I had already created symbolic links for each of my plugin packages into my disabled Indigo plugins folder.

I can change any single shared element directly or from any plugin package from within PyCharm. Changes made in one place show up everyplace. The hard link means the shared element shows up like any other file (because the hard link is a pointer to the same thing--there's ever only one). The key was making sure to add the thing to Version Control (right-click on the file from within PyCharm and add it to the Git). Then it commits like any other file. Jay was the one that turned me on to using hard links.


For the future: In terminal, type the following (you can also drag from finder into terminal which is my preferred approach):

Code: Select all

ln [path to file] [path to plugin package location]


for a symbolic link, add -s to the ln before adding the elements.

I actually didn't know of the template approach -- I had previously intercepted the call from the plugin and injected my own XML (such as the standard debugging items), but the Template is far cleaner and better. I'll have to add that in next time I touch the individual plugins. Thanks for sharing!

There probably is a fancy Git method for this but I couldn't find it... I settled on a Git repository for my framework and it ends up getting copied into each individual plugin's repository in a subfolder. Not the fanciest, but a simple script to push changes (e.g. copy) to the plugin sub directories on my dev machine works fine.

I actually didn't know of the template approach -- I had previously intercepted the call from the plugin and injected my own XML (such as the standard debugging items), but the Template is far cleaner and better. I'll have to add that in next time I touch the individual plugins. Thanks for sharing!

It's nice to be able to give you a little something for all the help that you've given me over the years.

A couple of updates as I continue to learn about the approach I'm using.

• I made an edit above to stress that making changes to the hard-linked files must be done to the original file. If I change the linked version, the changes only appear there and the link seems to break as a result.
• I wish I had listened to @ FlyingDiver's advice to use a subfolder. I'm doing that now, too.

[jay (support)]

I made an edit above to stress that making changes to the hard-linked files must be done to the original file. If I change the linked version, the changes only appear there and the link seems to break as a result.

Odd - I'm pretty sure that's not supposed to happen. Editing using PyCharm?

Now that you’ve said that, I went back and did another test with BBEdit. Of course, this time it worked like you and I would expect. I’ll do some more testing with PyCharm and report back.

The reason I’m using hard links instead of symlinks is because PyCharm will upload the link to Git rather than the linked file.


Sent from my iPhone using Tapatalk

I’ll do some more testing with PyCharm and report back.

It looks like PyCharm does something to the file that causes the hard link to break. When I do the same thing with BBEdit, things work as expected. I tried making a change "in the other direction" using PyCharm (editing the original source file instead of the hard-linked version), and things still broke. SO, I would say that when using hard-linked files with plugins managed with PyCharm, edit the file with a tool other than PyCharm.

I'm going to go and raise a bug report to see if I can learn anything.

[jay (support)]

I’ll do some more testing with PyCharm and report back.

It looks like PyCharm does something to the file that causes the hard link to break. When I do the same thing with BBEdit, things work as expected. I tried making a change "in the other direction" using PyCharm (editing the original source file instead of the hard-linked version), and things still broke. SO, I would say that when using hard-linked files with plugins managed with PyCharm, edit the file with a tool other than PyCharm.

I'm going to go and raise a bug report to see if I can learn anything.

Definitely, that certainly appears to be a bug in PyCharm. Let me know what it is and I'll +1 it in their system. I think the more people that chime in on a bug the more likely it is to get fixed. There was a bulk refactor bug that languished until more people started adding +1 posts.

Well, that was fast. DIdn't need to submit a bug because it's not a bug.

If this check box is selected, a changed file is first saved in a temporary file. If the save operation succeeds, the file being saved is replaced with the saved file. (Technically, the original file is deleted and the temporary file is renamed.)

Also, the ownership of such file changes.

If this check box is not selected, the ownership of a file does not change, but all the advantages of safe write will be lost.

Makes perfect sense why links would break. So the trade off is leave "safe write" enabled and use another tool, or disable "safe write" and accept the above risk.

I’ll do some more testing with PyCharm and report back.

It looks like PyCharm does something to the file that causes the hard link to break. When I do the same thing with BBEdit, things work as expected. I tried making a change "in the other direction" using PyCharm (editing the original source file instead of the hard-linked version), and things still broke. SO, I would say that when using hard-linked files with plugins managed with PyCharm, edit the file with a tool other than PyCharm.

I'm going to go and raise a bug report to see if I can learn anything.

It's almost certainly saving the file to a new (temp) file, then doing a rename. That's the safe way to replace a file. But it breaks the hard-link.

OK, I was a minute late with my post.

Makes perfect sense why links would break. So the trade off is leave "safe write" enabled and use another tool, or disable "safe write" and accept the above risk.

And if the other tool doesn't do the equivalent, then it's not a safe write either. TANSTAAFL

And if the other tool doesn't do the equivalent, then it's not a safe write either. TANSTAAFL

A fair point that.