Generating a Virtual Image... is that even possible?!?

ERPNext
1

We are currently having several custom modifications made to our ERPnext implementation on our local test server. Once we have it the way we want it, is there a way to then generate a Virtual Image from the local server with all of our modifications that we could then host on Google Cloud or some other cloud service?

Is there any documentation on how the current ERPnext virtual images are created? Maybe this would help us identify a path to our own image.

Virtual images are certainly easier to host, so it only makes sense for us to try to generate our own once we have completed our modifications.

BKM

2

To convert a physical server to virtual is possible but the results may not be entirely successful - preferably you want to start with VM in the first place.

3

@clarkej

I do not think that it is possible to start from a VM image since I am having several of the modules modified for my specific business use. Not sure how I would ever integrate modifications into the starting VM image.

Is it possible to start from a VM image and still modify the system?

I have several projects going on right now to modify the new POS module for better visibility and faster performance, the BOM module for my specific business needs, and some accounting changes in order to support a Sage50 interface. I have no idea how I would then get these special changes into a VM image?!?

BKM

4

So does that mean I am pretty much stuck with conventional hosting in order to keep my changes?

BKM

5

“I do not think that it is possible to start from a VM image since I am having several of the modules modified for my specific business use. Not sure how I would ever integrate modifications into the starting VM image.”

Well the question is are you sure and know for a fact that your ‘special changes’ POS module requirements etc are not supported, from valid test result VM experience?

Otherwise your case neatly illustrates the main reason and basis to use a VM in the first place: Rather than start with a physical machine then install the OS, ERPnext etc etc as you have done, instead start with the OS image running on the VM, on which you then install and modify all that you require. All the while these are contained in portable VM image.

6

I have not used a VM image up to this point. I only installed directly onto a bare metal Debian8 server. However, all of the changes I have in process are in the form of either Pull Requests on GitHub or complete customized apps to replace ERPnext modules.

So, if you know a way for me to get those kind of changes into a VM image when they are done, then you have my attention. Really that is kinda why I asked about VM images at the start. I would ultimately like to be hosting a VM image on a cloud server when we go live, but I MUST have all of the changes my development teams are working on included in the final implementation.

The stuff that goes through the githib pull requests and submissions will certainly be included in the next revision releases and therefore easily added, but the purely custom stuff that I need will be on my system only. That is why I wanted a way to generate a VM image when I was done.

However, to your point, if there is a way to make all of my very specialized changes integrate with a standard VM image, then I am all ears!!!

I just need someone with the proper experience to chime in here and let me know if either route is possible and maybe point me to a path that leads to my VM image goal.

BKM

7

Indeed yes both routes are possible.

However at this point you are on the non-standard route, where experience reports are p2v is not assured of a working VM image result.

Two ideas for you to seriously consider, to make an informed choice:

  1. Run p2v from now on during development, to test and confirm you get a satisfactory image. You will avoid disappointment.

  2. Confirm whether the standard route supports your ‘very special changes’ cases; IMO you must have valid reason to reject this route.

8

Hmm… Okay. You have just introduced a term I have never seen up to this point… p2v
What is p2v and how does it help me? Is it the script for generating a virtual image? Is there a brief reference guide for it that I could attempt to use?

Also, if there is a way to inject my code changes into an existing VM image, where would I find information on how to go about this process? Is it different from the process I have to use on my local box server?

Any pointers toward reading information on these 2 topics would save considerable time as opposed to the trial and error method of doing this. Mostly because the Error side of that equation usually means that I have to strip the server and start over again.:stuck_out_tongue_closed_eyes:

BKM

9

Maybe I am asking the wrong question to start.

How about a link to some information on how to install custom apps and customized modules on a running VM image? Then I can review the information to see if it would work for my application.

If it is possible to install my developers code changes to a running VM image, and then save the modified image for re-use, then I might be better off going that route. The portable image would make it easier to host the finished system on a commercial cloud server.

I came from a background of always having to hand load my servers on GoDaddy or HostGator and making sure all the operating system components were in place, etc. Loading a VM on something like Google Cloud Service is very new to me, but sounds like it might be the best approach. I have no trouble setting up a local VM host to try this method. I just need to know where to start and what path to take to my desired result for ERPNext.

Any pointers?

BKM

10

ah yes p2v refers to ‘physical to virtual’, ie the convert tools in my first response to you.

If by ‘inject’ you mean a ‘change script’ that captures the details of what you have manually added and configured to your physical image up to now? That record typically is something you would need to document and maintain. I am not aware of script generators or utilities for this.

I have no reading list - of course all the first-tier providers like IBM, Red Hat, VirtualBox et al have superb documents and tutorials - Google is your friend here…

11

“Any pointers?”

Try Google - you need to get out more :wink:

12

Yeah… not funny.:stuck_out_tongue_closed_eyes:

I have been through the “Google is your Friend” route and it comes up with very different information that has little to no value to my issues. My concerns are not about dealing with VM images in general. My concerns are more to do with modifying VM images of ERPNext installations specifically and how that works with (or breaks) the frappe framework. So reading from IBM, or Red Hat really has little to do with the specifics of how an ERPNext VM image responds to changes. I know how VM images work from an OS perspective. I want to know how to tweak ERPNext images specifically to avoid trouble.

That is why I come here… to this forum to find more specific answers that would hopefully keep ERPNext from crashing around my ears when I make changes.

Thanks but I don’t need an education how how VM images work in general. I got that already. Google searches return many, many, many old YouTube videos on how to work with ERPNext from 3 or 4 versions ago. and pretty much all crawled data links are fairly old and usually filled with incorrect information specific to ERPNext. Those links have been the frustrating part of my ERPNext education to this point. They have lead me down many a path to failure, stripping the server, and starting over.

So, for up to date information, I come here. Your referral to “Google is Your Friend” is the same mantra that so many elitists use here that send many a devoted user down a trail of deceit. If you don’t have anything specific to help with an issue, then why do you bother to respond. Or do you just like to see yourself in print?!? I mean, I really do not get it.

I know that ERPNext and frappe in particular have some quirks that need special attention. God knows I have tripped over more of them than I would wish on any new user. If Google were the end all, be all of accurate information, then why are there so many forums like this one that thrive on accurate, relevant information?

Those of us looking for information don’t need platitudes from elitists. We need relevant information on specifics. If you don’t have any idea where to get the relevant information then why bother responding? It wastes bandwidth, just like senseless and misdirected Google Searches.

So, I can see many individual posts in this very thread and the only relevant piece of information that came of it was for me to look into how to use p2v. You have created here a senseless string of useless information here that will take the next poor soul looking for information a long time to read through and find that there is nothing relevant to help them.

Google is not their friend, and it appears that you are not either. So here I am still with very little information and a bunch of useless reading. Thanks for not much.

If anyone else has relevant input (or even better questions to narrow my focus), I am still listening.

Thank you.

BKM

13

“You have created here a senseless string of useless information here that will take the next poor soul looking for information a long time to read through and find that there is
nothing relevant to help them.”

Ouch!

“Or do you just like to see yourself in print?!? I mean, I really do not get it.”

Indeed.

I agree you need more help than I am prepared to provide

14

Hi, not reading the whole discussion but this link should allow you to migrate your on premise server onto Computer Engine.

1 Like
15

Thanks for your excellent link and the timely bailout aidr3805.

My caution above that p2v may fail to work - is now rare I would think.

On a separate note my cheery daughter - who works in IT support at our local uni - showed me this refreshing tool where they refer the odd user.

Let Me Google That for You is a hoot http://lmgtfy.com/?q=how+do+i+feed+a+baby

16

@bkm best practice for the scenario you’re describing is to keep as much of the development as possible in a custom app.

If you do that, you can have a development environment where you develop, and then simply install/update the app in the production environment whenever needed. The developer videos are a starting point for how to do that.

If you use this practice, you will save yourself a LOT of headaches down the road with updates/upgrades.

Otherwise, if you must “move from dev to prod” then consider using “containers” - there’s quite a few discussion on using LXD containers on this forum, but as far as I know, there is no definitive guide on it, so it would have to be researched.

While physical to virtual conversion these days is better than it was in the past, there are still obtuse problems which pop up out of nowhere. Its generally not recommended unless there’s no other alternative.

1 Like
17

@iadr3805 Thanks for the link. I am setting up a test account there now to see how well the transition works. If this works, then I may have the problem solved. I was actually hoping (at least on this forum) to shake out the procedure the ERPNext devs use to generate their VM image that they typically setup for download. However, based on a few other threads lately, they may be having problems of their own. They have not updated the one on the website to a Version 8 image as of last night.I haven’t checked yet this morning. I had hoped if I could get the developers to share how they make their images, I could save myself a great deal of time tripping over those things that tend to get in the way of every specialized image. Your answer may work out in place of theirs.

@felix I really appreciate the advice. It made me think about my path forward again. Up to this point I have been trying to make all of my modifications based on GitHub Pull Requests so that the code changes could be put back into the community. However, with a few exceptions like yourself and @iadr3805 my search for answers to keep my development teams on path have been met with snarky remarks and and questions about why I would even want to make such changes. It has been tough not to be discouraged by them. So maybe your line of thinking is the better approach. Just create the new apps I need, plug them into the framework and let the community find a more receptive voice before I start donating back again. I currently have about $15k USD already spent on development teams trying to make things that can be donated back and another $22k USD spent on related projects that do not fit the ERPNext framework but are still needed for my clients. If I weren’t so far along on some of these modules, I might change the development focus toward new custom apps. Regardless, thank you for the reminder that there may be a less tedious development path for the rest of my projects rather than wasting expensive local developer time on trying to be a good community citizen.

BKM

18

Yes, to answer your question on “is there documentation on how current images are created” the direct answer is no.

There is no build system currently in place to create an ERPNext VM. (Someone can please correct me if this is wrong). I believe that the VM on the website is manually created. Someone does a base install of Ubuntu and then runs the easy install script (which you have used previously based on your previous threads). Then the VM is exported and hosted on the website for others to download.

These days, I myself no longer use the easy install method because there’s usually some problem unfortunately. I follow the steps listed at Guide: Manual Install ERPNext on Ubuntu 16.xx & Debian v8 & 9 and have found them to work well.

As for using custom apps, I hope I have not discouraged you from contributing. Based on the scenario you described, your 22k investment sounds like a perfect scenario for an app, while the other investments can hopefully be contributed back.

Overall, in terms of infrastructure and workflow, I think you’re approaching things the wrong way.

Keep separate development and production VMs, I have found that having two VMs work well. I just have to coordinate to ensure that they are in sync (i.e. run bench update at the same time for both, after proper testing in a sandbox and taking backups of course!). Even after you go-live, keep two VMs because one can be a staging/testing environment for updates and training. You also want your production system to run supervisor, nginx, and be in production mode, whereas you want your development machine to most likely be in development mode for debugging.

I highly recommend you select a cloud infrastructure provider like Google Compute or AWS, setup a new ERPNext instance, and then do the production configuration from scratch. It’s a one-time pain, but hopefully will prevent problems from happening down the line.

19

Hmm… that is partially true. I currently have a production version running on the local Debian 8 server and 3 different developer versions running with each of the dev teams currently working on projects. My timeline for a completed system required that I spread the work out to multiple teams and run them in parallel. I am even trying to get another team started on the barcode label printing. Hopefully they can come to a solution and we can get that one started as well. That would make a 4th developer instance.

Once the packages begin to enter the completion stage, I will have to work through the integration steps individually for each module and hopefully have very little additional issues. The first 3 dev systems and the production server were all started from the same bench point. When they are ready, I hope we can all bench update at once, sort out any issues and then combine the results.

If I can get the production system converted to a VM image, then all of the current 4 systems will be on equal terms.

The external apps that I have in process are very client specific and didn’t make sense to put them into the community because there is likely no other user that would ever want those changes. The POS and BOM changes though were fairly universal so they are still on track for community, but I am not sure I will do it that way again for future additions.

I used to do a great deal with the TinyCore Linux community because many of my other products are based on embedded linux for their OS. They have a more matured community mindset and the devs are very willing to lend a hand with technical answers when someone is trying to add to the ecosystem. I didn’t mind spending money to improve that package because there was always help when you needed it. Hopefully the community here catches up to their own product soon. When they do, you will see and amazing seismic shift in the world of business ERP systems. Until then, it takes great patience to not be frustrated.

BTW… Thank you for the link to the thread on manual installs. When I first started with ERPNext in Feb. the instructions for that were sparse and didn’t work. I spent a great deal of time figuring out how to get the install.py script to do what I wanted and never really checked back on the manual install. I think I will commit that thread to print and keep it handy for my next server install.

BKM