Visual Studio and Umbraco as a Service
In working with Umbraco as a Service (UaaS), we often get askeded the question is how to work with it using Visual Studio (VS). The answer up until now has been a rather manual process. I always get a little annoyed when advising people to do something manually when I know it can be automated. Automation is difficult to get right though, so I've been hacking away at a little script for the longest time but was never happy with it, it always seemed to be a little "half way there but not really". However, a few weeks ago I had a breakthrough that gets close to what I personally like to experience to start working with VS. TL;DR; - if you don't care about the background story, just scroll down to the end of this post to see a short video and a download link for the setup script.
Background
First let me start with a note on how UaaS currently works. When you create a new project you get 2 websites: a development environment and a live environment. The development environment has a git clone URL so you can get the website down to your machine and start building locally.
Once you're happy with the results you push the "end result" back to the development site, check that it's all working as expected and then you can push your changes to the live website.
The reason I said "end result" is because this is what your cloned website will basically look like locally:
You will note that there's a distinct lack of Visual Studio files, nor is there any code folders in there. In the UaaS documentation we refer to this as our "deployment repository".
When we started building Umbraco as a Service we of course had lofty goals of supporting all types of projects that could "just" be pushed to UaaS and we would take care of the rest. The option above is ONE of the possible options we worked on and currently the only one. It's the most simple one: give us an "end result" website and we'll make sure it works on UaaS.
Of course when you have lofty goals, at some point reality will catch up on you.. which it did for us. After all kinds of experiments with VS solutions in the git repository and Grunt/Gulp/Bower/Npm/etc. we realized that this would take significant extra effort to make work well. There were many other aspects around the automation experience that we would have to focus on as well, for example:
- The baseline feature (which complicates things if you start throwing in VS solutions, and javascript package managers)
- Server resources; if UaaS starts being a build server, we need many more resources to be able to do this acceptably fast
- Reliability; we need to make sure that all kinds of build setups work and run, we need to install pre-requisite software for that
- Speaking of reliability: eventually builds will fail, so we'll need to build ways to debug, try again, clean up etc.
We're a small team and don't have infinite resource to do all of this right now. We very much wish to still offer more advanced deployments but we have to choose to keep it simple for now. I think we currently have a happy medium on top of which we can build over time.
So, for now, we have a requirement that says: if you're a Visual Studio gal or guy, we ask you kindly to build things locally and drop it into the UaaS repository all pre-built. You can verify that it works on your local machine, at which point you can also be confident it works on UaaS once you push those changes up. This is what I mean when I say you push the "end result" up to UaaS (and why it's referred to as a "deploy repository" in our documentation).
This does mean that you're responsible for setting up your own VS environment and you will need to put your custom code in a different source control repository from the one on UaaS. The disadvantage is obvious, you're still going to be responsible for a small part of the infrastructure. Luckily it's easy to outsource that part to Github/Bitbucket/etc these days.
There is also a big advantage of this separation: it will force you to make your code more or less modular which means that you could end up dropping the same code into a different site and it will mostly "just" work. It might just teach you to blur the lines a little less and think more of separation of concerns.
Automating the setup
Okay, enough talk about how we got here, let's take away my one big annoyance: having to think about how to set up my VS solution. For the automation of this I've thought about how I would want to work: I want to have my website, it should automatically run from VS when I hit F5 and to make it simple, I want it to be a (VS) Website project. The reason to choose a Website project over a Web Application project is to make it reflect exactly what we're looking at, a web site in which we drop custom assets, like a dll with some code we want to add to the site.
So, not:
But:
Speaking of custom code, apart from the Website project I also want to have a (VS) Class library to write my code in that I can build and drop into the website's bin folder. This build and copy should happen automatically once I hit the Play button (debug button) in VS so that I can start putting in breakpoints and debug my code.
Here's where the biggest problem always was for me: I wanted to reference the UmbracoCms.Core libraries from the class library but I could never find a way to set up all the references in a reliable way. NuGet doesn't have a way to install the libraries from the command line either. Not having the references to UmbracoCms.Core would kind of defeat the purpose of automating this. "Hey, use this script to set it up. Oh and hey now you have to go and manually install the NuGet package after that, sorry!".
I finally accidentally stumbled upon some code from someone who managed to install NuGet packages from C# meaning it could now be fully automated!
Team
Alright, so where are we now:
- Create a new project on Umbraco as a Service
- Download and run the Visual Studio automation script
Tada! We have a UaaS site running in VS and we can immediately start being productive.
Note: If you don't know what Puppy Monkey Baby is.. I am still confused too.
That leaves us with one thing for which we don't have the optimal solution yet: sharing this with your team. Right now this is a bit harder then we want it to be due to the restrictions I told you about earlier in the "Background" section. But I think it's manageable for now.
We do try to help as much as we can though: while running the VS automation script, an empty git repository was also set up for you and a .gitignore file was added so you can start committing your new site immediately and push it to whichever git hosting you prefer. The ignore file will ignore anything that's going on in your deployment repository (the .Web folder, containing the "end result" site for UaaS).
So for a team member to get set up, you now have a 2-step process:
- git clone `your custom code repository`
- git clone `UaaS git url`
Conveniently, you also get a UaaSClone.cmd file in the root for that second step, so they don't have to go to the UaaS portal to go and find the git URL. They can just double click UaaSClone and go. The end result should look something like this:
So:
- Git repository for your custom code
- NuGet packages folder for the packages installed into YourNameSpace.Core
- YourNameSpace.Core - where your custom code goes
- YourNameSpace.Web - the "end result" / deploy repository with the Umbraco website in it
- Git Ignore file using a default Visual Studio ignore plus an added rule to ignore YourNameSpace.Web
- The Visual Studio solution file
- UaaSClone.cmd for your colleagues to execute to get a copy of YourNameSpace.Web in the correct place
Try it now
So, without further ado: download the batch file and start playing.
You will need to enter the git clone URL that you can find in the UaaS portal for each of you projects and a namespace for Visual Studio. The process is captured in a video below (recommend watching in full screen and at least 720p to be able to read the text):
Note that the script is something that has only just in the past few days fully materialized so there may be rough edges. We wanted to put this out there so people can have a play with it and we can receive feedback for the team to build on this. So please, provide us with feedback to make your experience better. Leave comments below!
31 comments on this article
Great article. Just tried this out and it works very well. One little wrinkle I just noticed was that, after setting up in VS, I created a site with a home page and some content. Everything uploaded except the content. So I deleted my local content and recreated it in the UaaS project's back office. Next time I ran the local VS project it downloaded everything I'd built on the UaaS project. So what I'm trying to say is that it appears you must create all content in UaaS and not in VS. Does that sound right?
I think you might've missed a memo:
- Git deploy to UaaS will deploy all your site structure
- You can deploy your content from the backoffice in Umbraco, right-click on any content to deploy it or click the up arrow next to "Save and publish" to get to the "Deploy" button.
I got that, but thought it was just between the UaaS instances. Just tried it from my VS hosted back office and it deployed direct to the UaaS dev site. Impressed :) I'm really happy atm as I've added controllers and models and packages and they've all gone up and deployed faultlessly. Just had to add my .code dll into the SmartGit client as it wasn't recognised but that's all :)
No, you need to deploy content from the back office, and you can do that from your local clone too.
Bit more feedback.......
Don't know if you can do anything about this but my GIT client doesn't pick up changes relating to the addition of plugins. i.e. when I installed Archetype and it's Courier.dll, I had to show untracked files and select them. As you won't necessarily know what files to include, is it possible to modify the .gitignore file to be able to pick these changes up. In this case it was additions to the /bin and /App_Plugins directories.
Is it okay if I just answer that I dislike Smartgit? ;-)))
They're definitely untracked files (they're new), so that's correct. I'm not familiar with the SmartGit UI so don't know if that's the problem. At least on my end, when I add stuff to the site it shows them immediately.
Care to share your preferred GIT GUI or are you hard-core CLI? ;)
I do a lot of command line yes, I love Git Extensions, but other people tell me it's too geeky. :-) I guess I've been using it for so long that I am stuck with this now and have a hard time using anything else. ;)
Thanks for this Sebastiaan!
I'm currently following advice from the UaaS docs that states:
"The easiest way is to simply open the cloned project folder as a Web Site in VS. This approach is great if you are working alone or in a small team and you are comfortable with having your c# code in the /App_Code/ and /App_Plugins/ folders. By doing this there is literally nothing to set up. All you have to do is open the folder and you can run the site locally by pressing Ctrl-F5."
This has been working well and removed the need for a seperate source project/repo. Are there major benefits from a separate source project? Or potential risks/performance issues running from App_Code?
Hi Sebastiaan
I wass trying running the batch file like you did in the video, but somehow Waasp is crashing. I get the following errors: https://bjarnefyrstenborg.tinytake.com/sf/NzYzNjY0XzM0NzA3OTM
I don't see this path on my drive 'C:\Visual\Studio.Web\web.config'
Try using it in a path that doesn't have spaces in it.. looks like I didn't account for spaces to exist (thought I did).
Yes, I found out that the issue was regarding spaces in path.
When having an existing project on BitBucket with basic stuff like gulp tasks and other stuff, how to include it in this project? I can manually copy/paste the files, but is there and easier way to do it?
Usually we just fork the "starter project" and then clone the project locally and start working from that. I wonder how to do this in a UaaS project?
For now that's up to you how you would do it. I think I would just make a 3rd folder in the site (next to x.Web and x.Core) and put all your .less and whatever files need building in gulp there. Then the gulp build output can go to the .Web directory.
Okay, at the moment we have our Site/Web directory (which actually is a web application project) where it looks like this: https://bjarnefyrstenborg.tinytake.com/sf/NzY0MjQ5XzM0NzIxNDI
and then .less, .js, images etc. from /app folder is builded to a /dist folder ...
But we don't want "app" and "gulp" folder and other stuff like bower.json, package.json, gulpfile.js added to UaaS, right?
Just the "end-result" files and add the complete solution to our source control repository.
Correct. So you want all that stuff to live outside of the x.Web folder and build the "end-result" files into the x.Web folder instead of the /dist folder. :)
Of course you can still build it into a /dist folder as long as you then copy it over to where it needs to be in the x.Web folder (which is the UaaS repository)
Okay, when I create a folder structure /app/less at root an create a "styles.less" inside that folder, VS just create "styles.less" at root (without folder structure).
If I manually paste a folder "app" with subfolders and files to root of solution and re-open VS, it doesn't seem to see the files.
It seems the files have to be within a webapplication project or website?
I don't know what you're trying to achieve now? All I am saying is that your app/ directory needs to be outside of x.Web. After that it's up to you to figure out how to get the end-result files into x.Web.
If you want to edit your app/ files in Visual Studio it is up to you to set that up. I'd recommend adding a new website to the project. I have very little experience with any of that though (being a C# guy..) so I can't be of much help I'm afraid.
Visual Studio allow creating folders at same levels as the webapplication project and website. However when adding folders and files it doesn't seem to keep the folder structure in the solution folder - it just put all files at the same level without any of these folders.
So I guess VS only like files inside webapplication project or website. Folders just under root (solution) seems more to be for grouping projects and websites.
Can you create a topic on Our? This comment section is not really suitable for a discussion with screenshots etc. :) Would be good if you could do a small screencast because I really don't understand what the goal is and what doesn't work.
Sure, I have created a topic on Our here: https://goo.gl/nFtwAk
Nice blog. It explains the process very well. I tried to download automated script and try to run it. however the script fails after installing first package. I am getting following error:
Unhandled Exception: Microsoft.Build.Exceptions.InvalidProjectFileException: The imported project "C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\WebApplications\Microsoft.WebApplication.targets" was not found. Confirm that the path in the <Import> declaration is correct, and that the file exists on disk. C:\Users\aus11123\Desktop\Tech\UaaS\Amway-BSI.Core\Amway-BSI.Core.csproj
at Microsoft.Build.Shared.ProjectErrorUtilities.ThrowInvalidProject(String errorSubCategoryResourceName, IElementLocation elementLocation, String resourceName, Object[] args)
at Microsoft.Build.Evaluation.Evaluator`4.ExpandAndLoadImports(String directoryOfImportingFile, String importExpressionEscaped, ProjectImportElement importElement)
I have Visual Studio 2012.. Does this script works with VS2012?
Thanks.
@Nirmit you'll have to use Visual Studio 2015 for now, will update this in the near future.
It might help to install the MSBuild.Microsoft.VisualStudio.Web.targets NuGet package as mentioned here: http://stackoverflow.com/a/32278656/5018
Hi Sebastiaan
Is it possible to modify the command the show asterisk when typing password?
Furthermore it doesn't seem to be possible to copy/paste a password - however if typing wrong password or just empty and hitting enter the command seems to crash and you have to start over.
It would be great if it just returned a message about wrong password :)
Thanks,
Bjarne
No, asterisks are not possible. That prompt is coming from git.exe which we have no control over.
You can right-click to paste a password.
Yes, we're fixing it in this sprint to show a better error when the password is wrong (you're probably still going to have to start over).
Hi Sebastiaan, great post. Thanks
Is there any advice against customizing the .gitignore that comes with the cloned git repository from Umbraco Cloud ?
I am asking because I would like to install uSync in the local repo *but* exclude all related files from committing to the Umbraco Cloud Dev environment
Does the .gitignore get copied from dev to live when syncing environments ?
Thanks
Thanks Jose, please note that you can absolutely not use uSync with Umbraco Cloud. It interferes with Courier (also known as Umbraco Deploy, also known as Content Flow).
What would you suggest to import doctypes definitions or dictionary items created by the team members in their local web site?
I mean -in our case- not all team members have access to the Umbraco Cloud dev env because we have a local QA step before deploying to dev
Thanks a lot
I would suggest giving all your developers access to the Umbraco Cloud dev environment, that is the only way you will be successful in your project. If you keep using uSync you will run into issues which UC will not be able to help you with. :-)
Sebastiaan, thanks a lot for your recommendations and warnings regarding uSync. We are reconsidering to give our developers access to UC and learn how to work with Git branches
Getting back to my original question, is there any advice against customizing the .gitignore that comes with the cloned git repo from UC ?
We don't really support branching at the moment on Umbraco Cloud, so if you play with them then do please note that only the master branch will ever get deployed to your website on Umbraco Cloud. You can create all the branches you want but only the master branch changes will appear on UC.
It's your .gitignore file, you can do with it whatever you want. I would strongly recommend you do not remove the ignores we've given you by default though, they're all there for a reason.
Lastly, instead of this back and forth with questions, I would recommend you start experimenting, you get a 14 day free trial, you can try everything out, break everything you want on a trial project, delete it, start over fresh etc. :-)
That clarifies a lot.
We already have an UC paid subscription but I agree with you we should create a dummy project to experiment a lot more.
Please note that, despite the bumps we had while learning, we like UC very much and are actively promoting the service to our customers.
Thank very much you for your time