Unable to push packages

daniels's Avatar

daniels

08 Sep, 2017 07:29 PM

Working with a newly created Octopus Server. A week ago or so - I had no issues, but now when I try to push packages I get the following

"D:\work\_tasks\OctopusPush_d05ad9a2-5d9e-4a1c-a887-14034334d6f2\2.0.78\Octo.exe" push --server=http://s01vci02/app#/ --apiKey=******** --package="D:\work\117\a\PartnerPortal.20170908.1.nupkg"
Octopus Deploy Command Line Tool, version 4.22.1
Unable to process response from server: Unexpected character encountered while parsing value: <. Path '', line 0, position 0.. Response content: <!DOCTYPE html>
<html lang="en">
<head>
  <title ng-bind="$root.pageTitle + ' - Octopus Deploy'">Octopus Deploy</title>
  
  <link href="vendor.8ba204e6d5237deeca85749438386f2c.hashedasset.css" rel="stylesheet">
  
  <link href="application.6a99da123c96a66f66ec38f07cf05017.hashedasset.css" rel="stylesheet">
  
  <link href="application.6a99da123c96a66f66ec38f07cf05017.hashedasset-blessed1.css" rel="stylesheet">
  
  <link href="application.6a99da123c96a66f66ec38f07cf05017.hashedasset-blessed2.css" rel="stylesheet">
  
  <link ng-repeat="cssFile in $root.ExtensionsCSS" rel="stylesheet" ng-href="{{ cssFile | resolveLink }}" />
  <link rel="apple-touch-icon" href="Octopus-96x96.0701d7ff661133eafb2bbd23656eb7db.hashedasset.png" />
  <link rel="icon" href="Octopus-96x96.0701d7ff661133eafb2bbd23656eb7db.hashedasset.png" />
  <!--[if IE]><link rel="shortcut icon" href="img/icons/Octopus-16x16.ico"><![endif]-->
  <meta name="msapplication-TileColor" content="#2F93E0">
  <meta
Error from Octopus server (HTTP 200 OK)
Exit code: -7
Process 'Octo.exe' exited with code '-7'.

  1. 1 Posted by daniels on 08 Sep, 2017 07:31 PM

    daniels's Avatar

    Further information, specifically pushing from a VSTS On-premise build agent. I have no problems manually uploading the packages from the Octopus Portal

  2. Support Staff 2 Posted by Dalmiro Grañas on 08 Sep, 2017 07:40 PM

    Dalmiro Grañas's Avatar

    Hi,

    Thanks for reaching out. The problem seems to be that you registered your Octopus server in VSTS with the URL http://s01vci02/app#/ when it should have been http://s01vci02 without that last bit.

    Try re-registering it like that and let me know if that works.

    Regards,
    Dalmiro

  3. 3 Posted by daniels on 08 Sep, 2017 07:48 PM

    daniels's Avatar

    Thanks. Oddly, when I first tried it today with that setting - run with

    2017-09-08T16:05:53.8432429Z ##[command]"D:\work\_tasks\OctopusPush_d05ad9a2-5d9e-4a1c-a887-14034334d6f2\2.0.78\Octo.exe" push --server=http://s01vci02/ --apiKey=******** --package="D:\work\119\a\Publish\InSyncWeb\packages\ActiveReports6.6.1.2577.2\ActiveReports6.6.1.2577.2.nupkg" --package="D:\work\119\a\Publish\InSyncWeb\packages\AjaxControlToolkit.4.1.51116\AjaxControlToolkit.4.1.51116.nupkg" --package="D:\work\119\a\Publish\InSyncWeb\packages\ASPNetSpell.4.0.3.24577\ASPNetSpell.4.0.3.24577.nupkg" --package="D:\work\119\a\Publish\InSyncWeb\packages\ATI.Logging.dll.1.0.0\ATI.Logging.dll.1.0.0.nupkg" --package="D:\work\119\a\Publish\InSyncWeb\packages\DBauer.Web.UI.WebControls.DynamicControlsPlaceholder.dll.2.2.0\DBauer.Web.UI.WebControls.DynamicControlsPlaceholder.dll.2.2.0.nupkg" --package="D:\work\119\a\Publish\InSyncWeb\packages\IntegrationClasses.dll.1.17.1020.1-RC04\IntegrationClasses.dll.1.17.1020.1-RC04.nupkg" --package="D:\work\119\a\Publish\InSyncWeb\packages\IntegrationClasses.dll.1.17.825.1-RC06\IntegrationClasses.dll.1.17.825.1-RC06.nupkg" --package="D:\work\119\a\Publish\InSyncWeb\packages\iTextSharp-LGPL.4.1.6\iTextSharp-LGPL.4.1.6.nupkg" --package="D:\work\119\a\Publish\InSyncWeb\packages\Lucene.Net.2.9.4.1\Lucene.Net.2.9.4.1.nupkg" --package="D:\work\119\a\Publish\InSyncWeb\packages\Microsoft.ApplicationBlocks.Data.dll.1.0.2774.15308\Microsoft.ApplicationBlocks.Data.dll.1.0.2774.15308.nupkg" --package="D:\work\119\a\Publish\InSyncWeb\packages\Newtonsoft.Json.7.0.1\Newtonsoft.Json.7.0.1.nupkg" --package="D:\work\119\a\Publish\InSyncWeb\packages\Newtonsoft.Json.9.0.1\Newtonsoft.Json.9.0.1.nupkg" --package="D:\work\119\a\Publish\InSyncWeb\packages\PDFLibNet.1.0.0.1\PDFLibNet.1.0.0.1.nupkg" --package="D:\work\119\a\Publish\InSyncWeb\packages\PostSharp.dll.2.0.9.30\PostSharp.dll.2.0.9.30.nupkg" --package="D:\work\119\a\Publish\InSyncWeb\packages\RFComAPILib.1.0.0.1\RFComAPILib.1.0.0.1.nupkg" --package="D:\work\119\a\Publish\InSyncWeb\packages\SharpZipLib.0.86.0\SharpZipLib.0.86.0.nupkg" --package="D:\work\119\a\Publish\InSyncWeb\packages\SmartOption.ServiceLibrary.Model.dll.1.17.825.1-RC04\SmartOption.ServiceLibrary.Model.dll.1.17.825.1-RC04.nupkg" --package="D:\work\119\a\Publish\InSyncWeb\packages\SubSonic.dll.2.2.0\SubSonic.dll.2.2.0.nupkg" --package="D:\work\119\a\Publish\InSyncWeb\packages\TelerikReporting.8.0.14.225\TelerikReporting.8.0.14.225.nupkg" --package="D:\work\119\a\Publish\InSyncWeb\packages\Touchstone.External.PatientPortal.BL.dll.1.17.1020.1-RC02\Touchstone.External.PatientPortal.BL.dll.1.17.1020.1-RC02.nupkg" --package="D:\work\119\a\Publish\InSyncWeb\packages\Touchstone.External.PatientPortal.BL.dll.1.17.825.1-RC08\Touchstone.External.PatientPortal.BL.dll.1.17.825.1-RC08.nupkg" --package="D:\work\119\a\InSync.20171020.20170908.2.nupkg"

    and then

    <div class="content-container"><fieldset>
    2017-09-08T16:05:54.3879126Z <h2>404 - File or directory not found.</h2>
    2017-09-08T16:05:54.3879126Z <h3>The resource you are looking for might have been removed, had its name changed, or is temporarily unavailable.</h3>
    2017-09-08T16:05:54.3879126Z </fieldset></div>

    So I changed to map what I put in the URL. I changed it back (also fixed sending over packages I don't need)

    Per your instructions, I switched it back and now it's working! Something obviously got out of sync and is now fixed.

    Much thanks!

  4. Support Staff 4 Posted by Dalmiro Grañas on 08 Sep, 2017 07:58 PM

    Dalmiro Grañas's Avatar

    Glad to hear it worked :)

    That "register URL" box that Microsoft provides unfortunately doesn't allow us to warn users when they register incorrect URLs. Registering it with app#/ is a very common and understandable mistake.

    I'm gonna submit a PR for our plugin to do some validations on that URL before we attempt to run the octo.exe command. Something that prints a warning if the URL ends in /app#/ or /api/ which are the 2 most common scenarios.

    Have a great weekend :)

  5. 5 Posted by daniels on 08 Sep, 2017 08:18 PM

    daniels's Avatar

    Unfortunately it worked exactly once and now everything is failing again.

    Now I'm getting the 404.

    Package step and push shown. Note that I even deleted the push step and recreated.

    ******************************************************************************
    Starting: Package InSync
    ******************************************************************************
    No Release notes file found
    "D:\work\_tasks\OctopusPack_179fac12-2402-486e-80cf-5a6a8571f7c0\2.0.78\Octo.exe" pack --id="InSync" --format=NuPkg --version=20171020.20170908.9 --outFolder="D:\work\119\a" --basePath="D:\work\119\a\Publish" --author="" --title="" --description="" --releaseNotes="" --overwrite=False
    Octopus Deploy Command Line Tool, version 4.22.1
    Packing InSync version "20171020.20170908.9"...
    Saving "InSync.20171020.20170908.9.nupkg" to "D:\work\119\a"...
    Done.
    ******************************************************************************
    Finishing: Package InSync
    ******************************************************************************
    ******************************************************************************
    Starting: Push Packages to Octopus
    ******************************************************************************
    "D:\work\_tasks\OctopusPush_d05ad9a2-5d9e-4a1c-a887-14034334d6f2\2.0.78\Octo.exe" push --server=http://s01vci02/ --apiKey=******** --package="D:\work\119\a\InSync.20171020.20170908.9.nupkg"
    Octopus Deploy Command Line Tool, version 4.22.1
    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
    <html xmlns="http://www.w3.org/1999/xhtml">
    <head>
    <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1"/>
    <title>404 - File or directory not found.</title>
    <style type="text/css">
    <!--
    body{margin:0;font-size:.7em;font-family:Verdana, Arial, Helvetica, sans-serif;background:#EEEEEE;}
    fieldset{padding:0 15px 10px 15px;}
    h1{font-size:2.4em;margin:0;color:#FFF;}
    h2{font-size:1.7em;margin:0;color:#CC0000;}
    h3{font-size:1.2em;margin:10px 0 0 0;color:#000000;}
    #header{width:96%;margin:0 0 0 0;padding:6px 2% 6px 2%;font-family:"trebuchet MS", Verdana, sans-serif;color:#FFF;
    background-color:#555555;}
    #content{margin:0 0 0 2%;position:relative;}
    .content-container{background:#FFF;width:96%;margin-top:8px;padding:10px;position:relative;}
    -->
    </style>
    </head>
    <body>
    <div id="header"><h1>Server Error</h1></div>
    <div id="content">
     <div class="content-container"><fieldset>
      <h2>404 - File or directory not found.</h2>
      <h3>The resource you are looking for might have been removed, had its name changed, or is temporarily unavailable.</h3>
     </fieldset></div>
    </div>
    </body>
    </html>
    Error from Octopus server (HTTP 404 NotFound)
    Exit code: -7
    Process 'Octo.exe' exited with code '-7'.

  6. 6 Posted by daniels on 08 Sep, 2017 08:19 PM

    daniels's Avatar

    Note that when I entered the URI it didn't have the trailing "/"

  7. Support Staff 7 Posted by Dalmiro Grañas on 08 Sep, 2017 08:40 PM

    Dalmiro Grañas's Avatar

    If it worked once with that same URL (which looks ok to me) and then it failed, then we have to start thinking about the network aspect.

    You mentioned your build agent is on-prem. In that case what I would try to do is:

    1) Remote into that build agent and download octo.exe to it (download link)

    2) Open a web browser on the agent and try to access the Octopus server from http://s01vci02/. You should be able to see the Octopus web portal for this to work.

    3) If (2) worked, try running the same Octo.exe command you see on the build log, but from a local powershell session in that build agent VM.

    4) If you keep getting intermittent results, you might wanna check your Octopus Server logs and see if the portal is working intermittently. If that's the case, you should see plenty of messages saying the service started/stopped. In the Octopus Server VM the logs would be in C:\Octopus\Logs by default.

    Let me know how that goes

  8. 8 Posted by daniels on 08 Sep, 2017 09:10 PM

    daniels's Avatar

    2) When I did that - there was a noticeable delay before the page loaded - so it seems we do have an issue on the network

    3) I did that and no error. I didn't remember API key - so I created a new one.

    I did wonder if perhaps there is an issue with the old API key and my updating the Service Endpoint -so to be safe -I deleted and recreated with the "new" API Key

    Running after that update - it succeeded. The build server is also under heavy load - I'm wondering if that may be a factor.

    I didn't see any errors on the server log -so wondering if that's another sign the problem is on the Build Agent side, not Octopus side.

    Thanks for your assistance

  9. Support Staff 9 Posted by Dalmiro Grañas on 11 Sep, 2017 07:36 PM

    Dalmiro Grañas's Avatar

    2) When I did that - there was a noticeable delay before the page loaded - so it seems we do have an issue on the network

    This is most likely the root cause of the issue.

    Running after that update - it succeeded. The build server is also under heavy load - I'm wondering if that may be a factor.

    All the build server does is send a POST call to the Octopus server, which starts the deployment and sends the order to the Tentacles. So the build server is not exactly doing a heavy-load work really. In this case I would investigate (in this order):

    • Resource usage on the Octopus Server VM while the process is running.
    • Resource usage on the Tentacles while process is running.
    • Network connection between Build agent <-> Octopus Server.
    • Network connection between Tentacle <-> Octopus Server
  10. 10 Posted by Neil on 03 Oct, 2017 10:34 AM

    Neil's Avatar

    Hi,
    I too have a strange problem with Octo.exe push.
    1. I have a 118MB zip file (created by octopack)
    2. From the command line on a windows VM, I execute octo.exe push
    3. It succeeds (every time).
    4. My colleague, logs onto the same VM and runs the exact same command (same API key, same file path, same url) and he gets the same error that has been reported here (every time he runs it):
    Unexpected character encountered while parsing value: <. Path '', line 0, position 0..
    5. Curiously, if the file is split into two smaller files, my colleague can successfully upload both files.
    6. The failure occurs within a couple of seconds (so it doesn’t seem to be timeout related).
    7. This behaviour occurs in the latest octo.exe (and in the previous version we were using).

    Any help would be most welcome.
    Regards,
    Neil

  11. Support Staff 11 Posted by Dalmiro Grañas on 03 Oct, 2017 07:07 PM

    Dalmiro Grañas's Avatar

    Hi Neil,

    Thanks for reaching out. The error Unexpected character encountered while parsing value: < usually means that the code was expecting a JSON blob as a result, but instead got an HTML page (which starts with the character <). This often means that the request returned an error in the form of an HTML page.

    There's nothing in the Octopus code that would limit package push depending on the user and size of the package. So I'm thinking this could be some sort of network.

    What I recommend you to do is:

    1) Run the Octo.exe command with the flag --debug to see if you get extra logging that might help with the troubleshooting.

    2) Setup Fiddler to monitor the network calls being made by that command. Sometimes Fiddler returns a bit more logging to help in these situations.

    3) Can your teammate try to push the package from another machine (lets say his workstation) and see if he gets the same result?

    Regards,
    Dalmiro

  12. 12 Posted by Neil on 03 Oct, 2017 11:53 PM

    Neil's Avatar

    Hi Dalmiro,
    Thanks for your response.
    The --debug option didn't give us much unfortunately, and before we could test with fiddler, another colleague of mine, (much smarter than I am!) suggested looking at the Proxy settings (under Local Area Network (LAN) Settings - From IE go to Tools/Internet Options/Connections/Lan settings).
    And sure enough we discovered that ticking the "Bypass proxy server for local addresses" resolved our issue.
    I guess our proxy were was being used to route local traffic and it didn't like the large file size.
    Problem solved.
    Regards,
    Neil

  13. Support Staff 13 Posted by Dalmiro Grañas on 04 Oct, 2017 12:20 AM

    Dalmiro Grañas's Avatar

    Hi Neil,

    Glad to hear you found someone smarter than both of us to help you out :)

    Cheers!

  14. 14 Posted by ianpaullin on 11 Oct, 2017 04:56 AM

    ianpaullin's Avatar

    We're having a similar issue pushing a zip file from VSTS to an Octopus instance of ours and we're getting a HTTP 404 error. How can we disable proxying from a VSTS build controller? We only seem to have this issue for one project that's using .zip files.

  15. Support Staff 15 Posted by Dalmiro Grañas on 11 Oct, 2017 06:41 PM

    Dalmiro Grañas's Avatar

    Hey Ian!

    Just to double check: When you say VSTS build controller you mean one hosted by Microsoft and not by your company, correct?

    Could you share me a build log so I can see what you are using to push the package (octo.exe,nuget.exe,etc) along with the rest of the build context? This conversation is public so I recommend you to send it to our email instead.

    Dalmiro

  16. 16 Posted by ianpaullin on 12 Oct, 2017 03:46 PM

    ianpaullin's Avatar

    Hi Dal! We figured out the issue. It was our IIS ARR configuration. By default, there's a requestlimit in bytes value set to 30,000 bytes (approx. 30MB). I changed the value to 2GB and that solved our problem. /doh

    Previously we only used AWS ELB and Route53, but moving to IIS ARR we forgot about this default setting. Most of our teams were deploying packages that were under this 30MB limit up until someone had to go over that limit. Lesson learned.

  17. Support Staff 17 Posted by Dalmiro Grañas on 12 Oct, 2017 05:15 PM

    Dalmiro Grañas's Avatar

    Glad to hear you figured it out :)

Reply to this discussion

Internal reply

Formatting help / Preview (switch to plain text) No formatting (switch to Markdown)

Attaching KB article:

»

Attached Files

You can attach files up to 10MB

If you don't have an account yet, we need to confirm you're human and not a machine trying to post spam.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac