STM32F4, USB HS with ULPI and Suspend/Wakeup

Hi guys,
I am in need of your help, unfortunately STs documentation is lacking some information here.

I am basically trying to implement a USB device (CDC-ACM to be precise) that utilizes suspend/wakeup. I am using the "STM32_USB-Host-Device_Lib_V2.1.0" and looked into the HID example for help. 
Unfortunately there is no example that shows the USB HS with suspend/wakeup using an external ULPI. I am using the SMSC USB3300 ULPI chip with an external 24MHz crystal.

The first thing that baffles me is in dcd_int.c:
Below the function DCD_HandleUSBSuspend_ISR gets called on a suspend interrupt. This interrupt gets called successfully.

uint32_t USBD_OTG_ISR_Handler (USB_OTG_CORE_HANDLE *pdev)
{
    ...
    if (gintr_status.b.usbsuspend)
    {
      retval |= DCD_HandleUSBSuspend_ISR(pdev);
    }
    ....
}

Now, inside this function I see:

if((pdev->cfg.low_power) && (dsts.b.suspsts == ) &&
  (pdev->dev.connection_status == ) &&
  (prev_status  == USB_OTG_CONFIGURED))
{
  /*  switch-off the clocks */
      ....
}

The strange part now is, that I cannot find another reference of dev.connection_status, which gets tested in this if-statement, thus this variable never gets to 1 and I wonder why and what I am missing.
Also prev_status (which gets set by dev.device_status) never equals USB_OTG_CONFIGURED which is defined as 3. It always is equal to 2, which is defined as USB_OTG_ADDRESSED

This lead to the problem that obviously, the ULPI never went into the low power mode. I commented out the two non-reachable if-conditions and now I can see the USB3300 to stop its external 24MHz oscillator. I should note that I commented out the deepsleep part in this function

Now comes the second thing, I can not yet understand. As in the HID example, I activated the EXTI_Line20 and enabled the OTG_HS_WKUP_IRQn in USB_OTG_BSP_Init. Now when I disable my device in the windows device manager, I no longer see the SOF microframes on my oscilloscope. The suspend interrupt gets called and I disable the USB3300 clocks as shown above. 
Unfortunately, the OTG_HS_WKUP_IRQHandler gets called now immediately and with the call to USB_OTG_UngateClock enables the USB3300 clocks again. After some time, the HS core sees the suspend condition on the bus and goes to suspend AGAIN. This keeps repeating on and on.

So I had some long looks into the Ref Man having the tongue at the right angle and read something like:
EXTI line 20 is connected to the USB OTG HS (configured in FS) Wakeup event

So I wondered: since I am using HS usb with external phy and the USB OTH HS core, is this even necessary for waking up? Maybe the strange behaviour is here? 
The sad thing is, that I can't seem to find anything on people trying to use USB suspend with external PHYs.
As far as I understood, the EXTI Line 20 is used to asynchronously detect any changes on the USB lines and feed it into the NVIC, which in turn knows that it has to call the OTG_HS_WKUP_IRQHandler.
Now since I am using the ULPI, I don't think this is valid anymore. However, there seems to be yet another instance of wakeup interrupts buried within the HS core: OTG_HS_GINTSTS contains an interrupt status flag for waking up: WKUINT
And now I am completely confused why there are so many configuraiton possibilites for the wakeup interrupt. 
Again, since the documentation is lacking information on this (or I simply can not find it, though searching for some days now) I can not figure out, how to correctly implement USB HS wake up using the external PHY.

This is why am I am searching for help here in the forum and hope someone can shed some light onto this. I am very much looking forward to any means of helping, thanks!

EDIT at 14:58. Found this in the reference manual of the USB stack:
Even if the #define USB_OTG_HS_LOW_PWR_MGMT_SUPPORT is uncommented, the
USB HID Device enters Low Power mode but cannot wakeup after a new Host
connection.
This behavior is normal since the wakeup signal in the USB OTG HS Core is available
only when the USB OTG HS core is working in Full Speed mode.

What exactly does that mean? Do I have to set the core into FS mode after suspend and then wait for wake up? Or should I use FS mode only?

Hi

Yes, I cannot get it to work either.
Im working on USB CDC/VCP.

"The strange part now is, that I cannot find another reference of dev.connection_status"
Yes, you have to add
#define VBUS_SENSING_ENABLED
to usb_conf.h

to enable the connect/disconnect ISRs which then change the state.

"After some time, the HS core sees the suspend condition on the bus and goes to suspend AGAIN. This keeps repeating on and on."
Yes, I got this as well.

I have put in a request for help with our ST rep

Hi!
I found the problem with dev.connection_status, you are right as in that I had to define
#define VBUS_SENSING_ENABLED
#define USB_OTG_INTERNAL_VBUS_ENABLED

to use with my USB3300 ULPI chip. Now i get the correct connection status in the suspend ISR.

In the meantime I managed to enumerate in Full Speed mode using the ULPI chip and my hope was, that suspend would work better here,
as the USB library manual states. But I was disappointed, because I get the exact same behaviour as before:
As soon as the device notices the suspend, it disables the clocks. But immediately after that, I get the wake interrupt and enter the loop I have described before.

I am somehow glad, that I am not the only one having this exact problem.

I found something else, that is rather curious: Why does "the" USBD_OTG_ISR_Handler contain this

if (gintr_status.b.wkupintr)

Although I could never get the HS core to set the wkupintr bit and execute the OTG_HS_IRQHandler, although the mask bit in GREGS->GINTMSK is set (in USB_OTG_EnableCommonInt)

And why does it even exist, when there is the OTG_HS_WKUP_IRQHandler that is supposed to be used for waking?

I guess I am doing something wrong regarding the wake up handling. Lets see what STM has to say about this.

Bye!

For the sake of completeness, here is the contents of my Wakeup ISR Handler

void OTG_HS_WKUP_IRQHandler(void)
{
      if(USB_OTG_dev.cfg.low_power) {
          USB_OTG_CORE_HANDLE* pdev = &USB_OTG_dev;
          USB_OTG_GINTSTS_TypeDef  gintsts;
          USB_OTG_DCTL_TypeDef     devctl;
          USB_OTG_PCGCCTL_TypeDef  power;

          USB_OTG_UngateClock(&USB_OTG_dev);

          /* Clear the Remote Wake-up Signaling */
          devctl.d32 = ;
          devctl.b.rmtwkupsig = ;
          USB_OTG_MODIFY_REG32(&pdev->regs.DREGS->DCTL, devctl.d32, );

          /* Inform upper layer by the Resume Event */
          USBD_DCD_INT_fops->Resume (pdev);

          /* Clear interrupt */
          gintsts.d32 = ;
          gintsts.b.wkupintr = ;
          USB_OTG_WRITE_REG32 (&pdev->regs.GREGS->GINTSTS, gintsts.d32);

          uint32_t A = USB_OTG_READ_REG32(&pdev->regs.PCGCCTL);
          uint32_t B = USB_OTG_READ_REG32(&pdev->regs.GREGS->GINTSTS);
          uint32_t C = USB_OTG_READ_REG32(&pdev->regs.GREGS->GUSBCFG);
      }

      NVIC_ClearPendingIRQ(OTG_HS_IRQn);
      EXTI_ClearITPendingBit(EXTI_Line20);
}

To be more precise: If i re-enable my USB device in the device manager again, and the host starts spitting out SOFs again, the STM32 breaks the loop (probably because the suspend function is not called anymore)

Just to let you know, I found something veeeery interesting in the code of the STM32CubeF4 here:
http://www.st.com/web/en/catalog/tools/PF259243#
File:
stm32cubef4.zip\STM32Cube_FW_F4_V1.1.0\Projects\STM324xG_EVAL\Applications\USB_Device\HID_Standalone\Src\usbd_conf.c

void HAL_PCD_SuspendCallback(PCD_HandleTypeDef *hpcd)
{
    __IO uint32_t i=;

  if  (hpcd->Instance == USB_OTG_HS)
  {
    __HAL_USB_HS_EXTI_DISABLE_IT();

    __HAL_PCD_GATE_PHYCLOCK(hpcd);

    /*wait tiemout of 6 ULPI PHY clock ~= 18 cpu clocks @168MHz*/
    for (i=; i<; i++)
    {
      __NOP();
    }

    if (__HAL_PCD_IS_PHY_SUSPENDED(hpcd))  /* when set then false resume condition*/
    {
      __HAL_USB_HS_EXTI_CLEAR_FLAG();
      __HAL_USB_HS_EXTI_ENABLE_IT();

      USBD_LL_Suspend(hpcd->pData);

      /*Enter in STOP mode */
      if (hpcd->Init.low_power_enable)
      {
        /* Set SLEEPDEEP bit and SleepOnExit of Cortex System Control Register */
        SCB->SCR |= (uint32_t)((uint32_t)(SCB_SCR_SLEEPDEEP_Msk | SCB_SCR_SLEEPONEXIT_Msk));
      }
    }
  }
  else
    {

        __HAL_PCD_GATE_PHYCLOCK(hpcd);
        USBD_LL_Suspend(hpcd->pData);

  /*Enter in STOP mode */
  if (hpcd->Init.low_power_enable)
  {

    /* Set SLEEPDEEP bit and SleepOnExit of Cortex System Control Register */
    SCB->SCR |= (uint32_t)((uint32_t)(SCB_SCR_SLEEPDEEP_Msk | SCB_SCR_SLEEPONEXIT_Msk));
  }

    }

}

It looks like there are some specialties regarding HS core and suspending. As far as I can see, the EXTI line for wake up is deactivated while the HS core is put into suspend.

And of course, THIS was the problem. I reverse engineered the code from the zip file and entered it into the usb stack
I quote from usb_dcd_int.c:

static uint32_t DCD_HandleUSBSuspend_ISR(USB_OTG_CORE_HANDLE *pdev)
{
  USB_OTG_GINTSTS_TypeDef  gintsts;
  USB_OTG_PCGCCTL_TypeDef  power;
  USB_OTG_DSTS_TypeDef     dsts;
  __IO uint8_t prev_status = ;

  prev_status = pdev->dev.device_status;
  USBD_DCD_INT_fops->Suspend (pdev);     

  dsts.d32 = USB_OTG_READ_REG32(&pdev->regs.DREGS->DSTS);

  /* Clear interrupt */
  gintsts.d32 = ;
  gintsts.b.usbsuspend = ;
  USB_OTG_WRITE_REG32(&pdev->regs.GREGS->GINTSTS, gintsts.d32);

  if((pdev->cfg.low_power) && (dsts.b.suspsts == )  &&
    (pdev->dev.connection_status == ) )//&&
   // (prev_status  == USB_OTG_CONFIGURED))
  {
#ifndef USE_USB_OTG_HS

    /*  switch-off the clocks */
    power.d32 = ;
    power.b.stoppclk = ;
    USB_OTG_MODIFY_REG32(pdev->regs.PCGCCTL, , power.d32); 

    power.b.gatehclk = ;
    USB_OTG_MODIFY_REG32(pdev->regs.PCGCCTL, , power.d32);
#else
#define USB_HS_EXTI_LINE_WAKEUP              ((uint32_t)0x00100000)  /*!< External interrupt line 20 Connected to the USB HS EXTI Line */

    /* Disable EXTI for wakupe */
      EXTI->IMR &= ~(USB_HS_EXTI_LINE_WAKEUP);

      /* Gate PHY clock */
      power.d32 = ;
      power.b.stoppclk = ;
      USB_OTG_MODIFY_REG32(pdev->regs.PCGCCTL, , power.d32);

      power.b.gatehclk = ;
      USB_OTG_MODIFY_REG32(pdev->regs.PCGCCTL, , power.d32);

      /* Wait timeout of 6 ULPI PHY clock. found this in STM324 HID eval code */
      for (uint32_t i=; i<; i++) {
          __NOP();
      }

      power.d32 = USB_OTG_READ_REG32(pdev->regs.PCGCCTL);

      if (power.b.phy_susp) {
          /* Clear flag and enable IT */
          EXTI->PR = (USB_HS_EXTI_LINE_WAKEUP);
          EXTI->IMR |= (USB_HS_EXTI_LINE_WAKEUP);
      }
#endif

    /* Request to enter Sleep mode after exit from current ISR */
    //SCB->SCR |= (SCB_SCR_SLEEPDEEP_Msk | SCB_SCR_SLEEPONEXIT_Msk);
  }

  return ;
}

And everything works as expected. I suspect the culprit to be the missing 6 ULPI clock delays. If you look carefully in the USB3300 datasheet, it says that after disabling the clocks, the ULPI is active for another 5 (or something) clock cycles, before it eventually shuts down and feeds through the USB lines to the ULPI in order for the OTG HS core to monitor them for wake up.