C6678 srio communication via Switch

First, I don't often give praise for support but I must say Travis, Karthik and Derek from TI have been extremely instrumental in getting my SRIO environment to work and bringing me up to speed on tips and tricks for SRIO.  It has been very nice to see success and progress.  Thanks again guys!

So in an effort to consolidate much of what I have learned, I will post here some information that I would have found extremely helpful 5 weeks ago :)

My environment:

I was using a aTCA chassis with a MCH that has an SRIO switch.  For more on my setup, please see this post:http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/t/164695.aspx  Here I say what parts I am using and some commands that I found helpful when working with the MCH.

My Goal:

Get DSP-A on C6678 EVM-A to send a message to DSP-B on C6678 EVM-B via an SRIO switch.

TI Example Programs:

When the TI programs are in loopback mode, they seem to work just fine.  They use the CSL (Chip Support Library) to access registers.  The CSL is nice.  Plus once you open a CSL handle, you can just access the structure with all the registers yourself (see example in the after port ok section).

To switch between loopback and "normal" mode use these CSLs:

CSL_SRIO_SetLoopbackMode(hSrio, 0);

CSL_SRIO_SetNormalMode(hSrio,0);

While it seemed to make sense for me to use the MultiCoreLoopback example or the ChipToChip example, it turns out these are extremely complex and make it difficult to learn/understand what is going on.

Travis recommended using the loopbackDioIsr example project.  This project is simpler using less of the queuing capabilities of the dsp.  This program essentially accesses an exact memory location as given in the LSU (load storage unit) registers at the destination ID.  So in the case of a loopback, it is just another location in DSP-A.  In normal mode, it is a memory location in DSP-B (assuming you set up the destination IDs).  Be careful, this also means you can write to any location in memory - any!

Switch and routing issues:

- Remember all device ideas have to be unique.  So if you use the same example program on each DSP (A&B), it won't work unless you change the device ids.

- Remember that the switch will need to be configured to properly route destination ids on the packets.  This can be done with maintanence packets or by direct configuration of the switch.  In my case, the switch has a default configuration file that I modified to route from DSP-A to DSP-B.

- Remember to make sure that the switch enables the input and output on the port you are using.  If it is not enabled only maintanence packets will be received.  All other types will be dropped.

- my switch could only accept one connection from one device ID.  The way the TI SRIO works is that you can have multiple port connections but they will always have the same device ID.  The examples are written to make (4) 1 lane connections to the other device.   You might need to adjust this portion of the examples if your switch is like mine.

- Travis says "port-writes" should be disabled unless you have a specific reason to use them.

- some switches need to be specially enabled to accept 16 bit destination ids.  Just something to keep in mind.

Understanding Ack IDs (from Travis):

Normal handshaking at the physical layer would be like this:

Device A sends a packet to Device B with ackid n

There is a transmission error on packet ackid n

Device B sees a CRC error and goes into Input error stopped state
(drops all new RX packets)

Device B sends a PNA control symbol to Device A

On reception of the PNA, Device A goes into output error stopped
state (stops sending any new packets)

Device A sends a LR Input status control symbol to device B

On reception of the LR input status, Device B sends a Link
maintenance response control symbol indicating packet ackid n was the PNA
packet.

Also, Device B then enters normal mode.

On reception of the link maintenance response packet, Device A goes
into normal mode and starts resending packets to device B stating with packet
ackid n

Things to check after "Port Ok":

After you get a port ok, if you are having problems sending messages here are some registers you should check (the listed register is for Port 0).

- ERR_Stat (TI register 0xB158)

- LM_RESP (TI register B144)

- ERR_Det (TI register C040)

- SP0_CTL (TI register B15C)

These collectively told me that the switch was not accepting my packets and in the end lead to the discovery that the switch had not enabled input and output messaging and was only accepting maintanence packets.

If you ever see the Output Error Stop condition or the Input Error Stop condition, there is a magic number that is to be written to a register.  In fact, Travis recommends doing this no matter what after receiving "Port Ok".

hSrio->RIO_PLM[i].RIO_PLM_SP_LONG_CS_TX1 = 0x2003F044;

System_printf("SRIO (Core %i): Correct Output Error Stop Condition.\n", coreNum);

After you have sent messages using the LSU, there is an LSU status register that is very helpful for indicating if the transfer was good or not.

Maintenance Packets:

Here is a short blip of code that I wrote to read a register from the switch via a maintanence packet.  this function will work with the dioIsr example.  Sorry about the formatting.

static Int32 maintanenceReadReg(Srio_SockHandle handle, UInt32srioReg)

{

Srio_SockAddrInfo       to;

uint16_t                compCode;

uint16_t                counter;

int32_t                 startTime;

UInt8 * pReadRespBuf = NULL;

UInt8 * pTmpRead = NULL;

pReadRespBuf = (uint8_t*)Osal_srioDataBufferMalloc(4);

if(pReadRespBuf == NULL)

{

System_printf("Error: pReadRespBuf Memory allocation failed.\n");

}

pTmpRead = pReadRespBuf;

for (counter = 0; counter < 4; counter++)

{

*pTmpRead++ = 0x55;

}

to.dio.rapidIOMSB    = 0x0;

to.dio.rapidIOLSB    = srioReg;//0x0015C;//(uint32_t)&dstDataBufPtr[srcDstBufIdx][0];

to.dio.dstID         = DEVICE_ID4_8BIT;

to.dio.ttype         = 0; //Read

to.dio.ftype         = 8; //Maintanence packets

/* Send the DIO Information. */

if (Srio_sockSend_DIO (handle, pReadRespBuf, 4, (Srio_SockAddrInfo*)&to) < 0)

{

System_printf ("Error: (Core %d): Could not send message.\n", coreNum);

return -1;

}

/* Wait for the interrupt to occur without touching the peripheral. */

/* Other useful work could be done here such as by invoking a scheduler */

startTime = TSCL;

while((! srioLsuIsrServiced) && ((TSCL - startTime) < SRIO_DIO_LSU_ISR_TIMEOUT));

if (! srioLsuIsrServiced) {

System_printf ("ISR didn't happen within set time - %d cycles. Example failed !!!\n", SRIO_DIO_LSU_ISR_TIMEOUT);

return -1;

}

Osal_srioDataBufferFree(pReadRespBuf, 4);

return 0;

}

Calling the function:

maintanenceReadReg(mySrioSocket, 0x15C);

CSL_SRIO_ClearLSUPendingInterrupt (hSrioCSL, 0xFFFFFFFF, 0xFFFFFFFF);

srioLsuIsrServiced = 0;

This clearingLSUPending interrupt is important - it has to happen after each transmission (at least in this example).

Various Other Posts to check:

http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/t/168310.aspx

http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/t/168310.aspx

http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/t/167006.aspx

http://e2e.ti.com/support/dsp/c6000_multi-core_dsps/f/639/t/165949.aspx

I am sure I have forgotten a few things but hopefully this will get you started and post away, hopefully Travis, Derek or Karthik will see it and be able to help!

Good luck!

Brandy

PS - thanks again guys, it feels great to be moving forward!!