I would have two questions in regards to preprocessor and output stream. I had some program in C, and I wanted to rewrite it to brook+.  I'm not sure if I am doing it properly, so : this my part of code in C:

void fun (int width, int height, short* in, short* o) {
 
  for(int y = 1; y < height-1; y ++) {
   for(int x = 1; x < width-1; x ++) {
   int pi = (x + y * width) * 3;
     o[pi + 0] = 10;
     o[pi + 1] = 20;
     o[pi + 2] = 30;

... }

My first question: how could I make this ( for ex. o[pi + 2] = 10; ) access in brook+. I've tried something like this: My kernel:

kernel
void fund(  int width,  int height, float input[], out float output<> { 
 int2 index = instance().xy;

int pi = (index.y + index.x * width) * 3;
   
  output+=( ( float )pi + 0)+10;
  output+=( ( float )pi + 1)+20;
  output+=( ( float )pi + 2)+30;

...}

I'm not sure if is it correct ??

The second question is about preprocessor: I would like to for ex. use

#define FX(xo, yo) in[(y + yo)*width + (x + xo)]

void fun (int width, int height, short* in, short* o) {..} i

n my code but I'm still getting error: "Problem with call expression in kernel: callee unknown"

Thank you for your answers in advice,

mpwm

\\Users\admin\Documents\ATI Stream\lib\x86_64

(ati-stream-sdk-v2.0-beta4-vista-win7-64.exe)

but for some samples it is needed (e.g. Mandelbrot)

However, in all the SDK sample, while setting CAL_FORMAT_FLOAT4 as a type of the data, width parameter is not devided by 4. Is it just a relaxed restrictions in samples (as type of the data is controlled via cmd args) and in practice if CAL_FORMAT_XXX4 is set, width must be devided by 4 to save the memory?

The y[] consists of {1.0f, 1.0f, 1.0f, 1.0f}

Thread 2 (Thread 0xb7ab6b90 (LWP 21941)):
#0  0xb7f4642e in __kernel_vsyscall ()
#1  0xb7f22cf9 in __lll_lock_wait () from /lib/tls/i686/cmov/libpthread.so.0
#2  0xb7f1e129 in _L_lock_89 () from /lib/tls/i686/cmov/libpthread.so.0
#3  0xb7f1da32 in pthread_mutex_lock () from /lib/tls/i686/cmov/libpthread.so.0
#4  0xb7268d2b in brook::ThreadLock::lock () from /usr/lib/libbrook.so
#5  0xb72a80c6 in CALBuffer::initializePinnedBuffer () from /usr/lib/libbrook_cal.so
#6  0xb729ac64 in CALBufferMgr::_createPinnedBuffer () from /usr/lib/libbrook_cal.so
#7  0xb729bf07 in CALBufferMgr::setBufferData () from /usr/lib/libbrook_cal.so
#8  0xb725a093 in StreamImpl::read () from /usr/lib/libbrook.so
#9  0xb7c0b20c in brook::StreamData::read () from /usr/lib/libbrook_d.so
#10 0xb7c5dce9 in brook::Stream<uint4>::read (this=0x9e43960, ptr=0x9e54900, flags=0xb7c71c99 "nocopy")
    at /usr/local/atibrook/sdk/include/brook/StreamDef.h:160
#11 0xb7c5b49c in A5Slice::tick (this=0x9b223c8) at A5Slice.cpp:366
#12 0xb7c4b5c2 in BrookA5:rocess (this=0x9b25870) at A5Brook.cpp:139
#13 0xb7c4b637 in BrookA5::thread_stub (arg=0x9b25870) at A5Brook.cpp:52
#14 0xb7f1c4ff in start_thread () from /lib/tls/i686/cmov/libpthread.so.0
#15 0xb7e5249e in clone () from /lib/tls/i686/cmov/libc.so.6

When this first happened I issued 18 async read calls, I tried serializing the read operations with isSync calls, but the result is the same. Also it does not appear to be a general race condition, as the hang occurs after the exact same number of kernel invocations.

Since this behaviour is highly reproducible I managed to set a breakpoint in pthread_lock just prior to the read call that I know will fail (trying to see who else takes the lock) However what I observe is a large amount of buffer destructors beeing called like this:

#11 0xb7303da7 in calResFree () from /usr/lib/libaticalrt.so
#12 0xb7344c01 in CALBuffer::~CALBuffer () from /usr/lib/libbrook_cal.so
#13 0xb7337c21 in CALBufferMgr::_createPinnedBuffer () from /usr/lib/libbrook_cal.so
#14 0xb7338f07 in CALBufferMgr::setBufferData () from /usr/lib/libbrook_cal.so
#15 0xb7c97093 in StreamImpl::read () from /usr/lib/libbrook.so
#16 0xb7ca820c in brook::StreamData::read () from /usr/lib/libbrook.so
#17 0xb7cfa929 in brook::Stream<uint4>::read (this=0x9349368, ptr=0x935a300, flags=0xb7d0e8d8 "nocopy")
    at /usr/local/atibrook/sdk/include/brook/StreamDef.h:160
#18 0xb7cf819e in A5Slice::tick (this=0x90283c8) at A5Slice.cpp:369

This seems to indicate that the pinned buffers are accumulated in GFX memory and are only occasionally flushed. When this flusing occurs someone forgets to realease the mutex, and the next create call hangs indefinelty.

Where can I find the libbrook sources ? - I tried installing 1.4.1 but it fails on Ubuntu (on of the legacy samples has a dependancy on an old libpthread) - but the shared library is the same as that found in 1.4.0 (checked md5 sum)

Frank

2. When CAL_RESALLOC_GLOBAL_BUFFER makes sense when calResAllocRemote2D is used, as remote (system) memory normally not tiled?

3.Whether using CAL_RESALLOC_GLOBAL_BUFFER is the requirement for CS output buffers?

we've got a problem with some GPU (e.g. ATI HD 4800) types. We use the Stream Kernel Analyzer to translate c code to GPU assembler code. To track down the error we created a small kernel, which produces the error:

"il_ps_2_0\n"
"dcl_output_generic o0\n"
"dcl_resource_id(0)_type(2d,unnorm)_fmtx(float)_fmty(float)_fmtz(float)_fmtw(float)\n"
"dcl_input_position_interp(linear_noperspective) v0.xy__\n"
"sample_resource(0)_sampler(0) o0, v0.x0\n"
"endmain\n"

We found out that the sample_resource(0)_sampler(0) gets back wrong data for the range 1...0x7F, 0x7F80...0x807F and 0xFF7F...0xFFFF. This means we get back, e.g. 0 for 1, 0 for 0x7F, etc.

On some GPUs it works correct (e.g. HD3870).

Seems that on newer GPUs there is something wrong.

Any thoughts?

Regards,

Franz

I know that certain CUDA cards allow the device code to access the host memory, given the memory segment is pinned and mapped to the device memory space. I was wondering if AMD Stream also has a similar feature.

Thanks,

B

http://forums.amd.com/devforum/messageview.cfm?catid=328&threadid=99919

however, there are number of simple questions outstanding:

a)      it seems to be conclusive, that wavefronts are executing within the Thread Group. Thread Group size is defined by gridBlock.width parameter of CALprogramGrid structure. And number of Thread Groups are defined as domain execution size (in pixels) devided by Thread Group size.

b)      If Thread Group size is twice more then actual execution units (for 7xx # of execution units seems to be == 64), and set in kernel and in gridBlock.width, whether Thread Group will queue 2 wavefronts on the same SIMD still being within the same Thread Group without interruption?

c)       If fence_ work per Group, and Group Size is more then available execution units, and execution split on 2 wavefornts (case b above), whether first wavefront will be deferred until second wavefront will reach the barrier, to have first wavefront to be continued? Or it is just incorrect setting to have Group Size > then actual execution units per SIMD?

d)      If wavefornt size is set to ½ of executing units of SIMD, whether half of SIMD will be wasted or another Group will be started on half of SIMD?

e)      If there are more Groups set then available SIMDs, whether groups will be scheduled for execution one after another in some non-predictive order until finished?

f)        Once wavefront execution finished, whether LDS content remains persistent between wavefront runs, so next Thread Group will find LDS content from previous wavefront and can be reused?

Is this true?

I'm trying to install brookplus from svn on a recent fedora which uses gcc4.4, and am getting loads of errors. I've sorted out the header issues (ie. updated the #include's as described in http://gcc.gnu.org/gcc-4.3/porting_to.html), but have a "multiple definitions of 'getvalueof" which is bugging me.

Has anyone managed to compile with very recent gcc?

Thanks

Haven't seen one till now

1. To execute multiple images without need of reloading it into CAL: If multiple contexts will be created, then every context will have it's own image to execute. whether context switch will be noticible if one context will be started after another? Whether it is better solution then load module every time before start execution? (assuming only one image can be load per contexts, so to run the second IL image must be reloaded).

2. Whether the same resource can be attached to different contexts for i/o? (context1 for input, context2 for output)?

3. What is the example of running multiple functions in calCtxRunProgramGridArray if only one image can be loaded? If multiple ILs linked into the same image, what function names must be specified to run one after another, as sample says only about "main" as the only function to execute?

4. Micah, in one of the posts you mentioned to use some input paramter int kernel to trigger between different kernel function calls (code paths) from the "main", so all functionality will be linked together. In this case, whether non-executing paths will be scheduled (code branches which will not be executed) degrading performance?

Apologies if this should have been posted under another topic. I attempted to install the experimental driver in order to run the new version of the SDK. I am running an opensuse 11.1 AMD64 box. The driver installed and I got the warning images (eg No 3D, experimental driver..). Aside from the loss of the 3D acceleration, scrolling did not work. Using the scroll bar on, say firefox, caused both cores of my cpu to approach 100%. Furthermore, none of the examples would run. Has anyone encountered anything similar?

Thank you,

-Greg

Something like XML error is happen

... why is the 64x1 block size performance so horrid?

Compute shader might be faster but you really need to know how to get perfect texture fetch to make it so.

Accessing naively (64x1) gives HORRIBLE performance... WAY worse than pixel shader mode. And if LDS isn't any faster... I mean how many applications out there really need LDS?

How ever if you creat a scatter/gatther array as in float4 ar[1024] and fill it up in main memory. How do you create that array in graphical memory and how do you copy it?

I think that you can just use that array in the kernel call and  brook+ will under the hood do the copy.

What happens if you first use a gather/scatter array as output for kernel1 and then use that array as input for kernel2. Is there a copy to main memory performed?

Where/how do you declare that array: I do't need it in main memory.

Any help appreciated.

Sven

   After checking the IL code in the samples of CAL, i think it's a little hard to learn comparing to the highlevel interfaces.

  How could i get start with my journey? Do i have to learn ISA too? i think it's hard for me to understand everything about ISA.

http://www.hisdigital.com/un/product2-448.shtml

http://www.hisdigital.com/un/product2-444.shtml

For both types Stream support is claimed.

Is it possible to use both cards in one system and make use of the Stream Technology?

The domain size is 1024x1024.

So that should be 4 bytes/float, 4floats/input+output.

So 24 inputs+1 output = 25, 25*4*4 = 400 bytes total

Now, the domain size is 1024*1024, *400 = *1024*1024 = 419, 430, 400 bytes.

There is 1GB on the card so what's the problem? Am I missing something here?

The same kernel runs fine on the 4870 with Catalyst 9.4.

Note that I am currently running Catalyst 9.10.

I will try the 4870 too and edit this if needed.

Are there some special tricks in Brook+, which I have not used for this kernel?

kernel void
kernel_brook1(int nCha, int pSize, int gSize, int AF, int nPatLin, int nCol, int nLines, int halbpSize,  int firstToSkip, int oWidth, int iWidth, int wWidth, int nChapSize, int SkipLines,float2 dataIn[][], float2 WsI[][], out float2 dataOut<>{

    float2 res = float2(0.0f,0.0f);
    int2 pos = instance().xy;
    float2 w1,w2,w3,w4,x1,x2,x3,x4;
    int Y = pos.y / 4;
    int X = pos.y%4*oWidth + pos.x;//(pos.y - Y * 4)*oWidth + pos.x;
    int cntG = Y / gSize;
    int cntAF = Y - gSize * cntG;
    int cntCha = X / nCol;
    int cntP = X%nCol; //X - cntCha*nCol;
    int dataN = nChapSize; // number of source samples
    int Widx, Inputidx;
    int k = 0;

    //compute start index in weights matrix
    Widx = nChapSize *gSize * cntCha + nChapSize * cntAF;//vvvvv*******

    //compute start index in input matrix
    if(cntG >= firstToSkip)cntG = cntG + SkipLines;
    Inputidx = nCha * (cntG - halbpSize + 1);


    //scalar product
    while(k < dataN){
        w1 = WsI[cntP][Widx];
        Widx += 1;
        w2 = WsI[cntP][Widx];
        Widx += 1;
        w3 = WsI[cntP][Widx];
        Widx += 1;
        w4 = WsI[cntP][Widx];
        Widx += 1;
        x1 = dataIn[cntP][Inputidx];
        Inputidx += 1;
        x2 = dataIn[cntP][Inputidx];
        Inputidx += 1;
        x3 = dataIn[cntP][Inputidx];
        Inputidx += 1;
        x4 = dataIn[cntP][Inputidx];
        Inputidx += 1;
        res.y += w1.y * x1.x + w1.x * x1.y + w2.y * x2.x + w2.x * x2.y + w3.y * x3.x + w3.x * x3.y + w4.y * x4.x + w4.x * x4.y;
        res.x += w1.x * x1.x - w1.y * x1.y + w2.x * x2.x - w2.y * x2.y + w3.x * x3.x - w3.y * x3.y + w4.x * x4.x - w4.y * x4.y;
        k += 4;
    }

    dataOut =  res;

}

In an attempt to test a newer kernel for hardware updates I have compiled 2.6.31.4 and 2.6.32-rc4 kernels.  After a successful boot I have tried executing the stream app I am using and it Seg Faults on me.

I have also tested on openSuSE 11.2 kernel 2.6.31.3-5 and also receive the Seg Falut.

Are there kernel functions being missed during the compile to allow Stream 1.4 to operate correctly?

Thanks!

To the devs:

I'm participating in a distributed computing project hosted by www.distributed.net and running the ATI stream version of the client.

I ran this project succesfully under Ubuntu 9.04 with driver 9.10 installed.

I then upgraded to Ubuntu 9.10 and installed the same driver version (9.10) but the client dies horribly with a segfault.

I filed a bug report at distributed.net (http://bugs.distributed.net/show_bug.cgi?id=4260) about this.

My question to you is: What does calInit() do internally? Can you show some pseudo code about what calInit() does?

Best thing would be to have source code for calInit() to look at, but the source are closed I guess.

My current theory is that calInit() fails when trying to connect to a display (e.g. :0.0)  because of newer libraries in U9.10 versus U9.04, but I don't know which, so I'm hoping you could help me with this issue.

Thank you in advance.

Sceptic

i would use many HD5870 card under my RedHat linux 5.3 for GPGPU compute.

I have installed ati-opencl-beta-driver-v2.0-beta4-lnx and ati-stream-sdk-v2.0-beta4-lnx64.

i have 4  hd5870 and 1 monitor under each

i make a aticonfig --adapter=all --initial for configure it.
my Xorg.conf is attached

when i start my Xserver , i have the badge "AMD Testing Use  only" in right-bottom of each monitor

in the amdcccle, i have 4 screen

for the command :

aticonfig --list-adapters
* 0. 13:00.0 ATI Radeon HD 5800 Series
  1. 0c:00.0 ATI Radeon HD 5800 Series
  2. 8d:00.0 ATI Radeon HD 5800 Series
  3. 86:00.0 ATI Radeon HD 5800 Series

* - Default adapter

but when i run FindNumDevices, i have

$./FindNumDevices
Supported CAL Runtime Version: 1.3.185
Found CAL Runtime Version: 1.4.467
Use -? for help
CAL initialized.
Finding out number of devices :-
Device Count = 1
CAL shutdown successful.

Press enter to exit...

for lspci i have :

$ /sbin/lspci | grep VGA
0c:00.0 VGA compatible controller: ATI Technologies Inc Unknown device 6898
13:00.0 VGA compatible controller: ATI Technologies Inc Unknown device 6898
86:00.0 VGA compatible controller: ATI Technologies Inc Unknown device 6898
8d:00.0 VGA compatible controller: ATI Technologies Inc Unknown device 6898

...?

ANSWER : how use many hd5870 for compute with CAL??

Thank

I had to enable address virtualization for it to work at all, even for an in<> stream.  Why?  It is 1D.  Do all compute shaders need -r option?

But now the puzzling part ... domainSize has effect on 1st dimension, domainOffset has no effect.

For 2d version of kernel, that is "out_s[global_id.y][global_id.x]=" and appropriate stream defs, domainSize on 2nd dim has no effect.  Only first dim.  Again no effect from domainOffset.

The brookplus svn shows no tests for domainOffset, domainSize, and only exec_domain example, not for computer shaders, and its only mentioned once in docs ... so is it an "experimental feature" ... or user error ?

I'm having problems with a simple scatter kernel for which I want to restrict the domainSize.

Here is a simple kernel which does what I want :

kernel void kernel_one(unsigned int m, float4 a<>, out float b[][]){
  int k = instance().x;
  unsigned int t;
  for(t=0; t<m; t++){
    b[4*k+0][t] = a.x + (float)t;
    b[4*k+1][t] = a.y + (float)t;
    b[4*k+2][t] = a.z + (float)t;
    b[4*k+3][t] = a.w + (float)t;
  };
}

it takes an input stream a of size n/4 float4s, and put it in a big matrix of size n,m, such that column t contains ((vector a) + t).

Prior to calling the kernel, I always set the domainSize to n/4 (ie size of the input stream).

This works fine.

Now, if I add another dummy normal output stream float4 c<>, things get bad:

kernel void kernel_two(unsigned int m, float4 a<>, out float4 c<>, out float b[][]){
  int k = instance().x;
  unsigned int t;
  for(t=0; t<m; t++){
    b[4*k+0][t] = a.x + (float)t;
    b[4*k+1][t] = a.y + (float)t;
    b[4*k+2][t] = a.z + (float)t;
    b[4*k+3][t] = a.w + (float)t;
  };
  c = a;
}

(c does nothing but copying a)

now, for n = 256, m=5, the result is correct :

0   1   2   3   4

1   2   3   4   5

2   3   4   5   6

......

255 256 257 258 259

but for n = 260, m=5, it breaks down :

0   1   2   3   4

0   0   0   0   0

1   2   3   4   5

0   0   0   0   0

.......

129 130 131 132 133

0   0   0   0   0

??

In fact, the first kernel works fine on GPU, but when I switch to the CPU backend, I get a segfault, which gdb backtracks there:

#0  0x00007f111651f372 in brt::CPUKernel::Map () from /opt/atibrook/sdk/lib/libbrook.so

So, shall I conclude that having one scatter output + one normal output stream is implicitely not allowed?

Thanks for the help,

Guillaume

Explicit template specialization cannot have a storage class

Specializations of templates cannot explicitly specify a storage class, and have the same storage as the primary template. This is a change from previous behavior, based on the feedback and commentary as part of the ISO C++ Core Defect Report 605.

Simply removing static as shown in the attached patch works great for me:

I recently released an open source pricing library based on GPU computing. You may find it on my homepage at www.albanese.co.uk by following the link to OPLib. The library includes a set of low-level routines written in CUDA and in C to which one can reduce most valuation and risk management tasks. In OPLib I also give an orchestration example for Monte Carlo pricing.

With CUDA and a 4-GPU system with Teslas 1060 I achieve a sustained performance of 340 GF/sec per card, i.e. about 1.36 TF/sec of sustained performance on a calibration task. Calibration is a very flop consuming operation as it takes about 5 petaflops per risk factor, give or take a factor two. 340 GF/sec is excellent if one considers that peak performance for matrix multiplication of large matrices on Teslas 1060 is 370 GF/sec while I have rather small matrices of size 512 and in the sustained performance benchmark I mentioned I am counting all the high level orchestration stuff and lots of glue code that are needed for a real life implementation. This makes me hope that once the crucial routines are optimized, sustained performance on one of the latest ATI cards can reach 2 TF/sec per card. 

Achieving this depends on the ability to port a few routines which I released in the public domain in OPLib, namely:

(i) SGEMM4, a routine which operates on an array of pairs matrices and multiplies them concurrently.

(ii) SGEMV3, a routine that takes as an argument a matrix and an array of vectors stored non contiguously in memory and applies the matrix to those vectors.

(iv) SGEMV4, a routine that batches a number of SGEMV3 calls.

(v) SDOT2, a routine that batches a number of calls to SDOT while storing the dot products in an array in global GPU memory.

(vi) SCOPY2, a routine that batches a number of calls to SCOPY. 

The single precision variants of these routines are my first priority. I would also be interested in double precision variations of course, but that's of secondary important as this sort of algorithm is quite robust also in single precision, with errors typically well below the tolerance level. 

If anyone in this forum is interested in finance applications and can optimize handwritten IL code, I would be very grateful if he would contact me with advice or ideally consider contributing to OPLib. This could be a good topic for graduate students or anyone who would like exposure to the finance sector by writing a paper that I can assure would find a broad readership.

Regards, Claudio

email: claudio@albanese.co.uk 

I receive this error:

"rpm: please use alien to install rpm packages on Debian, if you are really sure use --force-debian switch. See README.Debian for more details."

But i can't find the *.rpm to install with alien...

Thanks

.. I am curious if and what cache improvements for CAL have been made in the drivers? Can we start getting a list of the improvements in the driver release notes for CAL?

However, it seems hard to realize the implementation 'cause where brook+ does well is mat-to-mat calculation. When it's not that case, in my situation especially, concerning with random access to different elements in an array(used to store the times for which different values appear in the set), i don't know what to do with it. But i believe this is still in the field of paralleling processing.

Calling for help

I'm trying to use my Radeon HD 4670 with DX11 SDK (Aug.). In an AMD-Presentaion I found in the web I read that the DX10.1 Cards support Raw- nad Structured-Buffers. But when I'm calling the CheckFeatureSupport()-Funktion the result is ComputeShaders_Plus_RawAndStructuredBuffers_Via_Shader_4_x = false.

Does anybody have the same problem? I don't know what to do to get it work.

I've tryed the newest Driver (9.10) an searched the web - without any success.

Please help! Frank