Fastest way to Convert Olevariant 2D Array of Double to Dynamic 2D Double Array


But I have discovered that the address diff between Pointer(DestArray[0]) and Pointer(DestArray[1]) is 56 bytes.

That is very much to be expected. Well, to be more clear, there is no reason at all to expect that DestArray[0] and DestArray[1] will point to adjacent blocks of memory.

Your type is

array[0..1] of array of Double;

Note that I removed the packed keyword which is ignored when applied to arrays. What you have here is an array containing two pointers. These two pointers are independent. Look at how you allocate the dynamic arrays.

SetLength(DestArray[0], Count);
SetLength(DestArray[1], Count);

Each call to SetLength results in a separate heap allocation. No reason at all for the memory to be adjacent. That's before getting to the issue that a dynamic array has an extra block of meta data stored immediately before the payload of the array, and each block of memory has its own meta data used by the memory manager. So even if the memory manager by chance happened to serve up adjacent blocks of memory, the meta data would sit between the two arrays. Incidentally, this memory manager meta data is the answer to your question 3.

In technical terms, what you have here in DestArray is a jagged array. You on the other hand appear to be looking for a multi-dimensional array. Delphi does not actually support dynamic multi-dimensional arrays. All you have are jagged arrays. If you want a contiguous block of memory then you would need to allocate a one dimensional block of memory and perform the index calculation yourself.

So, as it stands, if you continue with jagged arrays then you will need to perform one copy for each inner array. If you switch to a linear array then you can get away with a single copy, but you will have to perform your own indexing. Of course, the indexing is very easy to do and that might be efficient. Finally, it's plausible that you could allocate a linear array in Delphi ahead of time, and put a pointer to that array into your variant and thereby avoid the copy completely.

There're two 'factory' methods for dense Vectors:

def dense(values: Array[Double]): Vector
def dense(firstValue: Double, otherValues: Double*): Vector

While the provided type above is Array[Tuple2[Double,Double]] and hence does not type-match:
(Extracting the logic above:)

val parseLineToTuple: String => Array[(Double,Double)] = s => s=> s.split(',').map(fields => (fields(2).toDouble,fields(3).toDouble))

What is needed here is to create a new Array out of the input String, like this: (again focusing only on the specific parsing logic)

val parseLineToArray: String => Array[Double] = s=> s.split(",").flatMap(fields => Array(fields(2).toDouble,fields(3).toDouble)))

Integrating that in the original code should solve the issue:

val data = sc.textFile("/user/test/location/*")
val vectors = data.map(s => Vectors.dense(parseLineToArray(s))

(You can of course inline that code, I separated it here to focus on the issue at hand)

You need to do a few steps to achieve this

// Construct a temporary list.
List<Double> ell = new ArrayList<Double>();
// iterate our input array 'a'.
for (double d : a) {
  // test if the value 'd' is greater then the threshold 'x'.
  if (d > x) {
    // if it is, add it to the temporary list.
    ell.add(d);
  }
}
// construct an array of double(s).
double[] r = new double[ell.size()];
for (int i = 0; i < r.length; i++) {
  // Add the element(s) to the output array.
  r[i] = ell.get(i);
}
return r; // Return the Array.

It seems that your current culture has a different decimal separator.

You can specify the invariant culture for the parsing, which has a period as decimal separator:

num[0] = double.Parse(array[1], CultureInfo.InvariantCulture);

This:

double (*ope)(double, double);

should be:

double (evaluator::*ope)(double, double);
//      ^^^^^^^^^^^^

because you are using a pointer to member.

Comments


Message :
Login to Add Your Comments .
Privacy Policy - Copyrights Notice - Feedback - Report Violation - RSS 2017 © bighow.org All Rights Reserved .